blog'o thnet

You can easily create and remove a remote printer queue using the BSD type spooler. You just have to fill the configuration file /tmp/lp.list properly, i.e. provide the local printer name, the remote LPD server, and the remote printer queue:

# cat << EOF > /tmp/lp.list
locname1 lpdserv1 remname1
locname2 lpdserv2 remname2
EOF

Then, just run the appropriate script depending of the desired behavior. Follow, an example when removing the two queues:

# cat << EOF > /tmp/lp.remove
#!/usr/bin/env sh

for lplocal in `awk '{print $1}' /tmp/lp.list`; do
  /usr/sbin/lpshut
  /usr/bin/cancel ${lplocal} -e 2> /dev/null
  /usr/sbin/lpadmin -x${lplocal}
  /usr/sbin/lpsched -v
  sleep 1
done

exit 0
EOF
# sh /tmp/lp.remove
scheduler stopped
scheduler is running
scheduler stopped
scheduler is running
# lpstat -olocname1
no system default destination
lpstat: "locname1" not a request id or a destination

And now, the creation:

# cat << EOF > /tmp/lp.create
#!/usr/bin/env sh

while read lp; do
  eval set -- `IFS=" "; printf '"%s" ' ${lp}`
  lplocal="$1"
  lpserver="$2"
  lpremote="$3"

  /usr/sbin/lpshut
  /usr/sbin/lpadmin -p${lplocal} -orm${lpserver} -orp${lpremote} \
   -mrmodel -v/dev/null -orc -ob3 -ocmrcmodel -osmrsmodel
  /usr/sbin/accept ${lplocal}
  /usr/bin/enable ${lplocal}
  /usr/sbin/lpsched -v
  sleep 1
done < /tmp/lp.list

exit 0
EOF
# sh /tmp/lp.create
scheduler stopped
destination "locname1" now accepting requests
printer "locname1" now enabled
scheduler is running
scheduler stopped
destination "locname2" now accepting requests
printer "locname2" now enabled
scheduler is running
# lpstat -olocname1
no system default destination
printer queue for locname1
                         Windows LPD Server
                   Printer \\lpdserv1
emname1
Owner       Status         Jobname          Job-Id    Size   Pages  Priority
----------------------------------------------------------------------------
hostname: locname1: ready and waiting
no entries

That's it!

When releasing a multipath device under RHEL4, different SCSI devices corresponding to different paths must be cleared properly before removing the SAN LUN effectively. When the LUN was delete before to clean up the paths at the OS level, it is always possible to remove them afterwards. In the following example, it is assume that the freeing LVM manipulations were already done, and that the LUN is managed by EMC PowerPath.

1. First, get and verify the SCSI devices corresponding to the multipath LUN:

   # grep "I/O error on device" /var/log/messages | tail -2
   Feb  4 00:20:47 beastie kernel: Buffer I/O error on device sdo, \
    logical block 12960479
   Feb  4 00:20:47 beastie kernel: Buffer I/O error on device sdp, \
    logical block 12960479
   # powermt display dev=sdo
   Bad dev value sdo, or not under Powerpath control.
   # powermt display dev=sdp
   Bad dev value sdp, or not under Powerpath control.
   

2. Then, get the appropriate scsi#:channel#:id#:lun# informations:

   # find /sys/devices -name "*block" -print | \
    xargs \ls -l | awk -F\/ '$NF ~ /sdo$/ || $NF ~ /sdp$/ \
    {print "HBA: "$7"\tscsi#:channel#:id#:lun#: "$9}'
   HBA: host0      scsi#:channel#:id#:lun#: 0:0:0:9
   HBA: host0      scsi#:channel#:id#:lun#: 0:0:1:9
   

3. When the individual SCSI paths are known, remove them from the system:

   # echo 1 > /sys/bus/scsi/devices/0\:0\:0\:9/delete
   # echo 1 > /sys/bus/scsi/devices/0\:0\:1\:9/delete
   # dmesg | grep "Synchronizing SCSI cache"
   Synchronizing SCSI cache for disk sdp:
   Synchronizing SCSI cache for disk sdo:
   

Here is a little but annoying bug I faced recently on a production system running Solaris 10 11/06: a process can not be killed.

The KILL signal can't be ignored by a process. The associated handler must be the default--taking the signal into account, and honor it--and it is the case (the incriminated process id is 26632):

# psig 26632 | grep KILL
KILL    default

Nevertheless, the process seems not to stop as expected:

# echo $$
2913
#
# kill -s KILL 26632
#
# dtrace -n 'fbt::sigtoproc:entry /pid == 2913/ {trace(arg2); trace(execname);}'
dtrace: description 'fbt::sigtoproc:entry ' matched 1 probe
CPU     ID                    FUNCTION:NAME
 18  16387                  sigtoproc:entry                 9  tcsh
#
# ps -ef | awk '$2 ~ /26632/ {print $0}'
nonpriv 26632     1   0   Nov 13 ?         0:01 /usr/sbin/in.ftpd -a

This not the correct behavior since a pending signal is not honored:

# pflags 26632
26632:  /usr/sbin/in.ftpd -a
       data model = _ILP32  flags = ORPHAN|MSACCT|MSFORK
       sigpend = 0x00006100,0x00000000
/1:    flags = 0

More, the process seems in a non-coherent system state: it is now nearly impossible to trace it using, truss, the proctools,...

# truss -alef -p 26632
truss: unanticipated system error: 26632
#
# pstack 26632
pstack: cannot examine 26632: unanticipated system error
#
# pfiles 26632
pfiles: unanticipated system error: 26632
#
# pldd 26632
pldd: cannot examine 26632: unanticipated system error

... or helped by DTace:

# dtrace -n 'profile-1000hz /pid ==26632/ { @[stack()] = count() }'
dtrace: description 'profile-1000hz ' matched 1 probe
^C
#
# dtrace -n 'profile-1000hz /pid ==26632/ { @[ustack()] = count() }'
dtrace: description 'profile-1000hz ' matched 1 probe
^C

Follow some information gathered through mdb in kernel debugging mode:

> ::status
debugging live kernel (64-bit) on socrate
operating system: 5.10 Generic_118833-36 (sun4u)
>
> ::showrev
Hostname: socrate
Release: 5.10
Kernel architecture: sun4u
Application architecture: sparcv9
Kernel version: SunOS 5.10 sun4u Generic_118833-36
Platform: SUNW,Sun-Fire-V490
>
> ::pgrep in.ftpd
S    PID   PPID   PGID    SID    UID      FLAGS             ADDR NAME
R  26632      1    261    261  30501 0x5a024b00 00000600144b0c28 in.ftpd
>
> 00000600144b0c28::kill
mdb: command is not supported by current target
>
> 00000600144b0c28::thread
            ADDR    STATE    FLG PFLG SFLG   PRI  EPRI PIL             INTR
00000600144b0c28 inval/2000 157e 5778    0     0     0   6                2
>
> 00000600144b0c28::walk thread | ::findstack
stack pointer for thread 30002aba360: 2a1022e7ec1
[ 000002a1022e7ec1 cv_wait+0x38() ]
  000002a1022e7f71 page_lock_es+0x204()
  000002a1022e8021 pvn_vplist_dirty+0x2a4()
  000002a1022e8101 nfs_putpages+0x124()
  000002a1022e81c1 nfs3_putpage+0xcc()
  000002a1022e8271 fop_putpage+0x1c()
  000002a1022e8321 nfs_purge_caches+0xe4()
  000002a1022e83d1 nfs_attr_cache+0x20c()
  000002a1022e8481 nfs3_getattr_otw+0x1b8()
  000002a1022e85f1 nfs3_validate_caches+0x4c()
  000002a1022e8731 nfs3_getpage+0xa4()
  000002a1022e8861 fop_getpage+0x44()
  000002a1022e8931 segmap_getmapflt+0x588()
  000002a1022e8a41 snf_segmap+0x13c()
  000002a1022e8bc1 sosendfile64+0x298()
  000002a1022e8d21 sendvec64+0xf8()
  000002a1022e8f61 sendfilev+0x178()
  000002a1022e92e1 syscall_trap32+0xcc()

We can see the use of the sendfilev() syscall: it appears to be a known bug around this (you must have a registered Sun support customer account to be able to view the second document).

The good news is the root cause is already fix in the development branch of the operating system. The bad news is, after opening a call to the Sun support team, that no patch will be released in a near future to correct this: the incorporation of the fix is currently planned for the next major Solaris 10 Update (Update 5) which is scheduled for summer 2008. However, the fix is already available via the Solaris Express program, if that is applicable to your environment. As a last note, and if the non-killable process owns a resource necessary to another program, there seems no other option than to plan a reboot of the system.

After creating a new boot environment (BE) named beastie, see below, to upgrade a system running Solaris 8 to Solaris 10 11/06 (as I do many times in the past without a hiccup), I encounter a problem when I tried to apply the appropriate Recommended cluster patch to the new BE with this message:

# luupgrade -t -n beastie -s /var/tmp/10_Recommended

Validating the contents of the media .
The media contains 76 software patches that can be added.
All 76 patches will be added because you did not specify any specific
patches to add.
Mounting the BE .
ERROR: The boot environment  supports non-global
zones.The current boot environment does not support non-global zones.
Releases prior to Solaris 10 cannot be used to maintain Solaris 10 and
later releases that include support for non-global zones. You may only
execute the specified operation on a system with Solaris 10 (or later)
installed.

I can't find any reference to a known bug or problem after looking for this against SunSolve, Sun Support, and Googling. Has anyone already seen this error, and solved it The Right Way? As for me, I needed to boot from the BE and apply the cluster patch: this was a pain since this bundle include the -36 kernel patch which is known to be relatively disruptive, since it need two reboot to apply the entire cluster patch (it contains a new version for the kernel).

Update #1 (2009-03-22): Seems to be explained in this excellent BigAdmin article.

Once again, well known and great developer Casper Dik from Sun Microsystems give us an interesting answer on the fact that there is not 32-bit emulation in the Solaris OS when running a 64-bit kernel, and why most installed binaries seems to be compiled as 32-bit executables, even if you boot on a 64-bit platform, as in the following example:

# isainfo -kv
64-bit amd64 kernel modules
# file /usr/bin/ls
/usr/bin/ls:    ELF 32-bit LSB executable 80386 Version 1 [FPU], dynamically linked, \
 not stripped, no debugging information available

Here is the explanation, shown as a little Q&A:

Q: Once I boot it in 64bit mode, i'd have to run emulation libraries to run 32bit bins right?

A: No; you run the exact same binaries and libraries under 32 and 64 bit. It's not emulation; it's basically two syscall entry tables one for 32 bit and one for 64 bit, mostly sharing the same code except where pointer sizes matter.

Q: Since it will be a very light server (only bare binaries to run my req.s) they probably end up most likely all being 64bit?

A: No, most binaries are 32 bit only those that need to be 64 bit are 64 bit.

Q: so you are saying that i can run both 32bit and 64bit code simultaneously, natively with the solaris kernel? That's pretty damn cool

A: Correct. There are several reasons for an OS which generally comes in binary distributions to do so:

• maintain complete binary compatibility with old applications (and yes, closed source does matter, even to a lot of Linux customers).
• allows a single distribution to work on both 32 and 64 bit systems of the same architecture.

As a last note, it is interesting to see that a full install of SXCE snv_70 provide 1188 32-bit files and only 82 64-bit files in the following paths /bin, /usr/bin, /usr/openwin/bin, /usr/ucb, /usr/sfw/bin, and /opt/SUNWspro/bin. The 64-bit files are mostly stored in a specific architecture subdirectory; i.e. amd64 in this case, while some of them are provided as both 32-bit and 64-bit incarnations:

# file /usr/bin/pfiles /usr/bin/amd64/pfiles
/usr/bin/pfiles:        ELF 32-bit LSB executable 80386 Version 1 [FPU], \
 dynamically linked, not stripped, no debugging information available
/usr/bin/amd64/pfiles:  ELF 64-bit LSB executable AMD64 Version 1 [SSE FXSR FPU], \
 dynamically linked, not stripped, no debugging information available

Update #1: 2008-04-07

Be sure to consult the excellent 64-bit FAQ for Solaris.