blog'o thnet

The last update to the excellent memconf utility (V2.5 22-Feb-2008) support properly recent Solaris Express releases, and my recent change from the stock AMD Opteron Processor 148 to an AMD Athlon 64 X2 Dual Core Processor 3800+. (I mostly did that change just to be able to access two run queues separately, not to gain more power per se.)

So, here is the new and appropriate memconf report:

# memconf -d
memconf:  V2.5 22-Feb-2008 http://www.4schmidts.com/unix.html
hostname: unic
manufacturer: Sun Microsystems, Inc.
model:    Sun Ultra 20 Workstation (AMD Athlon(tm) 64 X2 Dual Core \
 Processor 3800+ Socket 939 2010MHz)
Sun Family Part Number: A63
Solaris Express Community Edition snv_87 X86, 64-bit kernel, SunOS 5.11
1 AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ Socket 939 2010MHz cpu
diagbanner = Sun Ultra 20 Workstation
cpubanner = AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ Socket 939 2010MHz
model = Sun Ultra 20 Workstation
machine = i86pc
platform = i86pc
perl version: 5.008004
CPU Units:
==== Processor Sockets ====================================
Version                          Location Tag
-------------------------------- --------------------------
AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ Socket 939
AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ Socket 939
Memory Units:
Type    Status Set Device Locator      Bank Locator
------- ------ --- ------------------- --------------------
unknown in use 0   A0                  Bank0/1
unknown in use 0   A1                  Bank2/3
unknown in use 0   A2                  Bank4/5
unknown in use 0   A3                  Bank6/7
total memory = 2048MB (2GB)

You can check and compare with the previous report on my blog.

In a previous discussion about the GRUB boot archive and how it can be regenerated, I mentioned that it will not be as easy as it can be when the root file system use the md driver. I will now show two different methods to do the same thing when the root file system is build upon a SVM mirror (RAID-1):

1. Unmirror the root file system only.
2. Unmirror the entire system, i.e. all devices.

Note: Although this test case was done using Solaris 10 8/07 under a virtual machine build upon VirtualBox on latest Solaris Express Community Edition, the instructions must be valid for Solaris 10 1/06 and later.

Initial setup

As we can see, the system use only a root file system, and a swap device. Both are encapsulated with SVM.

# df -k -F ufs
Filesystem     kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d0 6147798 3455578 2630743      57%  /
# swap -l
swapfile             dev  swaplo blocks   free
/dev/md/dsk/d1      85,1       8 4194288 4194288
# metastat -c d0 d1
d0               m  6.0GB d10 d20
    d10          s  6.0GB c0d0s0
    d20          s  6.0GB c1d1s0
d1               m  2.0GB d11 d21
    d11          s  2.0GB c0d0s1
    d21          s  2.0GB c1d1s1

Unmirror the root file system only

The idea is to boot on the GRUB failsafe mode, select the first side of the mirror, and modify the system and vfstab configuration files to use the correct device path. For the system file, this means to actually remove the rootdev:/pseudo/md@0:0,0,bl entry, not just comment it. For the vfstab file, this means replacing the root file system metadevice path /dev/md/[r]dsk/d0 by the first underlying device path, i.e. /dev/[r]dsk/c0d0s0. Last, regenerate the boot archive on the alternate root path.

[...]
Booting to milestone "milestone/single-user:default".
Configuring devices.
Searching for installed OS instances...
/dev/dsk/c0d0s0 is under md control, skipping.
/dev/dsk/c1d1s0 is under md control, skipping.
No installed OS instance found.

Starting shell.
# fsck /dev/rdsk/c0d0s0
# mount -F ufs /dev/dsk/c0d0s0 /a
# cp /a/etc/system /a/etc/system.bckp
# cp /a/etc/vfstab /a/etc/vfstab.bckp
# TERM=vt100 vi /a/etc/system
# TERM=vt100 vi /a/etc/vfstab
# bootadm update-archive -R /a
# umount /a
# fsck /dev/rdsk/c0d0s0
# reboot

Then, boot into milestone/multi-user:default level and detach the second half of the mirror, since the first half correspond to the valid and updated underlying device. Next, restore the original configuration files which refers to the encapsulated metadevices, and reboot.

# df -k -F ufs
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0d0s0      6147798 3458810 2627511    57%    /
# swap -l
swapfile             dev  swaplo blocks   free
/dev/md/dsk/d1      85,1       8 4194288 4194288
# metastat -c d0
d0               m  6.0GB d10 d20
    d10          s  6.0GB c0d0s0
    d20          s  6.0GB c1d1s0
# metadetach d0 d20
d0: submirror d20 is detached
# metastat -c d0
d0               m  6.0GB d10
    d10          s  6.0GB c0d0s0
# cp /etc/system.orig /etc/system
# cp /etc/vfstab.orig /etc/vfstab
# shutdown -y -i 6 -g 0

After the reboot, just reattach the second half of the mirror, and wait for complete synchronization to be fully redundant again.

# df -k -F ufs
Filesystem            kbytes    used   avail capacity  Mounted on
/dev/md/dsk/d0       6147798 3458714 2627607    57%    /
# swap -l
swapfile             dev  swaplo blocks   free
/dev/md/dsk/d1      85,1       8 4194288 4194288
# metattach d0 d20
d0: submirror d20 is attached
# metastat -c d0
d0               m  6.0GB d10 d20 (resync-29%)
    d10          s  6.0GB c0d0s0
    d20          s  6.0GB c1d1s0

Unmirror the entire system, i.e. all devices

The idea is exactly the same as for unmirroring the root file system only, but adapting the vfstab file to change the swap entry, too. (So, I didn't reproduce the code listing here.)

Then, boot into milestone/single-user:default level modifying the corresponding GRUB entry as follow: kernel /platform/i86pc/multiboot -s. Completely delete all the metadevices and metadb configurations to clear SVM settings. Last, continue into milestone/multi-user:default level to boot unmirrored.

# metaclear -f -r d0 d1
# metadb -f -d  c1d0s4 c1d0s4
# ^D

Now, the system must be fully encapsulate by SVM again. Please refer to online Sun Documentation, or some past entries on this subject, depending on the system's architecture: SPARC systems, or x86 platforms.

You can easily create and remove a remote printer queue using the BSD type spooler. You just have to fill the configuration file /tmp/lp.list properly, i.e. provide the local printer name, the remote LPD server, and the remote printer queue:

# cat << EOF > /tmp/lp.list
locname1 lpdserv1 remname1
locname2 lpdserv2 remname2
EOF

Then, just run the appropriate script depending of the desired behavior. Follow, an example when removing the two queues:

# cat << EOF > /tmp/lp.remove
#!/usr/bin/env sh

for lplocal in `awk '{print $1}' /tmp/lp.list`; do
  /usr/sbin/lpshut
  /usr/bin/cancel ${lplocal} -e 2> /dev/null
  /usr/sbin/lpadmin -x${lplocal}
  /usr/sbin/lpsched -v
  sleep 1
done

exit 0
EOF
# sh /tmp/lp.remove
scheduler stopped
scheduler is running
scheduler stopped
scheduler is running
# lpstat -olocname1
no system default destination
lpstat: "locname1" not a request id or a destination

And now, the creation:

# cat << EOF > /tmp/lp.create
#!/usr/bin/env sh

while read lp; do
  eval set -- `IFS=" "; printf '"%s" ' ${lp}`
  lplocal="$1"
  lpserver="$2"
  lpremote="$3"

  /usr/sbin/lpshut
  /usr/sbin/lpadmin -p${lplocal} -orm${lpserver} -orp${lpremote} \
   -mrmodel -v/dev/null -orc -ob3 -ocmrcmodel -osmrsmodel
  /usr/sbin/accept ${lplocal}
  /usr/bin/enable ${lplocal}
  /usr/sbin/lpsched -v
  sleep 1
done < /tmp/lp.list

exit 0
EOF
# sh /tmp/lp.create
scheduler stopped
destination "locname1" now accepting requests
printer "locname1" now enabled
scheduler is running
scheduler stopped
destination "locname2" now accepting requests
printer "locname2" now enabled
scheduler is running
# lpstat -olocname1
no system default destination
printer queue for locname1
                         Windows LPD Server
                   Printer \\lpdserv1
emname1
Owner       Status         Jobname          Job-Id    Size   Pages  Priority
----------------------------------------------------------------------------
hostname: locname1: ready and waiting
no entries

That's it!

When releasing a multipath device under RHEL4, different SCSI devices corresponding to different paths must be cleared properly before removing the SAN LUN effectively. When the LUN was delete before to clean up the paths at the OS level, it is always possible to remove them afterwards. In the following example, it is assume that the freeing LVM manipulations were already done, and that the LUN is managed by EMC PowerPath.

1. First, get and verify the SCSI devices corresponding to the multipath LUN:

   # grep "I/O error on device" /var/log/messages | tail -2
   Feb  4 00:20:47 beastie kernel: Buffer I/O error on device sdo, \
    logical block 12960479
   Feb  4 00:20:47 beastie kernel: Buffer I/O error on device sdp, \
    logical block 12960479
   # powermt display dev=sdo
   Bad dev value sdo, or not under Powerpath control.
   # powermt display dev=sdp
   Bad dev value sdp, or not under Powerpath control.
   

2. Then, get the appropriate scsi#:channel#:id#:lun# informations:

   # find /sys/devices -name "*block" -print | \
    xargs \ls -l | awk -F\/ '$NF ~ /sdo$/ || $NF ~ /sdp$/ \
    {print "HBA: "$7"\tscsi#:channel#:id#:lun#: "$9}'
   HBA: host0      scsi#:channel#:id#:lun#: 0:0:0:9
   HBA: host0      scsi#:channel#:id#:lun#: 0:0:1:9
   

3. When the individual SCSI paths are known, remove them from the system:

   # echo 1 > /sys/bus/scsi/devices/0\:0\:0\:9/delete
   # echo 1 > /sys/bus/scsi/devices/0\:0\:1\:9/delete
   # dmesg | grep "Synchronizing SCSI cache"
   Synchronizing SCSI cache for disk sdp:
   Synchronizing SCSI cache for disk sdo: