The main purpose of this technical note is to demonstrate how to patch a running (live) system currently mirrored using SVM, minimizing the downtime as far as possible.
The idea is simple: detach one side of the mirror, apply the cluster
patch against it and reboot on it. If all seems OK, re-encapsulate the
system. This can achieve similar goal currently found in the Live
Upgrade feature of the Solaris OS (see live_upgrade(5)), with less
complexity and different requirement (LVM RAID-1 vs. spare disk, or free
slice).
Using this solution, the downtime can go between 10 to 30 minutes of service unavailability (depending on the hardware POST) and a maximum of two reboots are required, whatever is the number of patches to apply.
Here is a system encapsulated using SDS 4.x or SVM 1.x, and the
associated SVM encapsulation configuration:
# metastat -p
d3 -m d13 d23 1
d13 1 1 c0t0d0s3
d23 1 1 c0t1d0s3
d1 -m d11 d21 1
d11 1 1 c0t0d0s1
d21 1 1 c0t1d0s1
d0 -m d10 d20 1
d10 1 1 c0t0d0s0
d20 1 1 c0t1d0s0
#
# cat /etc/vfstab
#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options
#
fd - /dev/fd fd - no -
/proc - /proc proc - no -
/dev/md/dsk/d3 - - swap - no -
/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no -
/dev/md/dsk/d1 /dev/md/rdsk/d1 /var ufs 1 no -
swap - /tmp tmpfs - yes -
Run an explorer and generate a cluster patch, based on tools
provided by the OSE for example, if you are luckily enough to have one
included with your support plan (or just pick one provided at
SunSolve).
Then, be sure to be able to boot on the two disks, just in case:
# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t0d0s0
# installboot /usr/platform/`uname -i`/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0
The next step is to voluntarily detach one side of the mirror: take the
first one for the sake of simplicity (i.e. c0t0d0). Indeed, in this
case we are pretty sure that its alias name at the OBP is disk.
Note: You can always create it at the OBP (using the usual set of
commands, such as show-disks, devalias, etc.) if you want. That is
just a matter of personal preferences.
# lockfs -af /* Just to minimize the fs inconsistencies at next fsck(1m). */
#
# metadetach d0 d10
# metadetach d1 d11
# metadetach d3 d13
#
# metaclear d10
# metaclear d11
# metaclear d13
Check and repair the file systems if necessary, since we will boot on them the next time:
# fsck /dev/dsk/c0t0d0s0
# fsck /dev/dsk/c0t0d0s1
Next steps include mounting the recently detached file systems and
prepare the first disk to boot without SVM encapsulation:
# mkdir /mirror
# mount /dev/dsk/c0t0d0s0 /mirror
# mount /dev/dsk/c0t0d0s1 /mirror/var
#
# cat << EOF > /mirror/etc/vfstab
#device device mount FS fsck mount mount
#to mount to fsck point type pass at boot options
#
fd - /dev/fd fd - no -
/proc - /proc proc - no -
/dev/dsk/c0t0d0s3 - - swap - no -
/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no -
/dev/dsk/c0t0d0s1 /dev/rdsk/c0t0d0s1 /var ufs 1 no -
swap - /tmp tmpfs - yes -
EOF
#
# cp /mirror/etc/system /mirror/etc/system.orig
# sed -e 's;rootdev:/pseudo/md@0:0,0,blk;*rootdev:/pseudo/md@0:0,0,blk;' \
/mirror/etc/system.orig > /mirror/etc/system
Last, install patches against the first disk, clean things up a little and reboot if the install procedure went all smooth:
# ./install_all_patches -R /mirror
#
# umount /mirror/var
# umount /mirror
# rmdir /mirror
#
# shutdown -y -g 0 -i 6
After rebooting, carefully review the behavior of the very freshly patched system. If all seems well, don't forget to re-encapsulate the second disk. Here is a quick and easy way to this:
/* Recreate the metadb. */
# metadb -d c0t0d0s4 c0t1d0s4
# metadb -a -c3 -f c0t0d0s4 c0t1d0s4
#
/* Clean the system metadevices always present. */
# metaclear d0
# metaclear d1
# metaclear d3
# metaclear d20
# metaclear d21
# metaclear d23
#
/* Re-create them as part of a mirror. */
# metainit -f d10 1 1 c0t0d0s0
# metainit d0 -m d10
# metainit -f d11 1 1 c0t0d0s1
# metainit d1 -m d11
# metainit -f d13 1 1 c0t0d0s3
# metainit d3 -m d13
#
/* Be able to boot on the new metadevices. */
# metaroot d0
#
/* Reboot, and create the second side of the mirror. */
# shutdown -y -g 0 -i 6
[...]
# metainit d20 1 1 c0t1d0s0
# metattach d0 d20
# metainit d21 1 1 c0t1d0s1
# metattach d1 d21
# metainit d23 1 1 c0t1d0s3
# metattach d3 d23
For a little more detailed explanation about encapsulating the system using SVM on Sun Solaris, please refer to the dedicated entry in this blog.
Last, it must be mentioned that this documentation was written by our OSE, and that this procedure was officially marked as supported by Sun Microsystems.