Here, the final aim was to provide data access redundancy through SAN
storage hosted on remote sites across Wide Area Network (WAN) links.
After some relatively long and painful tries to mimic software mirroring
as found on HP-UX platform using Logical Volume Management (LVM), i.e.
at the logical volume
level, I finally give
up deciding this functionality will definitely not fit my need. Why?
Here are my comments.
- It is not possible to provide clear and manageable storage multipath
when the need to distinguish between the multiple sites is
important, ala mirror across controllers found on Veritas VxVM on
Sun Solaris system, for example. So, managing many physical volumes
along with lots of logical volumes is very complicated.
- There is no exact mapping capability between logical volume storage
on a given physical volume.
- The need to have a disk-based log, i.e. a persistent log. Yes, one
can always provide the option
--corelog at the creation time to
the logical volume initial build and have an in-memory log , i.e. a
non-persistent log, but this requires the entire copies (mirrors) be
resynchronized upon reboot. Not really viable on multi-TB
environments.
- A write-intensive workload on a file system living on a logical
volume mirror will suffer high latency: the overhead is important,
and the time to do mostly-write jobs grow dramatically. It is really
hard to get high level statistics, only low level metrics seems
consistent:
sd SCSI devices and dm- device mapper components for
each paths entries. Not from the multipath devices standpoint, which
is the more interesting from the end user and SA point of view.
- You can't extend a logical volume, which is really a no-go
per-se. On that point, the Red Hat support answered that this
functionality may be added in a future release, the current state
“may eventually be a Request For Enhancement (RFE), if a proper
business justification is provided”. One must break the logical
volume mirror copy, then rebuild it completely. Not realistic when
the logical volume is made of a lot of physical extents across
multiple physical volumes.
- A LVM configuration can be totally blocked by itself, and not usable
at all. The fact is, LVM use persistent storage blocks to keep track
of its own metadata. The metadata size is set at physical volume
creation time only, and can't be change afterward. This size is
statically defined as 255 physical volume blocks, and can be adjust
from the LVM configuration file. The problem is, when this circular
buffer space (stored in ASCII) fills up--such as when there are a
lot of logical volumes in a mirrored environment--it is not
possible to do anything more with LVM. So you can't add more
logical volume, can't add more logical volume copies,... and
can't delete them trying to reestablish a proper LVM configuration.
Well, here are the answers given by the Red Hat support to two keys
questions in this situation:
- How to size the metadata, i.e. if we need to change it from the
default value, how can we determine the new proper and
appropriate size, and from which information?
“I am afraid but Metadata size can only be defined at the time
of PV creation and there is no real formula for calculating the
size in advance. By changing the default value of 255 you can
get a higher space value. For general LVM setup (with less LV's
and VG's) default size works fine however in cases where high
number of LV's are required a custom value will be required.”
- We just want to delete all LV copies, which means to return to
the initial situation and have 0 copy for all LV, i.e. only one
LV per-se, in order to be able to change LVM configuration again
(we can't do anything on our production server right now)?
“I discussed this situation with peer engineers and also
referenced a similar previous case. From the notes of the same
the workaround is to use the backup file (/etc/lvm/backup) and
restore the PV's. I agree that this really not a production
environment method however seems the only workaround.”
So, the production RDBMS Oracle server is finally now being evacuate to
an other machine. Hum... Hope to see better enterprise experience using
the mdadm package to handle RAID software, instead of mirror (RAID-1)
LVM. Maybe more about that in an other blog entry?