Here, the final aim was to provide data access redundancy through SAN storage hosted on remote sites across Wide Area Network (WAN) links. After some relatively long and painful tries to mimic software mirroring as found on HP-UX platform using Logical Volume Management (LVM), i.e. at the logical volume level, I finally give up deciding this functionality will definitely not fi my need. Why? Here are my comments.
- It is not possible to provide clear and manageable storage multipath when the need to distinguish between the multiple sites is important, ala mirror across controllers found on Veritas VxVM on Sun Solaris system, for example. So, managing many physical volumes along with lots of logical volumes is very complicated.
- There is no exact mapping capability between logical volume storage on a given physical volume.
- The need to have a disk-based log, i.e. a persistent log. Yes, one can
always provide the option
--corelogat the creation time to the logical volume initial build and have an in-memory log , i.e. a non-persistent log, but this requires the entire copies (mirrors) be resynchronized upon reboot. Not really viable on multi-TB environments. - A write-intensive workload on a file system living on a logical volume
mirror will suffer high latency: the overhead is important, and the time to do
mostly-write jobs grow dramatically. It is really hard to get high level
statistics, only low level metrics seems consistent:
sdSCSI devices anddm-device mapper components for each paths entries. Not from the multipath devices standpoint, which is the more interesting from the end user and SA point of view. - You can't extend a logical volume, which is really a no-go
per-se. On that point, the Red Hat support answered that this functionality may
be added in a future release, the current state
may eventually be a Request For Enhancement (RFE), if a proper business justification is provided
. One must break the logical volume mirror copy, then rebuild it completely. Not realistic when the logical volume is made of a lot of physical extents across multiple physical volumes. - A LVM configuration can be totally blocked by itself, and not usable at
all. The fact is, LVM use persistent storage blocks to keep track of its own
metadata. The metadata size is set at physical volume creation time only, and
can't be change afterward. This size is statically defined as 255 physical
volume blocks, and can be adjust from the LVM configuration file. The problem
is, when this circular buffer space (stored in ASCII) fills up--such as when
there are a lot of logical volumes in a mirrored environment--it is not
possible to do anything more with LVM. So you can't add more
logical volume, can't add more logical volume copies,... and can't delete them
trying to reestablish a proper LVM configuration. Well, here are the answers
given by the Red Hat support to two keys questions in this situation:
- How to size the metadata, i.e. if we need to change it from the default
value, how can we determine the new proper and appropriate size, and from which
information?
I am afraid but Metadata size can only be defined at the time of PV creation and there is no real formula for calculating the size in advance. By changing the default value of 255 you can get a higher space value. For general LVM setup (with less LV's and VG's) default size works fine however in cases where high number of LV's are required a custom value will be required.
- We just want to delete all LV copies, which means to return to the initial
situation and have 0 copy for all LV, i.e. only one LV per-se, in order to be
able to change LVM configuration again (we can't do anything on our production
server right now)?
I discussed this situation with peer engineers and also referenced a similar previous case. From the notes of the same the workaround is to use the backup file (/etc/lvm/backup) and restore the PV's. I agree that this really not a production environment method however seems the only workaround.
- How to size the metadata, i.e. if we need to change it from the default
value, how can we determine the new proper and appropriate size, and from which
information?
So, the production RDBMS Oracle server is finally now being evacuate to an
other machine. Hum... Hope to see better enterprise experience using the
mdadm package to handle RAID software, instead of mirror (RAID-1)
LVM. Maybe more about that in an other blog entry?
