[NTLUG:Discuss] Software RAID under 2.6 kernel -- mirroring disks v. partitions (MD only does latter)

[Bryan J. Smith]

Ralph Green, Jr. wrote:
> Howdy,
> I asked about this a couple of weeks ago.  I tried it out and I
> thought I would report back.  My test system was a Intel 2.4GHz
> Celeron on an 845GVSR motherboard.  I installed two 10 GIG Maxtor
> drives and started up the install of SuSE 9.1.  I told SuSE to create
> a RAID 1 set using 4 GIG on each of the drives formatted as /.  The
> install went fine.  I rebooted, and it came up fine.  I skipped
> creating a boot floppy, which was a small risk, because the
> instructions say some systems won't boot the RAID set.  I applied all
> updates available, then powered off and unplugged the slave drive.  I
> powered on and was disappointed to have the system refuse to boot. 
> Mirrored drives that won't boot after one drive fails are kind of
> silly, so it is on the the next step.

Remember.  When using the volume management of any OS, you are _only_
mirroring _volumes_ (partitions), not _drives_.  That means the MBR and
bootstrap are _not_ being replicated.  You have to do that manually (I
used to with a modified LILO script).

You have this same issue in Linux, NT, etc...

Furthermore, there is always the issue of being _unable_ to store
special RAID meta-data in the legacy DOS Disk Label (primary/extended)
partitioning scheme.  That's why it is recommended that you use a more
advanced Disk Label for the OS when you use Software RAID.

For Linux, this is Logical Volume Management (LVM, 2.4) and LVM2 (2.6),
which is optional, but I highly recommend it.  LVM/LVM2 come with RAID-0
and spanning/appending built-in.  For RAID-1, RAID-10, RAID-5 and other
variations, you use MD atop of LVM partitions.  It is better to do MD
atop of LVM, than just MD with legacy DOS partitions, because LVM stores
extra meta-data.

For NT 5.x (2000, XP, 2003, onward), this is LDM (Logical Disk Manager)
aka "Dynamic Disks" and is now _required_ for any software RAID.  Those
of us who ran NT 4.x and use software RAID got bit the in @$$ because of
the lack of being able to store meta-data in the legacy DOS disk label
format (in other words, "Dynamic Disks" are not an anti-Linux Microsoft
thing).

[ SIDE NOTE:  LDM has been reverse engineered for years now, and the
Linux kernel has good support for it (part of the Linux-NTFS project --
but LDM is 100% _separate_ from NTFS, and not dependent on it).  I just
sure wish there were user-space tools to modify LDM, and GRUB supported
booting into it (LILO does, because LILO is a simplistic/static
sector-target boot-loader).  Especially since it looks like LDM will be
_required_ for NT 6.0 "Longhorn" and the legacy DOS Disk Label will no
longer be supported. ]

> I am going to install a Comact Flash to IDE adapter with a 64 meg CD
> card as the primary master.  I will put the /boot filesystem there. 
> Everything else will stay on the RAID 1 set on the 2 10 gig drives. 

Read-only solid state is always best, because it's the most reliable (as
long as you don't write to it too much).

Either that, or consider a real ATA hardware RAID solution like the
64-bit ASIC+SRAM 3Ware Escalade "Non-blocking I/O Storage Switch"
series, or a more traditional "Buffering Storage Host Adapter" 32-bit
microcontroller+DRAM solution in the Adaptec 2400A or 2800A.  They start
with the 2-channel, SerialATA 0.5GBps PCI (64-bit/66MHz) 3Ware Escalade
8006-2 with 2MB SRAM (0 wait state, like CPU cache) card is around $100,
and go upto to some of the 12-channel 3Ware Escalade 9500S-12 with both
4MB of SRAM (0 wait state, like CPU cache, best for RAID-0/1/10) plus
128MB of SDRAM (like traditional I/O buffering cards, best for RAID-5).

If you get "Sys Admin" magazine, check out the April article "Dissecting
ATA RAID Options."  It discusses the various options for ATA RAID:  
  http://www.sysadminmag.com/articles/2004/0404/  

> I'll let you know how that works.
> p.s.  I would hope it went without saying that if you saw something
> wrong with what I did, I invite you to tell me.  That was my hope, but I
> decided to be explicit and state it instead.

You did a great job.

As I pointed out, when you mirror a partition like /, you're still not
mirroring the MBR or bootstrap.  That's why software RAID is best for
non-booting volumes.

I sure wish someone would modify LVM/LVM2 to do RAID-1 natively (it only
does RAID-0 and spanning).  That way it could manage the MBR, bootstrap,
etc...  MD doesn't.


-- 
     Linux Enthusiasts call me anti-Linux.
   Windows Enthusisats call me anti-Microsoft.
 They both must be correct because I have over a
decade of experience with both in mission critical
environments, resulting in a bigotry dedicated to
 mitigating risk and focusing on technologies ...
           not products or vendors
--------------------------------------------------
Bryan J. Smith, E.I.            b.j.smith at ieee.org