[NTLUG:Discuss] VMWare server mirror strategy

[Daniel Hauck]

I'm seeking strategy advice.  Here's what I want to do:

I have two identical machines, each with 1TB (4 x 250MB) with SATA RAID 
controller.  Each RAID is configured the same with 4 volumes defined 
(all as RAID5) where 200MB is designated /boot, 2GB is designated 
{swap}, 10GB is designated / and 700+GB is designated "Tank."  I think 
the volumes are self explanatory.

The OS installed is CentOS 5.1 64 bit.  VMWare server 1.05 is installed 
as well.  A non-gui installation and running mode are active.

What I'd like to do is keep the "Tank" drives on each machine 
synchronized.  Ideally, this should be happening at all times.  The two 
machines will be running VMs and neither will be running both.  What I 
expect to see is a "cold failover" scenario where, if one machine is 
having problems or has failed, I can go to the working machine and turn 
up the remaining machines until I can get his twin brother back online.

The strategy that sounds most interesting to me is GFS + GNBD.  In this 
strategy, I would set up some sort of mirrored cluster between the two 
machines and then share them between each other.  But I have problems 
with this approach largely because I don't really seem to understand 
what I'm reading.  I get the general idea but the setup and 
configuration seems to be so difficult, I get dizzy just trying to read 
it.  Parts of what I read sounds like it could work and other parts seem 
to suggest that what I want to do with this will not.

It has been suggested that "rsync" should be used, but somehow I'm 
doubting rsync can be used against live, currently in use, files as 
would be the case with vmware server.  (If it can be done that way, I'd 
be really glad to learn this since it would seem to be the most simple 
solution.)  But my guess is that I'd have to set up a job to stop the 
VMs, then do the rsync, then start them up again.  Not the most ideal 
solution but certainly easier to wrap my head around.

Does anyone have anything to offer in this situation?  If GFS is a 
viable option, might someone offer some steps, tips or even assistance 
in getting it running?  I'm willing to pay some to make it happen if 
that's what it takes.  I worry that even if GFS is viable, it might suck 
too much processing power away from the VMs being hosted though.

I know I can't be the first person ever to want to do this.  I just 
can't seem to find much on the web about it.