[NTLUG:Discuss] NFS Synchronization

[Robert Pearson]

On Thu, May 20, 2010 at 4:08 PM, George Lass <george.lass at sbcglobal.net> wrote:
> I need to set up a pair of NFS servers that need to be synchronized and am wondering if rsync is the right way to do it?  The primary use of the pair of servers is for recording and playback of phone messages stored in .wav files.  The only "gotcha" is that any particular recording will originally be written (recorded) randomly to one of the servers or the other, while read (playback) requests can be made to either server.  Thus within just a minute or 2, the file must be propagated from one server to the other.  It should be assumed that the directory structures where the .wav files reside are identical on both machines.
>
> Thanks in advance for any advice on this,
>
> George Lass
>

This is an intriguing design requirement. It may be as simple as it
looks or it may not be.
Chris's solution is right on and a good working one for this type of scenario.
The exact design depends on whether you are going for a complete DIY,
homegrown, lowest possible cost or something more advanced and
expensive.
The scenario I interpret from your information is two asynchronous NFS
servers in an Active/Active mirror.
This is a caching nightmare to be avoided.
There might be a better way to do this depending on your true requirements.
It depends on the write bandwidth requirement for the NFS servers and
the read requirement.
If you could describe a typical operation it would be helpful. Or
worst case operation would be even better.
My worst case would be something like this.
These .wav file are logs of telephone calls. Two servers are needed
for bandwidth due to call volume. This means the same telephone number
could be accessing both servers separately simultaneously (call
waiting) or without much time lag between accesses. Now there are two
.wav files on two different servers with the same name but different
time stamps containing different information. How does the read know
which one it wants or does it want all the most recent ones?
It would be good to know if writes are imperative while reads are
deferrable within a very short delay. You did mention a one or two
minute sync delay is ok.
One solution to do this is to sync all the files on a separate "read
only" device using deduplication software (DIY solution). This leaves
the NFS front end completely asynchronous and free to just handle
writes. The "read only" device is the mirror. Some of the Web front
ends do this.
There are other considerations depending on the application and Security needs.

As per Chris's suggested solution the more done in the storage devices
the better.