[NTLUG:Discuss] NFS Synchronization

[George Lass]

--- On Fri, 5/21/10, George Lass <george.lass at sbcglobal.net> wrote:

> From: George Lass <george.lass at sbcglobal.net>
> Subject: Re: [NTLUG:Discuss] NFS Synchronization
> To: "NTLUG Discussion List" <discuss at ntlug.org>
> Date: Friday, May 21, 2010, 10:20 AM
> 
> 
> --- On Fri, 5/21/10, Robert Pearson <e2eiod at gmail.com>
> wrote:
> 
> > From: Robert Pearson <e2eiod at gmail.com>
> > Subject: Re: [NTLUG:Discuss] NFS Synchronization
> > To: "NTLUG Discussion List" <discuss at ntlug.org>
> > Date: Friday, May 21, 2010, 3:43 AM
> > On Thu, May 20, 2010 at 4:08 PM,
> > George Lass <george.lass at sbcglobal.net>
> > wrote:
> > > I need to set up a pair of NFS servers that need
> to be
> > synchronized and am wondering if rsync is the right
> way to
> > do it?  The primary use of the pair of servers is
> for
> > recording and playback of phone messages stored in
> .wav
> > files.  The only "gotcha" is that any particular
> recording
> > will originally be written (recorded) randomly to one
> of the
> > servers or the other, while read (playback) requests
> can be
> > made to either server.  Thus within just a minute or
> 2, the
> > file must be propagated from one server to the other.
>  It
> > should be assumed that the directory structures where
> the
> > .wav files reside are identical on both machines.
> > >
> > > Thanks in advance for any advice on this,
> > >
> > > George Lass
> > >
> > 
> > This is an intriguing design requirement. It may be
> as
> > simple as it
> > looks or it may not be.
> > Chris's solution is right on and a good working one
> for
> > this type of scenario.
> > The exact design depends on whether you are going for
> a
> > complete DIY,
> > homegrown, lowest possible cost or something more
> advanced
> > and
> > expensive.
> > The scenario I interpret from your information is two
> > asynchronous NFS
> > servers in an Active/Active mirror.
> > This is a caching nightmare to be avoided.
> > There might be a better way to do this depending on
> your
> > true requirements.
> > It depends on the write bandwidth requirement for the
> NFS
> > servers and
> > the read requirement.
> > If you could describe a typical operation it would be
> > helpful. Or
> > worst case operation would be even better.
> > My worst case would be something like this.
> > These .wav file are logs of telephone calls. Two
> servers
> > are needed
> > for bandwidth due to call volume. This means the same
> > telephone number
> > could be accessing both servers separately
> simultaneously
> > (call
> > waiting) or without much time lag between accesses.
> Now
> > there are two
> > .wav files on two different servers with the same name
> but
> > different
> > time stamps containing different information. How does
> the
> > read know
> > which one it wants or does it want all the most
> recent
> > ones?
> > It would be good to know if writes are imperative
> while
> > reads are
> > deferrable within a very short delay. You did mention
> a one
> > or two
> > minute sync delay is ok.
> > One solution to do this is to sync all the files on a
> > separate "read
> > only" device using deduplication software (DIY
> solution).
> > This leaves
> > the NFS front end completely asynchronous and free to
> just
> > handle
> > writes. The "read only" device is the mirror. Some of
> the
> > Web front
> > ends do this.
> > There are other considerations depending on the
> application
> > and Security needs.
> > 
> > As per Chris's suggested solution the more done in
> the
> > storage devices
> > the better.
> > 
> > _______________________________________________
> > http://www.ntlug.org/mailman/listinfo/discuss
> > 
> 
> Fortunately the worst case is not quite that bad, and in
> retrospect, I should have mentioned that the .wav files will
> all have unique names regardless of which server they are
> written to.  That should eliminate many of the
> previously mentioned issues.  That said, here are some
> more details concerning what it is that I need to do....
> 
> The phone switch that I am dealing with can read and write
> .wav files in real time via the NFS protocol.  During
> switch setup I can give it 2 different IP addresses and
> mount points for NFS servers.  When I tell the switch
> to play a .wav file, I specify which of the files is
> "primary" and which is "secondary".  It will request
> the file from the primary first, and if it's not there, or
> the primary server is down, it will make the same request of
> the secondary server.
> 
> Record operations are essentially the same.  I specify
> which of the 2 pre configured servers is primary and which
> is secondary.  Again, it will try to create the .wav
> file on the primary server, and if that fails it will try to
> create it on the secondary.
> 
> Given the above flexibility, I could normally specify
> server "A" as primary and "B" as secondary, and for the most
> part I would just need to sync from A to B.  But as
> previously mentioned, I would like the phone switch to load
> share the 2 servers.  So I will likely set up toggle
> functions that will tell the phone switch to alternate A
> primary, B secondary / B primary, A secondary for every
> other read, and for every other write.
> 
> Hope this helps and thanks!
> 
> George
> 
> 
> _______________________________________________
> http://www.ntlug.org/mailman/listinfo/discuss
> 

Oh, and one other thing......

Replication of the .wav file to the "other" server can be delayed by up to 5 minutes.  

George