[NTLUG:Discuss] File Replication & 2-way Synchronization

[Paul Ingendorf]

What your looking at doing is something that has been done for years before the Internet became popular via wan links from location to location.  Of course this requires extra hardware but it is more reliable more secure than any bailing wire/duct tape approach you can take using the Internet as your communications backbone.

What you might want to do is analyze your situation and look at the true pros and cons.  Make a list of things that your solutions must have in order to be effective.  This list should encompass everything that is required of your users.  Then take a step back and look at what it would be nice to have i.e. speed, reliability, cost etc.  Assign importance values to the features it would be nice to have in terms of increased productivity for the most part ease of administration would be another factor.  Then rate how well each of the solutions fills your particular needs.  This will help you make the decision on what is best for your situation which none of us could be expected to have a complete understanding of.

Given there is more than one location unless these are all home offices the optimal setup would be to invest in point to point channelized lines.  For instance if all the DSL lines have a maximum upstream of 128k then pick a central location for hubbing and you allow for two framed connections from there to each of the remote offices. or maybe you run a nice ring config between the locations.  You will notice much greater performance out of a setup like this plus you don't degrade the performance of the Internet link and you are able to maintain current levels of network security.  An rfp justifying the increase in monthly cost could easily be put together justifying it with the increased collaboration ability lowering the over all cost of doing business.

Of course if this is a no budget type thing you could also look at software approaches that allow you to map ftp sites as network shares like the following.
http://www.deerfield.com/products/internet_neighborhood/

Food for thought.

-----Original Message-----
From: discuss-admin at ntlug.org [mailto:discuss-admin at ntlug.org]On Behalf
Of Stan Tigrett
Sent: Wednesday, February 13, 2002 11:49 AM
To: discuss at ntlug.org
Subject: [NTLUG:Discuss] File Replication & 2-way Synchronization



PROBLEM: We have 3 offices in different locations who need access to each of the other office's files.  The offices each run a linux/samba fileserver, with clients using win9x to connect.  Each office is connected to the internet via a ADSL line (1Mb/128kb) and run a linux firewall (a different machine than the fileserver).  One office has about 10 gigs of data, and the other two have about 2 gigs each.  The users can't handle using FTP - they will only accept a 'new drive mapping' in windows.  It is not very likely that the same file will be accessed by different users at different locations at the same time.

POSSIBLE SOLUTIONS:
1 - Mount the data remotely using NFS across the internet, using strict firewall rules to only allow exporting to the 3 office IP addresses.  Use samba to share the nfs mount to the users.
	Pros - 	Data will always be current.  Changes will occur in real time.
	Cons - 	The DSL links not very stable.  Would probably need a cron job to test the nfs connection & re-mount it if neccessary.
		Data access will be extremely slow.

2 - Use rsync to perform 2-way synchronization of the files, using the same strict firewall rules above...  Use samba to share the replicated directories.
	Pros -	Data access will be as fast as local data.
		Don't have to worry about the DSL links
	Cons - 	Data may be stale.
		Initial replication will take a long time...
		Deleting files from one server to the other...

It seems that #2 is the best solution so far, using a cron job to synchronize the files every hour or so, but I have one major caveat.  One of the accounting programs these offices use creates several temporary *.dbf files while they are working on it.  Using rsync without the --delete option would eventually leave hundreds of these files laying around.  However, if I do use the --delete option, those temp files may be deleted if a synchronization occurs while a user is still working with them. 

If there were a way to only delete files that were created BEFORE the last synchronization occured, then we would be in good shape, I think.  I haven't found any kind of option like that, though.

Any ideas? Advice? Other approaches?

Thanks -
Stan

_______________________________________________
http://www.ntlug.org/mailman/listinfo/discuss