[NTLUG:Discuss] Copying a file to TWO destinations as the same time..

[Chris Cox]

Richard Geoffrion wrote:
> Chris Cox wrote:
...
>> How much data are you backing up?
>>  
>>
> ah....now that depends on the day.  Seeing as how it is a
> dirvish-rsync... we are backing up hundreds of megs and making a
> gaboodle of hard links. ( one gaboodle = 1024 gobs * 1024)
> 

Ok.. that's not a lot of data.  Shouldn't be taking long at all.
But I just use rsync... not dirvish.. not sure of any overhead
that dirvish might add.

>> What is your sustained drive writing speed going from the
>> "inhouse drive" to the "offsite drive".  Are you getting
>> 75M/sec or higher?
>>  
>>
> clarification point.    the 'inhouse drive' AND ' offsite drive' are
> both DESTINATION drives with the rsync of MULTIPLE SOURCES/MULTIPLE
> SERVERS occuring over a gigabit backbone.

Ok.  Gigabit, that's good.

> 
> David's suggestion to "modify the source code to rsync to connect to two
> different places at once, ..[and] make two different compare tables to
> tell what to transfer to each [destination device]." is in fact the
> proposed solution.

BUT.. is it the RIGHT solution.

> 
> How do you guys suggest I go about getting that
> done/sponsored/whatever?  That would be a great contribution to
> well....at least the dirvish project.

Are you rsync'ing all of the hosts at once (async) or
are you doing each one synchronously?  You should be
saturating your gigabit line... (you probably are, but
I have to ask).


If it were me, I do a normal copy, or better, an rsync
of the data to the second set going offsite.  I don't see how
your rsync to multiple areas would work well... since
the offsite drive and local drive would probably
not be identical.  Doing the coding you're proposing,
IMHO, would be VERY difficult.  Rsync is highly
optimized... might be hard for it to try to keep track
of multiple sources of synchronization.


So... IMHO...

hosta
hostb ----> rsync to ---> onsite drive
hostc
hostd

Then....


onsite drive  ---> rsync to ---> offsite drive

Depending on the difference between onsite
drive and offsite drive, worst case is total
filesystem copy time wise.

I'll assume the offsite drive might contain
some like data as it is a cyclic return
from your offsite location.