[NTLUG:Discuss] Humpty Dumpty Syndrome: split and cat on a 5 gig file

[Bryan J. Smith]

Top-posting here ...

I'm kinda confused on the exact procedure you used?
Did you backup the Windows system "live"?
Or were you saying you booted into Gentoo instead?
Was 1 system involved?  Or 2 systems?

Understand that dd copies _all_ sectors of the disk.
So the target partition must be the same or bigger than the original.
Also make sure your head/cylinder (typically 16/63 or 255/63, the latter
is often forced by NT5.1/XP, 255/63, is the same).

- Disk to Disk

When it comes to replacing disks in 1 system, put the new drive on a
different ATA channel and run (e.g., hda and hdc):  

  dd if=/dev/hda of=/dev/hdc bs=###M

Where "###" is a multiple of either 512KiB (16/63 hds/sec) or 8MiB
(255/63 hds/sec), and no more than your available userspace RAM in
Linux.

- Disk over network to another disk (via SSH)

Now if you want to image the disk to another disk on another machine,
one machine needs to be the SSH server and the other a client (you can
use RCP if you wish instead).

Assuming "/dev/hdc" is in the SSH server.

  dd if=/dev/hda bs=8M | ssh user at system "dd of=/dev/hdc"

- Streaming archive of disk over network to another disk (via SSH)

Now let's say dd isn't an option.  Now we can use a USTAR archive
(cpio/tar/pax) stream over SSH as well.  After this is done, you'll need
to boot XP in the recovery console and run "fixboot" and "fixmbr", but
it will at least let you make filesystems whatever size you want.

Assuming "/dev/hda1" is the original filesystem mounted on "/mnt/c", and
"/dev/hdc1" has been setup on the SSH server mounted as "/mnt/newc".

  mount -t vfat /dev/hda1 /mnt/c 
  cd /mnt/c
  tar clO . | ssh user at system "tar xvC /mnt/newc"
    [NOTE: GNU Tar syntax]

- Backups:  afio

Windows NT5.0 (2000) used to come with "pax" which is the new POSIX
USTAR archiver replacement for cpio/tar.  For some reason, Microsoft
yanked it in NT5.1 (XP/2003).

Although lots of people recommend different backup programs, I recommend
"afio" for its per-file compression and multi-volume capabilities.  That
way if you have a single byte error, you don't lose the entire archive
due to compression (only the file inside of the archive that was
compressed where the error occurred).  It also natively supports
separating itself into _independent_ volumes at X number of bytes (user
settable), so you don't have to use something like split which is at the
mercy of getting _all_ bytes back together as a whole.

It also uses the standard 5KiB block USTAR (SysV cpio, Tar is 10KiB)
stream, so it can be read by any program, and not just afio although the
individual files will have ".z" after them if you don't have afio.  Now
at >2GiB files, afio breaks SysV cpio compatibility, but that's not an
issue if you make multivolume afio files that are under 2GiB.

  http://freshmeat.net/projects/afio/  


On Sun, 2005-08-07 at 11:08 -0500, Lenrek Xunil wrote:
> I'm doing some pro bono work at my brother's church.
> I needed to transfer a 5 gig backup file over a wireless network, so I
> split it into 10 files.  I used the Gentoo installation CD-ROM, which
> gives you a command line when you boot up.
> Transferring the whole file was difficult because of interruptions
> (windows update trying to reboot, norton live update trying to reboot,
> acrobat reader trying to reboot as I was browsing the web, me getting
> bored and doing another task that forced me to reboot)...  Thinking
> now of the trouble it has caused me,  I would have taken my router/hub
> and cables... but it's too late now.  Also, when I used the tar
> command to backup the whole hard drive to another hard drive, to
> include the 5 gig file, it gave me trouble when it got to that file
> (eventually I just used "dd", but this is another topic).
> Anyway, as a test run, I copied the files to a second hard drive and
> tried to put them back together using a script:
> #!/bin/bash
> cat xab >> xaa
> cat xac >> xaa
> etc...
> When it got to xai and xaj, it put out an error message for each
> saying something like "file limit exceeded".  Now I know it was not
> the hard drive, because it was a 40 gig hard drive that was mostly
> empty.  This second hard drive, by the way, was partitioned to 30 gigs
> using linux fdisk ("c" filesystem) and formatted as fat32 using
> Windows XP.
> So why can't I put the file back together again?  Is it that the cat
> command can only handle so much?  Is it the formatting?  Please help. 
> Thank you.

-- 
Bryan J. Smith     b.j.smith at ieee.org     http://thebs413.blogspot.com
----------------------------------------------------------------------
The best things in life are NOT free - which is why life is easiest if
you save all the bills until you can share them with the perfect woman