[NTLUG:Discuss] Fastest Access for moving large (video) files?

[Chris Cox]

David Simmons wrote:
> Guys,
> 
> I have a box that does video work...so most of it's files are very large
> and I've noticed that most of the time my CPU usage is low, but
> hard-drive light is flashing like crazy - so, my bottle-neck is just
> moving this data around.
> 
> Root Question:  How do I create the (within a normal budget) fastest way
> to move large files around?  Considering that my 'work area' could be as
> small as 10Gb.
> 
> Thoughts:
> 
> 1).  I know that a RAM drive would probably be the fastest...but given
> that I'm looking for 10Gb, I don't know of 'normal' motherboards going
> that high...and I'm sure dedicated RAM-Drive cards are way out of the
> price-range

Price range?  Any specifics will help in determining what you can
do.

You have to admit, that if you're serious about doing large  and high
speed.. that you are usually outside of "normal" :)

Knowing your budget will help a lot in deciding what is feasible
I think.

> 
> 2).  Have been reading some good results using SATA-300 or SATA-II.  But
> given that I have the old standard PCI bus, will that be the limiting
> factor? (as I'll have to add a SATA-II card to the system).

It really depends on the number of drives you'll have and the number
of dedicated PCI bus channels you really have.

I mean a high-high end drive still doesn't go above 100M/sec.  Which
makes SATA-300 not as atractive as it first might seem.

But with striping, it's possible the 32-bit 33Mhz PCI could be
a problem... how many drives can you handle? (could be
greatly limited if internal to your existing machine).

> 
> 3).  Any comments on the WD Raptor?

The Raptor is a excellent choice for a really fast drive that
still reasonably priced.  However, a striped set of cheaper
drives might be faster and oddly enough, cheaper.

> 
> 4).  Instead of a single drive - do I go RAID0 (striping) - if so, any
> recommendations on setup? (ie. go ATA/IDE or SATA?  (Again, will the PCI
> bus be the limit?)

If the RAID0 is used as temporary storage, I think just striping
can be an excellent tool... you obviously you know that the "R" is
really missing from RAID0... lose one device, lose it all.

IDE can be problematic if you have a large number of devices.
SATA connections are dedicated.... I guess I'm trying to say,
you want to avoid having more than one device on a single
IDE channel (duh). SATA is starting to support some
SCSI like features... that's also a plus for SATA.

I think a bit more information is needed on all of this.

Some things to answer:

Does the storage have to be internal or is external an option?
How many drives max (especially if only looking at internal)?
What is your desired throughput?  Any estimates?  (e.g. 75M+/sec)
What is the PCI bus configuration right now?  Number of
channels... devices already on the bus...
And of course... how much money can you spend?!!