[NTLUG:Discuss] RE: true hardware/intelligent ATA RAID -- Fixed disks aren't very rugged

[Bryan J. Smith]

[ NOET:  I'll respond to 3 posts in order ]

Richard Geoffrion wrote:  
> OK...so what I'm reading is NO SOFTWARE RAID-5.  .....BUT.!...

Well, you _can_ use software RAID-5.  But you'll end up pushing
about 2*N times as much data around your system interconnect (where
N is your number of disks).  Ouch!

> What if the server's entire roll in life is to be a backup server?

I look at redundancy this way, if it saves me $250 in my time, then it's
worth $250 in hardware costs (e.g., a 4-channel 3Ware controller).

> In lieu of swapping tapes, I was hoping to swap several  four disk
> raid-5 sets.

Just don't use ATA drives setup as slaves.  Hot swapping becomes a real
problem then.  And you don't even have the option with SerialATA.

> Once I install the RAIDTAB file as a persistant superblock, those four
> disks would be forever married and handled beautifully.  If I put in a
> hardware RAID card, how will the system handle the swapping of
> different 'backup-sets' (of drives)?

_All_ intelligent RAID cards store the same information persistent.

As a test, I literally took a 4-disk RAID-10 volume out of 3Ware
Escalade 6410 and put them on "out-of-order" ATA channels in a 3Ware
Escalade 7800 (non-linearally).  The volume came up without issue
and _immediately_ in 3DM.

> Wait a minute.....I don't need to do raid-5 for this.  I can just do
> RAID-0 and stripe the data across three disks with no  redundancy.
> They are just glorified backup drives anyway. Three 250Gig ide drives 
> in scsi trays = 750Gig of backup storage.  Speed isn't necessarily the
> issue...the Gigabit backbone is going to be the limiting factor anyway.

Fixed disks are not rugged.  I'd recommend you add any redundancy you
can.  Also having an intelligent ATA RAID card that can do sector remapping
(ala SCSI-like) is well worth it.

> I do need to take a look at the 3ware controllers..but I need to make
> sure that the ide drives can be hot-swapped.

Um, I've been using 3Ware products for 4 years now.  I've got first-hand
experience with hot-swapping them.  Their 3DM (3Ware Disk Manager) lets
you control the card run-time, no issues.

But I've yet to find _any_ intelligent RAID card that did not let you
do so.  Every "real hardware" RAID card allows you to do similar.
That's because the system/OS _always_ talks to the ASIC or microcontroller
on board and _never_ the "raw disk."

>From the standpoint of the system/OS with an intelligent RAID card,
the drives do _not_ exist.  That's not so with "FRAID" or other, dumb
block controllers where the drive is served "raw."

In fact, SerialATA _supports_ hot-swap natively, just like a SCSI drive
with a SCA-connector.

> I have to say that I'm concerned that using a hardware controller for
> my specific task might limit the application of solution creativity
> to my particular problem.

???
I'm not into being creative.  I'm into reliability.
If $250 buys me piece of mind, then I buy piece of mind.
It's a major cost savings over my time.
It's not much added to the disk costs either.

Having GPL drivers in the stock kernel since 2.2.15 is a major bonus.
I can boot _any_ distro on it and get instant support.

> Off to do more reading.  Thanks guys.

It's good you do your own research for yourself.  I'll never try to persuade
anyone from not doing so.  But many of us have been using 3Ware cards
without issue.

The only issue I know of that ever occurred was when 3Ware tried to
introduce RAID-5 into an existing product.  When the 7000 series came out,
the released updated firmware for the 6000 series that added RAID-5 as
well.  That wasn't smart as the 6000 series was never designed for RAID-5.
And they found the 6000's FPGAs had an issue supporting it while
maintaining SRAM cache coherency (it was okay on the 6000 as long as you
disabled the write back cache).



Kipton Moravec wrote:  
> If the system is only for backup, why bother with RAID at all?
> Just compress and store.

Reliability.  Fixed disks are not very resistant to shock.  Even if
you don't lose the drive, sectors go bad far quickier.  RAID-1, 0+1 or
RAID-5 is beneficial right there.

Plus, using an intelligent ATA RAID card that adds SCSI-like sector
remapping is very nice for when you do lose some sectors.

> If the issue is hot swappability, then go with USB Drives.

Oh God no!  Especially not for multiple disks!

Use Firewire if anything.  USB was designed for character (byte-by-byte)
transfers and _not_ block I/O.  Intel didn't want to license Firewire
from Apple (even though it was an IEEE approved standard), and decided
to create USB 2.0.

There might be little difference when it comes to optical drives that
push sub-10MBps, but when you're talking disks, it starts to turn night
and day.

And when it comes to multiple disks -- _forget_it_.  USB dies.  It _must_
transfer _all_ data back to the host controller.  Firewire can do point-
to-point device transfers.  That's why you can _never_ have something
like the iPod for USB.

> I like to keep it simple.

I do too.  I don't use USB for high-speed block devices.

You should have seen some of the early code we wrote for USB networking
hardware.  Disgusting!  Trying to do memory mapped I/O (key to block
device transfers) with USB is horrendous!  It was _never_ designed for it.

There was a reason why USB was introduced on mainboards 3+ years before
the peripherials came out.  The host controller was designed to be simple
for Intel to put into chipsets, and Microsoft to support in the OS.  All
of the "brains" go into the end-user peripherials and drivers.

That's why you typically can't get more than 1-2 devices on a USB channel
in Windows, because the end-user device drivers conflict!  Firewire was
designed with a full command set from the get-go, hence inter-device
compatibility.  iPod is one beautiful example of it.



Richard Geoffrion wrote:  
> It is my understanding that RAID-0 let's me chain two more hard drives
> together to receive aggregate disk space.

That is "spanning" -- where you "chain" the end of one disk to another.

RAID-0 is where you interleave blocks (typically every 32KB, depending
on the volume management) aka "stripe" across disks.  RAID-3/4/5 work
similar, except you use XOR parity for redundancy (and lose one disk
in total, usable storage).  RAID-3/4 use a dedicated partiy disk, RAID-5
stripes the parity along with the data.

> The files that need to be stored are already compressed on the source
> media.  The files that need to be stored exceed the capacity of the
> largest single drive I have seen out in the general market today.

Ouch!  I hope that's not one big tarball!

Anything compressed can become _useless_ if there is a _single_byte_
_error_ -- at least any data from that error forward.  LZ77 (Gzip) is
_not_ known for its recoverability.  At least BWT (Bzip2) is more of a
recoverable block compressor, and can sometimes read after an error.

Even though cpio/tar/pax "ustar" streaming formats are very recoverable,
if you compress them with Gzip, they lose 100% of that recoverability
from the point of error on-ward.  That's why I use "afio" for backups --
"ustar" format (5KB block size SysV cpio-compatible) with per-file
compression inside the archive itself.

> The Gigabit segment is the current bottleneck of the system. If I go
> with USB drives, I fear that the USB drives would become the bottleneck.
> This is based on past performance issues using USB drives for file
> backups.

While Firewire is a better option than USB 2.0, especially with multiple
disks, I'd stick with [Serial]ATA for performance and portability.

You could use RAID-0 with regular ATA cards.  But if you do, I highly
recommend you use only 1 device per channel.  Do _not_ use slaves.
You'll not only have hot-swap issues, but you'll _kill_ the performance.
Especially for RAID-0 or RAID-5.

Intelligent ATA RAID cards don't even give you the option of slave ATA
devices.  3Ware Escalade 5000/6000/7000, Adaptec 2400A/2800A and Promise
SuperTraks only allow 1 ATA disk per channel.  SerialATA doesn't even
allow a slave (only 1 point-to-point connection per "port").


-- 
     Linux Enthusiasts call me anti-Linux.
   Windows Enthusisats call me anti-Microsoft.
 They both must be correct because I have over a
decade of experience with both in mission critical
environments, resulting in a bigotry dedicated to
 mitigating risk and focusing on technologies ...
           not products or vendors
--------------------------------------------------
Bryan J. Smith, E.I.            b.j.smith at ieee.org