[NTLUG:Discuss] RE: true hardware/intelligent ATA RAID -- WAS: compression and Filesystem question

[Bryan J. Smith]

Stuart Johnston wrote:  
> Do you know anything about the LSI MegaRAID SATA cards?  I did find
> one page saying that the 4 and 6 port cards are true hardware while
> the 2 port is based on a Silicon Image controller.
> http://www.linuxmafia.com/faq/Hardware/sata.html

Yeah, Rick Moen's site.  The same guy that didn't take the time to
listen to me on SVLUG when I tried to tell him about "true hardware" AA
RAID cards with command queuing, sector remapping, etc... about 3-4
years ago.  He thought everything was like the SuperTrak.

Glad to see he's changed.

But yeah, select Intel and LSI cards use the i80300-series
microcontroller, which is i860/960 core.  Pretty much the same, stock
design as most SCSI RAID cards -- they merely replace the SCSI HAs with
ATA controllers.

BTW, let me know if they are actually StrongARM/Xscale based.
I wasn't aware any in the i80300 series were.

> penguincomputing.com sells servers with LSI hardware RAID which I 
> occasionally drool over but have never had the chance to buy.

I prefer 3Ware, especially now that the Escalade 9000 series is out
with a large SDRAM buffer in addition to the non-blocking ASIC+SRAM
design.

Buffered I/O, which is the legacy approach of microcontroller (uC) plus
DRAM (such as Synchronous DRAM) is fine for SCSI, because you already
have latency in the SCSI overhead.  But for ATA, which is non-blocking
I/O, you want a 0 wait state ASIC + SRAM (static RAM, like CPU cache,
also used in network switches) for RAID-0, 1 and 0+1 (10).

About the only place where Buffered I/O is better is for RAID-5 writes
(RAID-5 reads work like a RAID-0 read).  Then the small SRAM cache of
the 3Ware Escalade 5000-8000 series overflows.  That's why the 3Ware
Escalade 9000 series was introduced, so it has both SRAM and SDRAM.

Comparing the ASIC+SRAM approach to uC+DRAM approach for RAID-0, 1 and 10
is literally like comparing a layer-3 switch fabric to a well-designed 
PC/Linux router.

Overall, the PC/Linux router can probably can handle the load -- assuming
a fast CPU, lots of memory and PCI-X network cards.  But the Ethernet
switching ASIC does it just as fast with a lot less overhead.  The only
time the PC/Linux router starts to take over is if the traffic is massive
and 10MB+ of data needs to be buffered.

But then you get poorly designed PC/Linux routers with a slow CPU and,
worse yet, crappy, legacy "shared" PCI.  This is like a RAID card trying
to use i860/960 microcontrollers these days.  They are going to _hurt_
RAID-0, 1, 0+1 (10) performance versus an ASIC solution.

In  a nutshell, I buy only two types of RAID cards:

ASIC+SRAM:   $100-500
- 3Ware Escalade series (9000s are not much more, well worth it)

uc+DRAM:  $1,000+
- StrongARM/xScale microcontroller based (e.g., Mylex eXtremeRAId)

90% of the "real hardware" ATA/SCSI RAID cards out there are still using
legacy i860/960 microcontrollers.  While they are fine for buffered RAID-5
performance, they are a _bottleneck_ for RAID-0, 1, 0+1 read/writes,
or RAID-5 reads.  Don't buy them, unless they are cheap (sub-$100).

Going with a 3Ware Escalade series would be far better, and the 9000 series
is the "best of all worlds" even for RAID-5.  3Ware has supported Linux with
GPL drivers well before anyone else, and Adaptec's official policy (we
have a support center here in Orlando) is that they only support Linux for
the _host_adapters_ and _not_ RAID (at least that what I was told as of
last fall).

You need something like a StrongARM or Xscale processor to deal with today's
ATA/SCSI DTR rates, especially for RAID-0, 1, 0+1 reads/writes, or RAID-5
reads.  Intel didn't buy Digital's fabs (and later Digital itself) for the
Alpha, it bought it for rights to the StrongARM chip.  The Xscale is based
on the StrongARM.

IBM's Mylex division supports Linux very well as well.  Even the most entry-
level StrongARM can push 5x as much data around as the fastest 100MHz i960
design -- of which, Adaptec only uses a 66MHz and some of the early
Promise SuperTraks used only a measly 33MHz.  The latter capped out at
40MBps, clearly because the slow i960 was a bottleneck!


-- 
     Linux Enthusiasts call me anti-Linux.
   Windows Enthusisats call me anti-Microsoft.
 They both must be correct because I have over a
decade of experience with both in mission critical
environments, resulting in a bigotry dedicated to
 mitigating risk and focusing on technologies ...
           not products or vendors
--------------------------------------------------
Bryan J. Smith, E.I.            b.j.smith at ieee.org