[NTLUG:Discuss] Re: High end PCIe storage/RAID adapters -- no PCI-X slots, but only PCI-X storage

[Bryan J. Smith]

Chris Cox <cjcox at acm.org> wrote:
> This is a new thread spinning off of the request from
> David Simmons about "Fastest Access for moving
> large (video) files"

Sorry about that.  There was a sister discussion on
PC_Support.  In a nutshell, the overriding issue is that:
A) 99.9% of PCs sold don't come with PCI-X channels
   (or multiple PCI busses)
B) 99.9% of storage controllers are PCI-X
   (or 64-bit PCI)

Even though PCIe is becoming common, there is virtual no
intelligent PCIe storage controllers available outside of the
$600 LSI Logic MegaRAID 320-2E which is U320 SCSI.  The world
is very much in need of a more cost-effective solution.

Now I see the low-cost, single PCI-X channel Broadcom HT-1000
for Athlon64/Opteron chipset as a possibility, but it's
typically implemented in a HT-2000+1000 (i.e., 3x as costly)
flavor.

So I figure the only option is to wait for the Broadcom
BCM8603-based PCIe x8 SAS/SATA RAID controllers come out --
hopefully before year's end.  That'll allow you to get
dedicated, high performance SATA I/O on a commodity
(sub-$200) mainboard -- hopefully with a product that will
cost under $300-400.

> I'm figuring there are some problem in SAS-land.

Oh, there's always going to be problems with new technology.
But the "nice fallback" with SAS is that it does SATA as
well.

> The new Sun line of servers was due out already... I think
> something has held them back... I'm wondering if there's
> some kind of problem with the LSI chipset or with
> the 2.5" drives... not sure.
> Tunneling SATA in SAS is theoretically possible, but
> I don't think there's an available implementation.

I'm not really talking about tunneling.  I'm talking about
SAS controllers directly controlling SATA drives.

In a nutshell, the future is a single, commodity RAID
controller that will do both SAS and SATA.  If you want
multiple targets, etc..., you'll use SAS.  If you just want
local storage, you'll use SATA.

> Again, I think they're still working out some kinks.
> I was not aware of Broadcom's entry into SAS space.
> Could be interesting.

They are aiming for the commodity space.  Broadcom bought
ServerWorks, the company who has designed every Intel
server chipset after the 450NX.

> My next home workstation would ideall have 2xdual core
> Opterons and an all PCIe board (someday).  I haven't
> seen anything great on the PCIe end with regards to
> good high end RAID controllers though...

The Intel IOP332 is the PCIe version of the IOP331 (an
improvement over the IOP321) typically used in XScale driven
RAID designs.  I haven't seen anything other than the
MegaRAID 320-2E using it.

The Broadcom BCM8603 is, seemingly, it's first _real_
intelligent RAID design, and boy is it innovative IMHO. 
Prior to this design, Broadcom was selling the acquired
RAIDCore "software" (i.e., main CPU host/interconnect drive)
suite of RAID logic.  Very advanced, but it kills the host
system with I/O traffic.

The BCM8603 has both PCI-X and PCIe (x8) arbitration, and can
even act as a bridge (which is nice for embedded systems). 
It has up to eight (8) SAS channels, and also does SATA
natively.  Can't remember what the integrated SRAM buffering
it (I think rather limited, maybe only 256KiB), but upto
three (3) DDR SDRAM channels can be glued (768MiB).  IC cost
is $60/unit in quantity.

I don't know how much "intelligence" is in the firmware, but
it's clear that Broadcom is going to continue to sell its
software logic for features as add-ons.  But still, the
buffering, XOR operation and RAID recovery/rebuild is hosted
intelligently in the IC, and uses directly connected DRAM.

> I'm going to make this a new thread and see what
> others may have seen/heard.

Yeah, the MegaRAID 320-2E is the only thing I've seen.
It's really sad that 3Ware hasn't addressed the PCIe market
yet.


-- 
Bryan J. Smith                | Sent from Yahoo Mail
mailto:b.j.smith at ieee.org     |  (please excuse any
http://thebs413.blogspot.com/ |   missing headers)