[NTLUG:Discuss] Sun gone AMD NUMA/HyperTransport -- WAS: what companies are using linux in the dfw area?

[Bryan J. Smith]

On Thu, 2004-11-11 at 14:32, Chris Cox wrote:
> Let's just say, I'd be a bit concerned if I were a SPARC user....
> but I'm telling more than Sun will say.

The AMD NUMA/HyperTransport is the first commodity PC platform that
offers the I/O performance of RISC/UNIX platforms.

Sun's 1 and 2-way 1100z/2200z designs have the same I/O capacity as
AMD's 4-way reference design which is implemented in the HP Proliant
DL585.  And the DL585 tore up the benchmarks for TPC-C, SAP DB and
Exchange, even at 32-bit Windows (not even considering Linux/x86-64).

For more on how the AMD 4-way reference design / Proliant DL585 works,
see this diagram (NOTE:  the DL585 uses (2) AMD8131 chips):  
  http://www.samag.com/documents/s=9408/sam0411b/0411b_f5.htm  

For a general discussion on the differences between the old "front side
bottleneck" approach of Intel GTL/GTL+/AGTL+, some proprietary Xeon NUMA
designs and AMD's new NUMA/HyperTransport approach, see this article
that has no less than 7 diagrams detailing various designs:  
  www.sysadminmag.com/documents/sam0411b/  

BTW, the "CK8Pro" HyperTransport chip is now officially the "nForce4." 
Although it is designed for desktops, a few vendors are producing a
workstation/server version for dual-Opteron that puts the nForce4
(PCI-Express) on the HyperTransport like of one Opteron 200 while using
an AMD8131/8132 PCI-X 1.0/2.0 chip on another Opteron 200:  
  http://www.samag.com/documents/s=9408/sam0411b/0411b_f7.htm  

> This is a stab at IBM btw.  Though IBM has the Sanmina connection
> (old Newisys)... Sun is probably outspending IBM 2-to-1 on Opteron
> research at the moment.  I believe Sun will bring 16-way and 32-way
> Opterons to market first.

Actually, there are several vendors already buzzing around this space. 
Heck, even Cray has an optical implementation of 1600MT 16+16 (6.4GBps)
HyperTransport.  I can't wait for some of the benchmarks to come out on
those "shared memory clusters" -- it's a total blur of the difference
between a "shared memory" system and a "cluster."

> IBM loved the Opteron first,

Er, um, well, IBM was contracted to foundary the A64/Opteron.  IBM has
stated they do not believe Opteron will be the most ideal solution after
2005, but it's own Power4/Power5 and their PowerPC equivalents (e.g.,
the PowerPC 970 is based on Power4, known as the Apple G5).

BTW, UMC was actually contracted first,  but couldn't deliver on its
promises.  UMC has been little more than a foundary for AMD's "trailing
edge" processors.

> Sun appears to love the Opteron even more.

Of course!  No vendor can resist a _commodity_ NUMA/HyperTransport
platform.  Now every traditional RISC/UNIX vendor essentially has the
ultimate refinement of the Digital Alpha platform with an x86 processor.

Alpha?  Where's I get that?  You have to know the history of the
development of the "Lightspeed Data Transport" (LDT) now known as
"HyperTransport."  Long story short, the same engineers who developed
the first PCI bridges, AGP logic, various NUMA implementations, first
integrated NIC ASIC, etc... was Digital Semiconductor.

As many will note, the 32-bit Athlon adopted a 3-4 node EV6 (Alpha
21264) design for its interconnect (which was actually 40-bit, long and
interesting story -- especially for the 32-bit Athlon that was capable
of addressing 1TB -- yes, actually 1,024GB ;-).  HyperTransport was the
ultimate evolution of where this was headed.  Combined with NUMA in
direct memory connections to each CPU, it is now the _ultimate_ platform
for workstations and servers -- at a commodity price.

In the mid-'90s, with Palmer's "sell-off-a-thon" a lot of engineers left
for AMD, especially after Intel's fab licensing and then their final
purchase by Intel outright.  Although some founded Alpha Processor, Inc.
(API), what then became API Networks was the mastermind behind LDT,
ultimately HyperTransport.  The rest is history, including API Networks
being a subsiderary and primary R&D arm of AMD.

But in the end, I would argue that HP loves it most.  HP co-designed the
EPIC/Predication design of IA-64 -- a CS ideal that _flopped_ in
silicon.  It has nothing to do with Intel x86 or HP PA-RISC
compatibility, IA-64 _flopped_ at even its own, native ISA (instruction
set architecture) performance.  Run-time execution is not optional, and
the Digital Semiconductor team stated that in the mid-'90s when IA-64
was announced with its EPIC/Predication.  Sure enough, they were right.

So as Intel/HP now "retrofit" traditional, run-time microprocessors
optimizations like out-of-order execution, register renaming and branch
prediction onto IA-64, the IA-64 platform still doesn't offer the raw
interconnect capacity of AMD's inherent NUMA/HyperTransport platform. 
Sure, IA-64 is available in NUMA flavors, but not nearly at the
commodity pricing -- let alone it's typically more expensive than even
some RISC/UNIX platform equivalents.  No, HP's support of Opteron is a
clear "slap in the face" to Intel.

God knows Itanium3 is looking more and more like IA-64 hacked up with
approaches Digital was using on Alpha over 7 years ago like binary
translation (FX!32 and related technologies originally developed for VMS
from VAX->Alpha, but worked well for Linux and Windows x86->Alpha too),
Alpha optimization approaches, etc...  Quite sad, Intel could have just
bought Alpha outright in the mid-'90s and solve the problem.

Heck, most people don't know that Intel and Digital _were_ standardizing
on the 64-bit Alpha _before_ Intel broken it off in the early '90s!  [
Resulting in the lawsuit where Intel adopted the superscalar approaches
of the Alpha into the Pentium Pro (the current ALU/FPU for _all_ Intel
x86 processors, including IA-32e 64-bit extended ones), solving a lot of
ALU and FPU design issues from the original the Pentium -;].
 
> How long ago was it that I told everyone that Sun should make an
> exclusive deal with AMD??  I read this company like a book... all
> too predictable.

Everyone is going Opteron, except for maybe IBM.  IBM seems to be
dedicated to Power/PowerPC -- at least for the long run.  But IBM
doesn't seem to be making on its promises of OEM microelectronics and
"white box" PowerPC solutions.  They are rare, costly and expensive.

Not the commodities-of-scale of the 30%+ selling A64/Opteron.


-- 
Bryan J. Smith                                  b.j.smith at ieee.org 
------------------------------------------------------------------ 
"Communities don't have rights. Only individuals in the community
 have rights. ... That idea of community rights is firmly rooted
 in the 'Communist Manifesto.'" -- Michael Badnarik