[NTLUG:Discuss] Re: why swap? -- welcome to Gigabit Ethernet

[Robert Citek]

On Friday, Sep 10, 2004, at 11:00 US/Central, Bryan J. Smith wrote:
> In a nutshell, at 1500 byte frames, the tiny 16-32KB SRAM (if even 
> that)
> in typical, low-cost, integrated GbE NICs isn't enough.  In the case of
> a "cheap" switch (aka "desktop"), the switch can't buffer it, because 
> it
> typically has only 1MB or less of SRAM.
>
> The result is the receiver is changing the framing, because it can't
> handle the incoming stream.  This is horrendous for performance.
>
> I haven't done enough testing to see if it is the result of 802.3x Flow
> Control (_not_ to be confused with 802.1x Authentication) or, more
> likely, the lack of 802.3x and use of cheap hardware.

Any suggestions on how to test this?  I imagine connect the two 
machines directly via crossover cable, push a lot of data down the 
pipe, and measure the rate.  Then repeat via the switch.  Or are there 
better ways?

> As such, I'd be curious what NICs and switches you are using?

On one machine (10.4.0.5) using the e1000 driver:
# lspci -s 00:0b.0 -v
00:0b.0 Ethernet controller: Intel Corp. 82545EM Gigabit Ethernet 
Controller (Copper) (rev 01)
         Subsystem: Intel Corp. PRO/1000 MT Server Adapter
         Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 19
         Memory at f5000000 (32-bit, non-prefetchable) [size=128K]
         I/O ports at 1000 [size=64]
         Capabilities: [dc] Power Management version 2
         Capabilities: [e4] PCI-X non-bridge device.
         Capabilities: [f0] Message Signalled Interrupts: 64bit+ 
Queue=0/0 Enable-

On the other (10.4.0.4) using the tg3 driver:
# lspci -s 01:06.0 -v
01:06.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5700 
Gigabit Ethernet (rev 14)
         Subsystem: Dell Computer Corporation Broadcom BCM5700
         Flags: bus master, 66Mhz, medium devsel, latency 32, IRQ 22
         Memory at fcf00000 (64-bit, non-prefetchable) [size=64K]
         Capabilities: [40] PCI-X non-bridge device.
         Capabilities: [48] Power Management version 2
         Capabilities: [50] Vital Product Data
         Capabilities: [58] Message Signalled Interrupts: 64bit+ 
Queue=0/3 Enable-

Don't know what the switch is (don't have physical access to it right 
now).  Will find that out later.

Regards,
- Robert
http://www.cwelug.org