[NTLUG:Discuss] Re: why swap? -- welcome to Gigabit Ethernet

[Bryan J. Smith]

On Fri, 2004-09-10 at 11:19, Robert Citek wrote:
> One more thing: I noticed these message appearing in dmesg:
> TCP: Treason uncloaked! Peer 10.4.0.4:514/1023 shrinks window 
> 885933436:885934884. Repaired.
> TCP: Treason uncloaked! Peer 10.4.0.4:514/1023 shrinks window 
> 885933436:885934884. Repaired.
> TCP: Treason uncloaked! Peer 10.4.0.4:514/1023 shrinks window 
> 886440236:886441684. Repaired.
> TCP: Treason uncloaked! Peer 10.4.0.4:514/1023 shrinks window 
> 886440236:886441684. Repaired.
> Don't know if this is cause/effect or simply a coincidence.  Still 
> googling to figure out what this means.  From what I can tell so far, 
> this has something to do with an old TCP stack, which seems strange 
> given that the kernels on both boxes are 2.4.8 or later.

First off, I can't remember if we had the GbE discussion here?
Or over on JAXLUG (?).

As such, I've personally seen this with either:  
  1)  Direct GbE-to-GbE NIC connection
  2)  Cheap GbE switches

In a nutshell, at 1500 byte frames, the tiny 16-32KB SRAM (if even that)
in typical, low-cost, integrated GbE NICs isn't enough.  In the case of
a "cheap" switch (aka "desktop"), the switch can't buffer it, because it
typically has only 1MB or less of SRAM.

The result is the receiver is changing the framing, because it can't
handle the incoming stream.  This is horrendous for performance.

I haven't done enough testing to see if it is the result of 802.3x Flow
Control (_not_ to be confused with 802.1x Authentication) or, more
likely, the lack of 802.3x and use of cheap hardware.

I assume 802.3x Flow Control at the frame (Ethernet) level would prevent
you from even seeing segment (Transport) level errors such as this.  But
that is only an assumption because I've yet to be able to deal with it
other than passing.  I.e., buying a better GbE switch(es) solves the
problem instantly.

As such, I'd be curious what NICs and switches you are using?


-- 
     Linux Enthusiasts call me anti-Linux.
   Windows Enthusisats call me anti-Microsoft.
 They both must be correct because I have over a
decade of experience with both in mission critical
environments, resulting in a bigotry dedicated to
 mitigating risk and focusing on technologies ...
           not products or vendors
--------------------------------------------------
Bryan J. Smith, E.I.         b.j.smith at ieee.org