[NTLUG:Discuss] Re: Linux article

[Bryan J. Smith]

Steve Baker wrote:  
> OMG!  Really?   Wow - that's such an elementary screwup.

Not really.  Take any elementary programming course and they will talk
about I/O with "binary records" and "character."  A _lot_ of developers
just do binary record reads and writes.

On x86, alignment of these records can vary widely!  Even a struct in C
does _not_ always mean it will be that format exactly -- only when you 
use bit defines does it.

As Microsoft found out, the slightest compiler changes can have the
most profound effects!  Rendering even patches to MS Word incompatible
with documents from prior versions, etc...

And then you have endianness issues.

"Network order" (Sun-style big endian) is the de-facto standard used
for most data formats.  Even when you use struct, you then _write_
in a byte-by-byte fashion, doing any word/byte endian conversion
as necessary for the platform.

This is _commonplace_ with "good programming practices" for GCC
developers.  This combined with Autoconf for porting does a lot.

These practices are virtually _unheard_of_ in the PC/Windows world.
Why?  My "Land locked" analogy works.

Even Apple has an internal x86 and x86-64 ports of MacOS X, including
Cocoa and Aqua.  Not just Darwin/x86, but right down to the full
environment.  It keeps them aware of porting issues.  They also use GCC,
which helps too.  And even though PowerPC is bi-endian, they use it
in big endian for MacOS X, which means it's the little endian port
to x86 is no small feat.

Sun has the same thing with SPARC, which uses 8-byte (64-bit) alignment.
It is big endian-only.  Now that Sun is looking to go x86-64 soon, and
cancel SPARC after SPARC V (circa 2009), it is key that Solaris has
always maintained a little endian x86 port.  The Solaris kernel might
not be able to go GPL, but GNU/Solaris is a definite possibility.

[ Frankly, I think if Sun can stop its hemorraging, they should merge
with Red Hat.  I'd love to see Sun's "One" authentication-directory
system be GPL'd by Red Hat, providing a _real_ end-to-end, 100% standards
based solution for Linux -- outside of ActiveDirectory and eDirectory,
which have clear vendor tie-ins. ]

> Does this perhaps explain M$'s recalcitrance when it comes to 64 bit
> processors?

Yes and no.  Even AMD's x86-64's "long" mode still has no alignment
requirement, for full compatibility.  But that will probably not last.

But yes, in the case of Intel IA-64 EPIC, yes, there is a major issue.

> The world has been needing 64 bit CPU's in mainstream PC's for
> several years now - but we don't seem to be getting much closer to it.

Intel believed IA-64 EPIC/Predication would be the bomb that killed 32-bit
5 years ago.  Unfortunately they were wrong, and EPIC/Predication's
reliance on compile-time optimization failed, just like Digital Semi-
conductor engineers who worked at Alpha said it would.

In the meantime, AMD continued to stick with a single, x86 focus, and
extension.  But they went beyond that.  The integrated northbridge
and I/O memory management unit (MMU) on the Athlon64/Opteron goes well
beyond just 64-bit capabilities.  It addresses serious I/O throughput
issues in the PC server platform.

Intel's "Yamhill" now known as EM64T still doesn't compare, and won't
for another 2-years as Intel scrambles to produce another 18-month
"interim" Pentium design.  Pentium 6 will finally bring them on the
same footing from a serverstandpoint.

> I have an application that would really benefit from >4Gb of main
> memory - which essentially requires a 64 bit CPU.

Yes.  Xeon/Athlon32 PAE36 uses "paging" to swap in 512MB "pages" under
4GB (upto 64GB), just like EMS did with 64KB "pages" under 1MB (upto
32MB).  It's a hack that performs _horrendously_.

So it's x86-64 or IA-64.  EPIC Megagames and idSoftware went with
x86-64 18 months ago to develop their nexgen games, and most
engineering firms did the same.

> That's no problem with Linux - but it's becoming noticable that M$
> are locked into the4Gb limit in the same way they were locked into
> the 640Kb limit back in the 1980's...

Depends.

Win64 includes WoW (Windows on Windows) to run Win32, just like Win32
included WoW for Win16 via the NTDVM (NT DOS Virtual Machine) on NT.
MS-DOS 7 (Windows 95/98/ME) actually did some really "bad things"
to get the same effect (which is why stability was crap v. NT).

Unfortunately, Win64 WoW only runs Win32, but _not_ Win16.  That means
a lot of Visual Studio 6 (and earlier) code (especially VB6) won't run.
Microsoft's own tool developers have a _nasty_ habit of allowing such
"hacks," instead of forcing people to move to the full API.

Not just ISV (independent software vendor) developers, but Microsoft's
own applications division is guilty of this!

A lot of it had to do with the fact that "Chicago" (MS-DOS 7) and
"Daytona/Cario" (NT3.50-4.0) did _not_ unify in their Win32
implementation until NT 5.0 (2000).  Where they differed, Gates
sided with his cash cow -- "Chicago."

And a lot of promised "Cario" technologies did not make it into _any_
NT version, past for present.  Furthermore, a _lot_ of the security
model in Win32 was _never_ enforced in Visual Studio tools.  This
has resulted in a lot of consumer applications that did _not_ run
on NT 5.0 (2000), requirement Microsoft to "crack" the model in
NT 5.1 (XP/2003).  That's why XP is the most "secure" _consumer_
release of Windows -- but is _less_ secure than NT before it.

> with nasty bank-switching kludges going on in 32 bit x86 rather
> than the 'obvious' switch to a wider CPU.

PAE36 is a major performance killer.  And everyone has said it is,
hence why EM64T now exists for Xeon.  Eventually it will be supported
in the Pentium 5 later in the year (or next year) on Socket-775.
It is disabled (by solder) in the current Socket-775 chips.

EM64T still doesn't have the same level of features, like the I/O
MMU, and other missing x86-64 functionality that AMD's x86-64 chips
have.  The Linux Kernel Mailing List (LKML) talked about this a few
months ago -- and the changes/hacks required for EM64T to work.

> Could WORD be the boat-anchor that's causing that drag?

According to MS Office for Mac developers, yes.

The rumor I heard is tht Microsoft has addressed it with MS Office 2003
by using "binary XML."  In a nutshell, they _have_ come up with a XML
format and "compile" it into well-defined byte-code for MS Office
2003+.  This at least addresses the data-alignment issues for the
future.

Of course, the XML format is not published with DTDs, schema and other
meta-data.  XML itself is not a standard, but a framework so vendors
can build their own standards.

The current XML export in MS Office apps is content-only.  The XML
import in MS Office apps is for allowing 3rd party interchange.  In
other words, XML in MS Office is designed to allow 3rd parties to
write to an XML spec for MS Office, so it can interact, but MS Office
itself does _not_ use those specs for its own file format.

This is in stark contrast to the OASIS Open Office XML standardization
by Boeing, Corel, Sun and others.  OpenOffice.org (OOo) is also LGPL,
so Microsoft is _free_ to create full import/export 100% royalty-free.
You won't see them though, because their XML strategy is marketing and
interface only -- with MS Office 2003 as the _required_ back-end.


-- 
     Linux Enthusiasts call me anti-Linux.
   Windows Enthusisats call me anti-Microsoft.
 They both must be correct because I have over a
decade of experience with both in mission critical
environments, resulting in a bigotry dedicated to
 mitigating risk and focusing on technologies ...
           not products or vendors
--------------------------------------------------
Bryan J. Smith, E.I.            b.j.smith at ieee.org