[NTLUG:Discuss] Filesystem Performance

[David Neeley]

A week or so ago there was a discussion on the Internet with Hans Reiser. 
He pointed to some white papers on file system comparisons and performance 
issues...but these are all comparisons of various Linux file systems (see 
Reiser's www.namesys.com).

As far as comparing file systems between Windows and Linux--for it to be a 
valid comparison of file systems themselves, you would have to eliminate 
the differences between any Windows version and Linux.

I believe that most people's experience with the two is that on any given 
piece of hardware, Linux is generally faster under any load conditions (and 
there are various comparisons between the operating systems themselves out 
there).

For a very large datastorage system, you should pay particular attention to 
the multi-processor comparisons. Scalability is definitely an issue here.

If you want a comparison of file system performance, ask your "suits" a 
simple question: If Windows file systems are both fast and robust, why 
isn't Windows used for SANs? 8:)

There are many other considerations--indexing the files becomes a major 
pain when you are storing really large numbers of files. Depending upon 
your application, it might behoove you to examine storage of the files in 
an object or object/relational database.

When you consider robustness, I think you must examine journaling file 
systems like ReiserFS and the others available on Linux such as XFS. These 
are very crash-resistant and are often far faster than non-journaling 
systems like FAT or NTFS for that matter.

Finally, if you are considering speed, there is really no single-disk 
solution that begins to compare with a good RAID system. Today, you can use 
multi-channel Ultra SCSI 160 controllers with corresponding drives and then 
optimize for fast accesses (small files), fast transfer (large files), or 
the best compromise. This also adds a layer of protection from failures. 
(Those who want increased protection from problems often supply separate, 
supplemental power to the RAID subsystem so drive operations can be 
concluded even if the rest of the system loses power...and the really 
serious folks employ mirrored systems to give full redundancy...but I digress).

In short, it clearly is more difficult than just to compare the file 
systems for performance alone when there are so many other 
issues--operating system efficiency, robustness and performance; retrieval 
requirements; data security; data accessibility--all of these factors must 
be considered for a complete solution together with cost/benefit analysis.

Sorry for the "stream of consciousness" response, but your question really 
can be complicated to answer properly when you give so few details of what 
your requirements are.

Based upon my own informal tests, I am presently running Reiser on my Linux 
system--a dual boot machine also containing Win2k on NTFS. There is no 
comparison--in terms of speed alone, the Linux side is much faster than the 
Windows.

I hope this helps!

David