[NTLUG:Discuss] Re: Transport Tool - opinions

[Greg Edwards]

Kyle Davenport wrote:
> I meant to say that my magic threshhold had not been reached yet.  I just
> checked on MonarchComputer.com and a minimal node would be:
> 
> UPS:   $50.00
> Motherboards: Asus NForce2 A7N8X Deluxe 333 FSB AMD DDR   $149.00
> Processor: AMD Athlon XP 2400+ .13 Micron   $135.00
> Memory: 512 MB DDR (400) TwinX512-3200LLPT Corsair   $175.00_____
> Total: about $500
> 
> (note I do not include storage in the node)  The price killer unfortunately
> is the ram (and we simply must have 2 sticks for the nForce2 ;), and you
> really need a GB ($375 instead).  Note the nForce2 already has 2 on-board
> 10/100Mb ethernet controllers.  If you could get a similar smp board for
> less than twice as much, it would be worth it too.
> 
> Your ascii art worked, but still doesn't tell me what the bottleneck might
> be.  If it's for database access, I would invest in fast storage and ram
> instead of a cluster.  Or are you trying to build your own Google?   They
> have 4000 linux servers and still growing.
> 
> Kyle
> 

The ASUS A7N8X non deluxe has 1 nic and is about $30 cheaper.  For nic 
performance I'd rather use a PCI card with specific drivers.  For 
general network access the onboard nic is fine but for server to server 
I want as much performance as I can get and the 3Com cards fill the bill 
nicely.  I've got an A7N8X and the drivers for onboard devices leave a 
little to be desired.  My parallel port is not working 100% of the time 
(about 60%) and locks up the system with seg faults if a device 
(printer) is activated during startup.  The USB is a little flaky and 
the sound just plain don't work.  ASUS has the nforce drivers for RH and 
SUSE but not Mandrake.  They say NOT to use the NVidia nforce drivers 
but I'd don't have any choice.  Maybe Mandrake 9.1 will have replacement 
nforce drivers that work with this board but 9.0 doesn't.

On to farm issues.

This is for my project Galactic Outlaw (see link in sig) which is a 
cross between a Role Playing Game, Simulator, and IRC.  Galactic Outlaw 
is just the first product for an ASP that I'm trying to build.

When I do an analysis of what internal resources are required to support 
a huge set of users accessing a variable number of applications in a 
thin client/server environment I come to the following conclusions.

1.  the gateway (web server) must be devoted to traffic control only
       no processing beyond simple page (and/or image) rendering
       all decision processing transferred to Farm servers via CGI
       as little interpreted processing as possible (binary CGIs)
       ?? embedded Apache modules instead of CGIs
       all graphics delivered from local storage
2.  the DB server(s) must be devoted to data service only
       no processing of application specific needs
       automated DB maintenance processes local to DB server
       SCSI is a must
       DBMS performance over features is primary
       no web server direct access allowed
       general staff user access limited
3.  the Farm servers must be load balanced and hot swappable
       processes must be optimized, no interpreted processes (binary)
       processes need to be modularized to individual task logic
       a master process control must exist to oversee request steps
       a distribution server needs to control task load balancing
       a master distribution server must exist to delegate request

A request is designated as the initial request to the server Farm while 
a task is an action within the server Farm needed to complete the 
request.  Generally 1 request and many tasks.  For example the user 
would ask to login to the application which would cause the web server 
to generate a request to the server Farm to validate the user.  The 
server Farm process that handles that request would start a task that 
would take the user name/password and validate it.  As part of the user 
login request another task may be started to check if the user needs to 
be issued a notice about their subscription status.  Once the entire 
request has been completed all collected information is assembled and 
passed back to the web server for rendering to the user.

Using the above example the traditional load balance approach would be 
to replicate the complete validation program over many servers and share 
data between servers.  This is a major inefficiency of resources.  My 
solution is to distribute tasks among servers.  Not every task would 
need to be replicated on every server but every server that has a given 
task can participate in the load balance for that task.

-- 
Greg Edwards
New Age Software, Inc. - http://www.nas-inet.com
======================================================
Galactic Outlaw        - http://goutlaw.nas-inet.com
   The ultimate cyberspace adventure!