[NTLUG:Discuss] Distributed processing

[Stephen Davidson]

Greetings.

I wonder if Javelin by ObjectDesign would be worth considering?
It is not free, but my understanding is that it is not that expensive. 
Basically, it is a sophisticated object/data cache.  The implementation
is quite interesting, and highly proprietary (this is where they make
their money), but the general idea is that there is a replicated cache
accross the hosts.  The cache can store JavaObjects and/or Database
data, or anything else that you can get into a computer's memory.  Data
is not duplicated in all of the caches, only the ones that need it.  Say
machines A & D need a given piece of data, with machine C acting as a
"failover" machine.  Then A, C, and D would have the data, but none of
the other machines in the cluster would receive a copy of the data,
unless they specifically request it.  The software is extremely
efficient, and has low processor/system overhead.

Just a suggestion.

-Steve


Greg Edwards wrote:
> 
> cbbrowne at hex.net wrote:
> >
> > On Wed, 18 Jul 2001 21:34:28 CDT, the world broke into rejoicing as
> > Greg Edwards <greg at nas-inet.com>  said:
> > > Each host would be assigned a set of functions that it would support and
> > > not a set of applications.  This does not fit the load balance or
> > > clustering model very well.  Some hosts could act as database servers,
> > > some in a Beowulf cluster, some as game tree processors, some as search
> > > engines (these are just examples).  Not every host would support every
> > > request that could be generated.  This makes for a poor fit for a strict
> > > message queue solution as you don't want to send the queue to hosts that
> > > are non players in the logic thread.
> >
> > I would object _slightly_ on the message queue comment; it would be quite
> > reasonable to have a set of queues named suitably for better sharing...
> > I'm not sure what your exact objection is here, but would speculate it's
> > of two things:
> >
> 
> >
> > Sounds fairly suited for the message queueing approach, albeit not as
> > an automatic result of the messaging scheme, but rather as an aspect
> > of the design of the work process.
> > --
> 
> A message queue solution would certainly work but it would not be
> efficient for what I'm looking at.  I envision several hundred functions
> distributed throughout the farm with many functions capable of retaining
> results for an aged period.  Trying to maintain response factor
> heuristics with message queues would (IMO) flood the backbone with more
> traffic than is necessary.  I believe that maintaining heuristics on the
> response from the current host to the host that processed the request
> using an open hash would be simpler to maintain and a considerably
> lighter load on the network.
> 
> Consider a multiplier game spanning a game board with 64 grids and 15
> players currently in the game.  Host A is currently the game manager.
> Hosts C and D are responsible for determining which players are visible
> to each other.  Players 1, 5, 9, and 10 are each in grid 32.  Hosts C
> and D currently have the same response factor (as known to Host A).
> Player 1 gets processed by host C and player 5 by host D.  All other
> things being equal host C and D provide equal response factors on the
> returned results.  Now player 9 comes along and host C gets the
> request.  Since host C and D currently both have the search results from
> the 1 and 5 players cached the performance factor of host C becomes
> better than D (using simple averaging).  Now when player 10 comes along
> host C has already had player 9 results added to its data set and host D
> has not.  In this scenario you want host C to get the request and not
> host D.  In a typical message queue solution (first come first serve)
> host D has a better than average chance of ending up with the request.
> Yes we could get into weighting the queues by host response factors, but
> would that be worth the work.
> 
> For those thinking "hey this sounds like an OOPS solution".  I want it
> to be a procedural solution for process control reasons.  I want to be
> able to backup and request from a different host if I receive a
> timeout/non-response on any requests.  I also do NOT want to pass the
> entire data set between functions (hosts).
> 
> --
> Greg Edwards
> New Age Software, Inc.
> http://www.nas-inet.com
> _______________________________________________
> http://www.ntlug.org/mailman/listinfo/discuss