web farm -newbie

[Madison Kelly]

Jose wrote:
> Hi List,
> 
> I am looking to learn how to setup a web farm, I have been reading 
> reviews about some books like:
> 
> Linux Enterprise Cluster by Karl Kopper
> 
> High Performance Linux Clusters with OSCAR, Rocks, OpenMosix, and MPI
> 
> Building Linux Clusters  David H.M. Spector
> 
> Linux High Performance Clusters [Import] (Paperback)
> by Alex Vrenios (Author)
> 
> I got a couple of computer I rescue from the job dump and I would like 
> to build a mini cluster, but not sure where to start, could you guys 
> advice a newbie on this matters?
> 
> thanks in advance
> 
> J

There are many types of "clusters", so the first question you need to 
answer is "what kind of cluster" do you want to build?

The cluster I have is a modest "HA" (high-availability) cluster. It 
provides no performance benefit (in fact, it arguably slows things down 
a touch), but it servers very fast "fail over" in the case of the master 
node failing. This is probably the simplest cluster to build and can be 
built on modest hardware. It's goal is "+9" reliability. That is, it 
adds a '9' to your uptime.

There are N+1 clusters, which provides load balancing with hot spares 
(very much like RAID 3/4/5). In this cluster, you have the load spread 
out over X number of nodes who share, generally, a common data store, 
with (at least) one more node than you need to maintain performance. 
This is the logical progression from the first type, but requires 
increasingly fancy hardware.

Web clusters generally use a load balancer that keeps track of multiple 
web servers and assigns incoming requests to the give node with the 
lowest load. These generally need to have their webserver nodes 
connected to a common database, but this should be pretty simple given 
that most RDBMS support network connections. This sounds like the type 
of cluster you are interested in, given your email's subject. This 
should be do-able relatively easily with commodity hardware. You may 
even want to use two machines on a HA cluster for the load balancing and 
backend DB and then a bunch of independant webserver nodes for handling 
client requests.

Then there are render-farms and many, many other types of "farms" out 
there. In the end, a "cluster" is much like a "server"; it defines a 
type of service more than anything else.

To get started, take a look at http://linux-ha.org. It's were I went to 
get started when I needed to build my cluster.

One last note; when it came to clustering the email, I had to roll my 
own back-end schema for Postgres for my Postfix installation. Cluster 
stuff is generally very individual, so exact implementations are 
sometimes hard to come by.

Hope you have fun! I did. :)

Madi
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists