clustering is SO AWESOME

[Madison Kelly]

Darryl Moore wrote:
> I wrote a script to automate the building of basic block device clusters
> using DRBD. I've started writing other scripts to build services on top
> of that. So far just NFS, by I plan to do MySQL, and others too.

Awesome, you have those up anywhere?

> I'm really impressed with DRBD so far, though I haven't put it into a
> production environment yet. Soon I hope.

I've been using DRBD for ~3y now. Just recently though have I switched 
to the new version and began playing with primary/primary mode.

> The only down side of DRBD is that only one machine is the master at any
> given time which means that the other one is idle and a waste of resources.

Not true any more! :)

> It is a good idea to give the slave a few other duties so that it
> doesn't ever get too bored. The other thing you can do is make your DRBD
> cluster doubled headed. I.E. have each machine be the master of separate
> resources and also be the backup for each other. I've recently updated
> my build scripts to do this, though I haven't tested it yet. As soon as
> I get my high availability SQL build scripts going I'm going to build a
> double headed NFS / MySQL cluster and take it for a spin.

Check out the new version. With a cluster-aware FS (I personally use LVM 
with locking), you have have both servers using the DRBD partition at 
the same time. Also useful are OCFS2, GFS and others.

> One thing to watch out for, regardless of how you build it, is that you
> don't load down the slaves during normal operations to such an extent
> that they will not be able to cope with the additional load in the event
> that the master goes down.
> 
> cheers and happy thanksgiving weekend,
> darryl

In my case, I've got dual CPU, quad-core Opeterons (total of 8 cores) 
with 32GB/CPU and bring up virtual machines on either server set to use 
a minimum of X resources and let them balloon out to Y (to use up the 
unused resources on each node). This way, when one node fails or is 
taken off line for maintenance, I know I have enough resources to run 
all VMs on the one node without wasting the resources available when 
both nodes are alive.

If you want any help/hints/whatever getting dual-primary running, let me 
know. I've bashed my head off this stuff enough... It'd be nice to help 
save someone else some of the hassle.

Madi

--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists