Parallel programming

[jing]

>> Does anyone have experience or can recommend a good resource for
>> high-performance parallel programming?
>>
>> Specifically, I have been asked to help one of my colleagues parallelize
>> a simulation.  Ideally I would like to be able to run it on many, many
>> Linux machines via Amazon Web Services or similar service, but it would
>> be great to maximize an arbitrary number of cores as well as
>> physical/virtual machines.  Thanks.

Just throwing in an extra option.....

There's a local research project (at the U of T) called Snowflock
(spawned into a company called GridCentric), which provides high-speed
VM forking across a local cluster (based on Xen).  The product they
have right now works really well for CPU-bound work.  I'm not sure
what state the technology is at for cross-sharing large amounts of
data or network traffic.

http://gridcentric.ca/technology.shtml

http://sysweb.cs.toronto.edu/snowflock

The idea of snowflock is to provide a high-speed VM-fork analogous to
unix fork.  You can fork off one or many machines simultaneously (at
millisecond spawn-and-run speeds) across a cluster, get them all to do
parallel computation, and join the results back at the master machine.
 The programming paradigm is very natural and similar to unix fork,
except you can take advantage of many physical machines transparently.

The guys working on this were in the same lab as me as graduate
students.  Really smart and nice guys.
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists