beginners questions/rent computing time

[m***@yahoo.com]

Hello,
it happens from time to time that I have to run some cpu intensive
processes in the order of 1000 CPU hours (estimated from my single 2
GHz Pentium). Most of these tasks can do without interprocess
communication similar to the boinc network (http://
boinc.berkeley.edu/).

Does some distributed computing network exist, where I - as a private
person without university affiliation - can have these 1000 CPU hours
run within 1 hour? What is the approximate cost? Ideally payment would
be done with PayPal, no long registration procedures involved.

And now for two very basic, more technical questions:

How is the job management done in such an environment? I have heard of
a globus toolkit. Because I assume that most networks will be
heterogenous (at least some nodes will probably have larger memory and
faster CPUs than others), I wonder how you can guarantee that your job
runs everywhere. Is it common to write the jobs in Java in order to be
system independent? Or as system independent C/C++ source code, which
is compiled on each node at the beginning of the job execution? You
see, you could help me with a link to a distributed computing
introduction...

Secondly, mostly for curiosity, I'd like to have low-cost access to a
multiprocessor cluster with Gigabit ethernet interconnect to teach
myself the MPI concepts. Since I expect very little CPU time
consumption, but frequent short logins during learning, it is fine if
the cost is ~1$/h/CPU or even more as long as there is no high per-
session-fee. Is such a learning platform available for rent for
private individuals?

Thanks
Mark

[Henry]

Post by m***@yahoo.com
Hello,
it happens from time to time that I have to run some cpu intensive
processes in the order of 1000 CPU hours (estimated from my single 2
GHz Pentium). Most of these tasks can do without interprocess
communication similar to the boinc network (http://
boinc.berkeley.edu/).
Does some distributed computing network exist, where I - as a private
person without university affiliation - can have these 1000 CPU hours
run within 1 hour? What is the approximate cost?

There was a lot of hype a while back for Grid computing
solutions providing CPU time on a commercial basis - I believe
IBM, Avaki and United Devices all offered this at some point.
Not sure if they still do - they may only be interested in much
larger scales than yours.
IIRC World Community Grid is actually a "free" instance of a
commercial project.
I've not managed to work out what they're actually trying to sell
at http://www.distributedpotential.com/

Depending on where you are and what you're doing there may
be other options - e.g. in the UK the NGS used to be willing to
consider proposals from private individuals. Or you could start
your own BOINC or WCG project...

You could look at
http://en.wikipedia.org/wiki/Grid_computing
http://en.wikipedia.org/wiki/Cloud_computing

Post by m***@yahoo.com
Ideally payment would
be done with PayPal, no long registration procedures involved.

I doubt anyone is going to let you upload and run your own
code on their machines without them making darn sure they
know who you are...

Post by m***@yahoo.com
How is the job management done in such an environment? I have heard of
a globus toolkit. Because I assume that most networks will be
heterogenous (at least some nodes will probably have larger memory and
faster CPUs than others), I wonder how you can guarantee that your job
runs everywhere.

Usually you specify the requirements of your jobs (e.g. memory
needed) using a mechanism such as classads/JDL, and the
framework distributes your jobs only among suitable nodes.

Post by m***@yahoo.com
Is it common to write the jobs in Java in order to be
system independent? Or as system independent C/C++ source code, which
is compiled on each node at the beginning of the job execution?

The Java route certainly works. Compiling in-situ is tricky (how do
you know what the compiler's called?) and while it could work for C
or F77, there seem to be a lot of C++ compiler version issues, at
least in the gcc world.

Usual approach seems to be a statically-linked executable with
matching requirements for OS and CPU architecture.

Bear in mind this stuff will work best with "simple" programs
that just read some input files and dump the output; if you want
to monitor progress or get user input then the whole thing gets
much trickier.

Post by m***@yahoo.com
see, you could help me with a link to a distributed computing
introduction...

http://en.wikipedia.org/wiki/Distributed_computing ?

Hth

Henry

[Chris Wilk]

Hello Mark,

Post by m***@yahoo.com
Does some distributed computing network exist, where I - as a private
person without university affiliation - can have these 1000 CPU hours
run within 1 hour? What is the approximate cost? Ideally payment would
be done with PayPal, no long registration procedures involved.

Take a look at Sun Grid <http://www.network.com> and see it if meets
your requirements.

Best,
Chris