I have been trying to find an answer to a question that I thought would be straightforward, but can't find an answer. Will BOINC/SETI run on a Beowulf style Linux Cluster?

I am looking at a good deal on some old server motherboards that have 4 AMD dual core Operton CPUs. They are really huge mothers that are 16 inches by 13 inches. Since they won't fit in ordinary affordable case, I was going to put them together in a MicroWulf stacked cluster like the link below.

http://www.calvin.edu/~adams/research/microwulf/

I think I can get about 64 GFLOPS and nearly 300,000 MIPS for a bit more than $500 total. I am wanting to run them in an Ubuntu cluster using MPICH/NFS/SSH so they will appear as a single entity.

Does anyone know if SETI will run on a Beowulf style Linux cluster by installing BOINC on the main system, or would I have to run multiple BOINC clients on each computer? The cluster theoretically would appear to the BOINC clients as a 64 core 64 bit AMD CPU.

I am also interested in this idea because I'm thinking of a project for batch of diskless thin clients I've got. Now I'm running SETI on them using their embedded WinXP, which is a bit of a pain to admin them individually. They use low power and a GFLOP each. As a Ubuntu cluster they will all be booting from the head of the cluster so there would be low power and no additional admin per client.

Thanks,
Greg Tippitt
team founder
STARFLEET - Star Trek Fan Association
http://boincstats.com/signature/-1/team/42609907/sig.png

Kinda pointless.

What is SETI@Home? A huge data file is broken down into smaller pieces, those pieces sent out to individual computing devices. The individual computing devices crunch the small pieces of data and return the result to the computer that is keeping track of the smaller pieces. Isn't that how beowulf cluster work?

---

... I am looking at a good deal on some old server motherboards that have 4 AMD dual core Operton CPUs. They are really huge mothers that are 16 inches by 13 inches. ...

I think I can get about 64 GFLOPS and nearly 300,000 MIPS for a bit more than $500 total. I am wanting to run them in an Ubuntu cluster using MPICH/NFS/SSH so they will appear as a single entity.

Does anyone know if SETI will run on a Beowulf style Linux cluster by installing BOINC on the main system, or would I have to run multiple BOINC clients on each computer? The cluster theoretically would appear to the BOINC clients as a 64 core 64 bit AMD CPU.

I am also interested in this idea because I'm thinking of a project for batch of diskless thin clients I've got. Now I'm running SETI on them using their embedded WinXP, which is a bit of a pain to admin them individually. They use low power and a GFLOP each. As a Ubuntu cluster they will all be booting from the head of the cluster so there would be low power and no additional admin per client.

OK, in brief:

Does that old kit cost out still when compared to what compute power you can get from a single GPU or from multiple more recent CPUs? Or even from a cluster of (a Brambleweeny of) RaspberryPi or BeagleBone or such as the $99 Linux Supercomputer...


I don't know of Boinc supporting Beowulf. You may well have to farm out multiple Boinc instances...

And as part of a thin-terminal-client setup, you should be able to include Boinc as a startup service or at least start it as part of the startup script.


All very good for experimenting and learning and fun. However, there are faster ways to crunch numbers!

Keep us posted for your crunchin' adventures!

Happy crunchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

... Isn't that how beowulf cluster work?

All a question of "granularity"...


s@h implements what is called "Embarrassingly simple" parallelism. The tasks to be run can be broken down into discrete units that can be executed in isolation with no compute-time interdependencies. Hell, it doesn't even matter if we're processing data 'tapes' from 1989 or 2019: Any useful data will be from hundreds or thousands of years ago in any case! (We may strike frightening luck if there is something closer!)


Beowulf includes scheduling and message passing to allow "more difficult" tasks to be speeded up by exploiting any parallelism possible.

And to set up something like that is a very good learning experiment in itself!


Keep searchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

Using Beowulf, or any of the other Linux cluster distributions, to admin a set
of boxes is a good idea. I have started going the other way. I had about 15
Linux boxes that I administered manually which didn't bother me too much. And
building each machine was fun. Then I started getting my power bill. The
increased power bill would easily buy a new hi end Nvidia every few months.
Now I have retired many of the old boxes and put in Nvidia boards that easily
beat your $500 price point. I plan to further reduce the number of computers
and try to get more efficient GPU solutions.

How many PCI cards can each of the old server motherboards handle. Some can
handle 4 at a time. One old server with 2 to 4 GPU cards would generate some
results.

Good luck with your project. Let us know what you put together.

Cheers!

---

I haven't tried it myself but I believe there are hooks to use BOINC as a fill-in process under a condor cluster. BOINC itself doesn't support MPI that most clusters use to communicate with worker nodes in the cluster. Jason is/was looking to doing something in this area (superdreadnought).

You could install BOINC on each and use BOINCtasks to monitor them however for OS and other software updates etc you'd probably have to SSH into each one which is what I do with my bramble every so often.

---

BOINC blog

I think your main limitation will be how much power the beast sucks in and how much that will cost you.

As a comparison, not including my Intel i7 2600K, the two nVidia video cards I have in my single PC has just a tad over 3000 GFLOPS, so well over 40 times the expected processing power of your proposal. Its total power draw is under 500W, so in theory uses around 12kWh's of power per day if it was running 24 hours a day, or around $5 per day with Australian power prices.

My guess is that the power draw from your proposed system will be at least 1,000W (1kW). So you have double the power draw, which means on a performance per watt basis my rig suddenly becomes 80 to 100 times more powerful than yours.

(BTW Im not saying mines better than yours, just that there are likely better ways of achieving what you want.)

If on the other hand you have deep pockets, and you have room for several video cards on the cluster then that changes the efficiency of your proposal greatly. Just be prepared for a large electricity bill. ;)

Cheers

Mark

---

BOINC uses shared memory. IIRC for the communication between BOINC and the science apps, but I might be off of which part uses shared memory.

A shared memory cluster solution often runs in the 5-6 figure cost range. Even for old used stuff.

Also something to consider. SETI@home has a limit of 100 in progress CPU tasks per host. So with 64 processors your cache would be < 1 day. While 8 separate machines would each be able to have a cache of 100 tasks.

As far as admining several clients. Are you taking advantage of the remote admin abilities in BOINC or using the interface on each machine?
Perhaps some other software like BOINCTasks would be a help to you. I use BOINCView to monitor my machines and then use the built in boinccmd to manage them.

---

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[

Thank you all for your replies. “Kinda pointless”, fairly much summarized it. I was wanting to know if the BOINC/SETI client would run on a Beowulf cluster. I didn't say I was building a cluster to run SETI, I said I was building a cluster and asked if I would be able to run SETI on it from a single BOINC client. I've been running SETI@Home since before BOINC, and am not completely clueless. I normally avoid this board, because the replies to questions are often more insulting that informative.

I know I can run SETI on the individual members of the cluster. My post said I was founder of a BOINC team and included my BOINCstats signature, so I didn't need to be told about BOINCstats. I know there are limits on how how much SETI work you can get, so I run a mix of projects to make sure my machines don't sit idle waiting for work, per BOINC recommendations. As far as GPUs, they are great at what they do, but they can't do everything. I'm planning to add Tesla C1060s to each of the motherboards as I can pick them up cheap.

“Kinda pointless”

Greg Tippitt
team founder 
STARFLEET - Star Trek Fan Association 
http://boincstats.com/signature/-1/team/42609907/sig.png

---

“Kinda pointless”

It depends on the point. Maybe just because.

---

I normally avoid this board, because the replies to questions are often more insulting that informative.

To get informative replies you need to be very specific in your questions.
You didn't say explicitly "I am building a cluster for other purpose" in your first post.

My post said I was founder of a BOINC team and included my BOINCstats signature, so I didn't need to be told about BOINCstats.

Wrong - Nobody told you about BOINCstats
You were told about BoincTasks


“Kinda pointless”

---

 


- ALF - "Find out what you don't do well ..... then don't do it!" :)
 

You may look at this:
http://en.wikipedia.org/wiki/LinuxPMI

---

 


- ALF - "Find out what you don't do well ..... then don't do it!" :)
 

BilBg, Peace and Long Life.

You are correct regarding the BOINCstats vs BOINCtasks. I'm dyslexic and sadly spell checkers help my writing, but not my reading.

Thanks for the link about LinuxPMI. I've read about OpenMosix and Kerrighed, which are both stagnant if not dead, but had not heard about LinuxPMI. I'll edit the Wikipedia articles for OpenMosix and Kerrighed and add a link to that page in the "See Also" section.

I like your ALF quote. I guess I tend to do the opposite, since I get bored doing something I already know how to do well. I enjoy looking for something I don't know anything about and seeing if I can do it. I worked as an application programmer for an insurance for 20 years. I started programming in 1977 on a Cyber supercomputer. It was the size of house, was cooled with liquid Freon, and had less capability than single AMD or Intel 64 bit core. I taught myself to program and mostly wrote dull applications. Now that I'm retired, I enjoy reading and tinkering with Linux. Lately I'm interested in clustering. When I get my cluster built and working, I want to start playing with my next interest, which is AI and language. I want to see if I can teach my cluster to read, listen, and talk like in the Turing Tests competitions.

I love the idea of the IBM HAL system on Jeopardy, but I think its capabilities were over-hyped a great deal. It had a tremendous unfair advantage over the humans in the way the questions were presented. The host read the clues to the humans, while HAL got the clues electronically. IBM cheated by not making HAL use voice recognition. Hal only used voice recognition to listen to the human contestants give wrong answers, so that HAL would not give the same wrong answer. The humans tried to figure out the question while the clue is being read, and the first to buzz in gets the opportunity to answer. By the time the host has read the first word, HAL had gotten the entire clue and was working on it. Before the humans had heard enough to understand the clue, HAL had decided upon a question. The humans only really had a fair chance when HAL would misunderstand the clue and come up with something stupid. If HAL had not gotten the clues faster than the humans, I don't think the machine would have had a chance speed-wise against the humans.


Thanks,
Greg

---