Boinc Linux cluster How-To??


log in

Advanced search

Questions and Answers : Unix/Linux : Boinc Linux cluster How-To??

Author Message
Anders
Send message
Joined: 15 Sep 02
Posts: 2
Credit: 10,837
RAC: 0
Sweden
Message 761540 - Posted: 1 Jun 2008, 2:26:08 UTC

I have been googleing around for a How-to guide on how to set up a Boinc Cluster on linux machines but i havent really found any, is there any??

I found http://www.boinc-wiki.info/Creating_a_diskless_cluster
but it gets wery vague at the end.

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13620
Credit: 30,561,269
RAC: 21,178
United States
Message 761542 - Posted: 1 Jun 2008, 2:30:13 UTC

BOINC doesn't support clustering as it defeats the purpose of using spare CPU cycles of standard machines. The entire BOINC software, as well as each science app would have to be rewritten to support clusters. If you're interested in doing it yourself, BOINC and SETI are both open source.
____________

Anders
Send message
Joined: 15 Sep 02
Posts: 2
Credit: 10,837
RAC: 0
Sweden
Message 761546 - Posted: 1 Jun 2008, 2:40:45 UTC - in response to Message 761542.
Last modified: 1 Jun 2008, 2:43:36 UTC

BOINC doesn't support clustering as it defeats the purpose of using spare CPU cycles of standard machines. The entire BOINC software, as well as each science app would have to be rewritten to support clusters. If you're interested in doing it yourself, BOINC and SETI are both open source.



I have found many users on this forum doing this so i dont think its an imposibility.

I have allways seen the purpose of BOINC is to help with research and in the end make this world a better place, not a way "of using spare CPU cycles of standard machines"

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13620
Credit: 30,561,269
RAC: 21,178
United States
Message 761548 - Posted: 1 Jun 2008, 2:49:22 UTC - in response to Message 761546.
Last modified: 1 Jun 2008, 2:52:02 UTC

BOINC doesn't support clustering as it defeats the purpose of using spare CPU cycles of standard machines. The entire BOINC software, as well as each science app would have to be rewritten to support clusters. If you're interested in doing it yourself, BOINC and SETI are both open source.



I have found many users on this forum doing this so i dont think its an imposibility.

I have allways seen the purpose of BOINC is to help with research and in the end make this world a better place, not a way "of using spare CPU cycles of standard machines"


No user that I've ever seen on these boards is using BOINC in a cluster because no one has been able to rewrite the software to do it, but they do use a network of standalone Linux machines, if that's what you mean by 'cluster'. Many help request have asked how to do a Linux cluster or Windows cluster, but BOINC simply isn't programmed to work that way. It is currently designed to run on standalone machines networked together.

The purpose of BOINC is to provide an infrastructure for under-funded science projects to utilize all the spare CPU cycles in the world (which have the BOINC client installed) to be put to use. The theory is that all the CPU's spare cycles would amount to more power than a supercomputer.

All this is with the intent of making the world a better place, but even David Anderson (Lead BOINC developer and creator) will tell you that the entire concept is to use the spare CPU cycles that would otherwise go to waste sitting idle.

Profile Slasko
Send message
Joined: 3 Jun 01
Posts: 4
Credit: 997,257
RAC: 444
Denmark
Message 768187 - Posted: 14 Jun 2008, 18:12:33 UTC - in response to Message 761548.

BOINC doesn't support clustering as it defeats the purpose of using spare CPU cycles of standard machines. The entire BOINC software, as well as each science app would have to be rewritten to support clusters. If you're interested in doing it yourself, BOINC and SETI are both open source.



I have found many users on this forum doing this so i dont think its an imposibility.

I have allways seen the purpose of BOINC is to help with research and in the end make this world a better place, not a way "of using spare CPU cycles of standard machines"


No user that I've ever seen on these boards is using BOINC in a cluster because no one has been able to rewrite the software to do it, but they do use a network of standalone Linux machines, if that's what you mean by 'cluster'. Many help request have asked how to do a Linux cluster or Windows cluster, but BOINC simply isn't programmed to work that way. It is currently designed to run on standalone machines networked together.

The purpose of BOINC is to provide an infrastructure for under-funded science projects to utilize all the spare CPU cycles in the world (which have the BOINC client installed) to be put to use. The theory is that all the CPU's spare cycles would amount to more power than a supercomputer.

All this is with the intent of making the world a better place, but even David Anderson (Lead BOINC developer and creator) will tell you that the entire concept is to use the spare CPU cycles that would otherwise go to waste sitting idle.


Hi

Is it possible to boot a cluster of linux machines from a server via PXE?
Can boinc handle that?

____________

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13620
Credit: 30,561,269
RAC: 21,178
United States
Message 768192 - Posted: 14 Jun 2008, 18:23:14 UTC - in response to Message 768187.

BOINC doesn't support clustering as it defeats the purpose of using spare CPU cycles of standard machines. The entire BOINC software, as well as each science app would have to be rewritten to support clusters. If you're interested in doing it yourself, BOINC and SETI are both open source.



I have found many users on this forum doing this so i dont think its an imposibility.

I have allways seen the purpose of BOINC is to help with research and in the end make this world a better place, not a way "of using spare CPU cycles of standard machines"


No user that I've ever seen on these boards is using BOINC in a cluster because no one has been able to rewrite the software to do it, but they do use a network of standalone Linux machines, if that's what you mean by 'cluster'. Many help request have asked how to do a Linux cluster or Windows cluster, but BOINC simply isn't programmed to work that way. It is currently designed to run on standalone machines networked together.

The purpose of BOINC is to provide an infrastructure for under-funded science projects to utilize all the spare CPU cycles in the world (which have the BOINC client installed) to be put to use. The theory is that all the CPU's spare cycles would amount to more power than a supercomputer.

All this is with the intent of making the world a better place, but even David Anderson (Lead BOINC developer and creator) will tell you that the entire concept is to use the spare CPU cycles that would otherwise go to waste sitting idle.


Hi

Is it possible to boot a cluster of linux machines from a server via PXE?
Can boinc handle that?


BOINC has no built-in mechanism for doing the booting or for hosting on a server to be identified by PXE clients. You'd have to set up a PXE environment and include BOINC as a runtime program manually.

Just a note about the terminology though (just to be clear), a network of workstations, be they Linux, Unix or Windows, is not necessarily a 'cluster'. A 'cluster' is when they are programmed to act as one large machine. Each machine working individually would simply be a workstation which you can have networked together, but you'd have to be careful not to call them a cluster as people would immediately think of the supercomputer idea as opposed to simply a bunch of workstations.

Profile Slasko
Send message
Joined: 3 Jun 01
Posts: 4
Credit: 997,257
RAC: 444
Denmark
Message 768206 - Posted: 14 Jun 2008, 19:19:03 UTC - in response to Message 768192.

Hi

Yes i am talking about a bunch of machines having there own operating system and there own BOINC client installed. I understand that BOINC can not run in a cluster environment.

I was wondering if it could be possible to have the main operating system with the BOINC client installed on 1 server, and then have like 10 motherboards with no harddrive, cd-rom, floppy.
Only motherboards with CPU, RAM and a network card.

Then the 10 motherboards could boot the operating system with BOINC from the one server via network. Thereby having 10 BOINC machines running.

Would this be possible?
____________

clive G1FYE
Volunteer moderator
Send message
Joined: 4 Nov 04
Posts: 1300
Credit: 23,054,144
RAC: 0
United Kingdom
Message 768284 - Posted: 14 Jun 2008, 21:46:10 UTC
Last modified: 14 Jun 2008, 21:52:41 UTC

This is something I am slowly investigating, and so far the Linux Terminal Server Project seems to be the way I will try to set it up.
I have all the hardware needed its just a case of instalation and configuring it all, and thats the steep learning curve Iv`e hit....
If the motherboards you have can boot using PXE that simplifies things a bit,
There is no limit to the number of boards that can be connected to the server
Otherwise you have to get a lan card that can have an eprom fitted to it and this will need to have the boot code written onto it, you have to look out for lan cards that have a dule inline socket on them,
And check to see if it can be enabled to work, this shoud be written on the box, this is where the eprom will fit, but BIG problem is finding an eprom writer, and learning how to use it, they cost a LOT of cash, BUT, you can also put the boot code on a floppy disk [we have all got them, ok it`s not a 100% diskless system but very close] that boot code starts up the lan card and fetches the minnamalist operating system from the network server [witch also needs to be set up and configured].
Everything has to run in ram so the less there is to load the better on old machines with limited ram capability {some of mine max out at a mere 512mb} and using swap over lan is either very slow or unreliable.
The motherboards cpu`s dont have to be the same, it just saves some setup time if they are, mine are anything i can get hold of for free [well almost]
Nowhere have I found a compleat and uptodate guide on how to do this and that is because there are so many possible variations of hardware and configuration that none can cover every aspect of the task, It is a matter of becoming your own system administrator and learning what makes Linux tick.
One day I know I get it done.
Hope this helps if just a little bit.
____________

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13620
Credit: 30,561,269
RAC: 21,178
United States
Message 768403 - Posted: 15 Jun 2008, 2:00:35 UTC - in response to Message 768206.

I was wondering if it could be possible to have the main operating system with the BOINC client installed on 1 server, and then have like 10 motherboards with no harddrive, cd-rom, floppy.
Only motherboards with CPU, RAM and a network card.

Then the 10 motherboards could boot the operating system with BOINC from the one server via network. Thereby having 10 BOINC machines running.

Would this be possible?


You would run into sharing violations. Each workstation needs to have its own local copy of BOINC, complete with their own workunits in order to run properly.

Dotsch
Volunteer tester
Avatar
Send message
Joined: 9 Jun 99
Posts: 2422
Credit: 847,738
RAC: 11
Germany
Message 768549 - Posted: 15 Jun 2008, 8:46:26 UTC - in response to Message 768403.

You would run into sharing violations. Each workstation needs to have its own local copy of BOINC, complete with their own workunits in order to run properly.

It should posible if each BOINC instance from each node is on his seperate NFS or Samba directory on the fileserver.
____________

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13620
Credit: 30,561,269
RAC: 21,178
United States
Message 768585 - Posted: 15 Jun 2008, 11:59:31 UTC - in response to Message 768549.

You would run into sharing violations. Each workstation needs to have its own local copy of BOINC, complete with their own workunits in order to run properly.

It should posible if each BOINC instance from each node is on his seperate NFS or Samba directory on the fileserver.


Yes, that would work.

Profile Slasko
Send message
Joined: 3 Jun 01
Posts: 4
Credit: 997,257
RAC: 444
Denmark
Message 768592 - Posted: 15 Jun 2008, 12:19:17 UTC - in response to Message 768585.

You would run into sharing violations. Each workstation needs to have its own local copy of BOINC, complete with their own workunits in order to run properly.

It should posible if each BOINC instance from each node is on his seperate NFS or Samba directory on the fileserver.


Yes, that would work.


Heh, This is not going to be easy i can hear :)

But surely someone must have achieved this?
____________

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13620
Credit: 30,561,269
RAC: 21,178
United States
Message 768623 - Posted: 15 Jun 2008, 13:44:04 UTC - in response to Message 768592.

You would run into sharing violations. Each workstation needs to have its own local copy of BOINC, complete with their own workunits in order to run properly.

It should posible if each BOINC instance from each node is on his seperate NFS or Samba directory on the fileserver.


Yes, that would work.


Heh, This is not going to be easy i can hear :)

But surely someone must have achieved this?


It's very probable that someone has done it, but most people I know use SSDs in place of hard drives, eliminating a single point of failure (the PXE server) for an entire farm of crunchers while reducing electrical use since SSDs use a lot less electricity than HDDs.

Profile Michael Buckingham
Volunteer tester
Avatar
Send message
Joined: 21 Aug 99
Posts: 4508
Credit: 2,676,597
RAC: 0
United States
Message 779019 - Posted: 5 Jul 2008, 13:37:15 UTC - in response to Message 768549.

You would run into sharing violations. Each workstation needs to have its own local copy of BOINC, complete with their own workunits in order to run properly.

It should posible if each BOINC instance from each node is on his seperate NFS or Samba directory on the fileserver.


Using VCS, I have succesfully ran BOINC as a service group in a 2 node cluster (Solaris 10) using veritas cluster.

Basically, the boinc installation was out on a SAN..during failover the disk group and virtual hostname was moved to Node 1 (or vice versa) and boinc was then relaunched on the failover node.

It defeats the purpose though...as only one node in the cluster can run the boinc software in any given time.

This might not be what you mean by cluster though...because VCS is really only for high availability...

____________


http://www.mikesbawx.org/photo/

Questions and Answers : Unix/Linux : Boinc Linux cluster How-To??

Copyright © 2014 University of California