Was thinking of setting up a clustered computer using the "spare power" of an entire web-hotel using Linux and OpenMosix, my guess is that Boinc would think that every attached computer is a CPU and use it as such, but what will happen when one such CPU "dissapears" because it has been rented by a customer or become used otherwise? I don't have the hardware in place yet, that's why asking this question.

---

> but what will happen when one such CPU "dissapears" because it has been rented by a customer or become used otherwise?

I expect nothing much- no real difference compared to a multiple CPU unit releasing it for use by an appliaction as required.

---

Grant
Darwin NT

> Was thinking of setting up a clustered computer using the "spare power" of an
> entire web-hotel using Linux and OpenMosix, my guess is that Boinc would think
> that every attached computer is a CPU and use it as such, but what will happen
> when one such CPU "dissapears" because it has been rented by a customer or
> become used otherwise? I don't have the hardware in place yet, that's why
> asking this question.
>
>
I have tested boinc on a 4 node OpenMosix cluster system end of last year. It was very disappointing to me, because the core client processes didn't migrate. I expectet the system's behavior as a 8 cpu system ( 4 x P4 hyper threading). But there was only 2 cc processes, just like a standard P4 with ht. The other 3 boxes didn't nothing at all!

BTW, the classic seti@home client is running just fine on a OpenMosix cluster. All processes are migrating straight away, and it looks like 1 node is doing all the work.

---

> I have tested boinc on a 4 node OpenMosix cluster system end of last year. It
> was very disappointing to me, because the core client processes didn't
> migrate. I expectet the system's behavior as a 8 cpu system ( 4 x P4 hyper
> threading). But there was only 2 cc processes, just like a standard P4 with
> ht. The other 3 boxes didn't nothing at all!

Use more than 2 CPUs enabled in your settings?

---

Grant
Darwin NT

> Use more than 2 CPUs enabled in your settings?

You know what? I'm not pretty sure if i enabled the settings using more than 2 cpus. You caught me on the wrong feet, as we say in germany.
I guess i will have to test again.

rattelschneck

I did another test run on a really small OpenMosix cluster setup, just 2 hosts. This time i set the preferences to use 8 cpus, to be on the save side ;)

After downloading the official boinc client for linx and starting it, i've got the same result again. The core client process won't migrate. I did a snapshot, so if somebody is interested in this, please take a look here:
Boinc snapshot

A closer look at the processes exposed them as 'monkeys'.

The documentation says:
There are several reasons, why the process may not be eligible for migration in OpenMosix. If the process cannot be migrated, the reason is listed in the file /proc/pid/cantmove. Here is the description of some of the possible reasons:
* monkey: the application is using files as shared memory

For comparison purposes i did a another test run with the seti classic client on the same setup. Two command line clients has been started. As expected the second process migrated straight away. I did another snapshot as you can see here:

Seti classic snapshot

At the end of the day it looks to me that OpenMosix is not the way to go if it comes to Boinc.
If there are some other opinions on this issue, i would very much appreciate to hear about it.

---

Hmmm... Perhaps the common filesystem built into OpenMosix can be used, haven't experimented with it yet thou...
I imagine it can be somehow used to allow the processes to migrate, but the docus for it is somewhat vague...

---

I've been thinking about this same issue during the last days and here are more points about it:

- Does an openMosix cluster gives information about the number of cpus by adding all available cpus in the cluster? I mean BOINC uses some kind of system call to know how many cpus does the system have. Does an openmosix cluster patch this system call to report a different number of cpus? I think the answer will be no, and that's why boinc only launches 2 processes in your cluster.

- Another problem would be what happens with a disconnecting node. Of course it would be beautiful that boinc notices when some new cpus are added / removed, but I really don't think that the program is prepared to do such thing right now.

- Maybe is only the seti client that is using files as shared memory and that's why those 2 processes are not migrating to other cpus? Maybe with another projects it would work? Could this "files as shared memory" be the way that the boinc and seti applications communicate?

I'm not shure about it, but patching the boinc client (it is open source, you can do it! ;-) ) and allow it to launch more tasks than the number of cpus that it sees could be more or less straightforward.

How to deal with nodes that "leave" the cluster would be a more complicated issue. Of course, if the node is gratefully shouted down the crunching process would migrate into another working node, but then that node would had 2 crunching processes in it, which would not be appropiate due to competition for resources. Maybe the boinc core should examine the system every minute or so to know how many available computing cpus we have... Of course, that could be difficult if the system does not provide the exact number of processors with the system call I already mentioned. Maybe some more patches would be neccessary to check the real and actual number of processors on the cluster.

And of course if the communication between boinc and applications is always made by "files as shared memory" the problems would increase as they cannot directly migrate. Then, we would need to also patch the applications for boinc, which is by far more complicated, and even impossible, since not all of them are open source.

More thoughts about it? Would it be interesting to release a "special openMosix edition" for BOINC? Note that for "farmers" could be just a great idea. With cds like ClusterKnoppix make a BOINC farm would be a straightforward issue. And think about mainteneance... just one BOINC application. Just upgrading in the "master" node would be wonderful!

Greetings

Zerjillo

---

BOINC uses shared memory ...

---

> I've been thinking about this same issue during the last days and here are
> more points about it:
>
> - Does an openMosix cluster gives information about the number of cpus by
> adding all available cpus in the cluster? I mean BOINC uses some kind of
> system call to know how many cpus does the system have. Does an openmosix
> cluster patch this system call to report a different number of cpus? I think
> the answer will be no, and that's why boinc only launches 2 processes in your
> cluster.
>
> - Another problem would be what happens with a disconnecting node. Of course
> it would be beautiful that boinc notices when some new cpus are added /
> removed, but I really don't think that the program is prepared to do such
> thing right now.
>
> - Maybe is only the seti client that is using files as shared memory and
> that's why those 2 processes are not migrating to other cpus? Maybe with
> another projects it would work? Could this "files as shared memory" be the way
> that the boinc and seti applications communicate?
>
> I'm not shure about it, but patching the boinc client (it is open source, you
> can do it! ;-) ) and allow it to launch more tasks than the number of cpus
> that it sees could be more or less straightforward.
>
> How to deal with nodes that "leave" the cluster would be a more complicated
> issue. Of course, if the node is gratefully shouted down the crunching process
> would migrate into another working node, but then that node would had 2
> crunching processes in it, which would not be appropiate due to competition
> for resources. Maybe the boinc core should examine the system every minute or
> so to know how many available computing cpus we have... Of course, that could
> be difficult if the system does not provide the exact number of processors
> with the system call I already mentioned. Maybe some more patches would be
> neccessary to check the real and actual number of processors on the cluster.
>
> And of course if the communication between boinc and applications is always
> made by "files as shared memory" the problems would increase as they cannot
> directly migrate. Then, we would need to also patch the applications for
> boinc, which is by far more complicated, and even impossible, since not all of
> them are open source.
>
> More thoughts about it? Would it be interesting to release a "special
> openMosix edition" for BOINC? Note that for "farmers" could be just a great
> idea. With cds like ClusterKnoppix make a BOINC farm would be a
> straightforward issue. And think about mainteneance... just one BOINC
> application. Just upgrading in the "master" node would be wonderful!
>

A lot of things to think of I agree, have to take a good peek into the docus of both boinc and Omosix... Have a 5-system cluster at our computers-club, perhaps I should utilize it for a weekend or so to do some real "field-testing"...

If I do remember correctly the docs for Mos said something like that all those common filesystems were mirrored to all nodes, this means that if the common memory is on such filesystem it should be able to migrate to another node since it should have the same name, content et.c.

---

I wonder if this is something worth looking into in this matter...

---

> I wonder if this is something worth
> looking into in this matter...

The last day I posted a message here I found that patch, but I'm not really shure if it is what we are looking for. If I found time I'll try to check it.

Zerjillo

---

>
> More thoughts about it? Would it be interesting to release a "special
> openMosix edition" for BOINC? Note that for "farmers" could be just a great
> idea. With cds like ClusterKnoppix make a BOINC farm would be a
> straightforward issue. And think about mainteneance... just one BOINC
> application. Just upgrading in the "master" node would be wonderful!
>


Overclockix has boinc and folding on it.
http://overclockix.octeams.com/

I think at one time it ran openmosix. Not sure what it runs now.
Last time I used it, the folding@home client was set up for someone else's account, not sure what they have it set up for now but the config file is editable.

> Overclockix has boinc and folding on it.
> http://overclockix.octeams.com/
>

Better go for this.
This is OpenMosix ready.
Installation of the boinc client isn't a big issue.

---

> I wonder if this is something worth
> looking into in this matter...
>

Hmm, it could be worth looking deeper into it. I will have a try asap.

BTW, is there a way to rename your thread? Giving it a name like "Running BOINC on a linux cluster" could possibly catch more interested users.
What do you think?

---

> I've been thinking about this same issue during the last days and here are
> more points about it:

Very interesting thoughts, really. I'm in a bit of a hurry right now.
I will come back to it, later on.

---

> Better go for this.
> This is OpenMosix ready.
> Installation of the boinc client isn't a big issue.

Yep, I downloaded it last friday but I have no chance to test anything. By the way, I'm not sure if clusterknoppix directly implements "Migshm" (if with migshm the processes migrate, the changes to the boinc core would be easy, I think). Anyway, we could think about creating a derived clusterknoppix distro with just the minimum requirements to run boinc (delete some software that would not be useful, add migshm (if possible) & add modified boinc core).

If you have the chance to test cluster knoppix and patch it with migshm I will be very interested in your results.

Greetings

Zerjillo

---

>
> BTW, is there a way to rename your thread? Giving it a name like "Running
> BOINC on a linux cluster" could possibly catch more interested users.
> What do you think?
>

Sounds like a good idea to me, I can't do that myself however, but I'll bet there is an admin or two that can...

---

So, guys... any news about running BOINC on a Linux cluster?
I'd like to give it a try...

Thanks...

---

>
> BTW, is there a way to rename your thread? Giving it a name like "Running
> BOINC on a linux cluster" could possibly catch more interested users.
> What do you think?
>

Sounds like a good idea to me, I can't do that myself however, but I'll bet there is an admin or two that can...

Chiana, Pilot says, "you are the only one (except devs/moderators) that CAN change the title of your thread. You need to make a post, then "Edit" the post, to the left will be the title of the thread in the box. Edit the title and click submit and you're done.

Signed
the crew of Moya,

You trollup ;)