Questions and Answers :
Unix/Linux :
Linux Beowulf Cluster
Message board moderation
Author | Message |
---|---|
Dave Send message Joined: 9 Nov 15 Posts: 8 Credit: 364,873 RAC: 0 |
Hello, I have a number of functioning retired servers all with dual E5-2697v3 CPUs (56 cores) and 64 GB of RAM, I had these machines running in a Linux Beowulf Cluster(I did this just for fun), and connected them to a 10GB switch, Will your Linux software function in a parallel multi-machine environment? I used MPICH and HYDRA. Kind regards, Dave |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
Hello, Yes. Maybe. As long as the cpus are visibly discrete one task will run on one cpu. It doesn't matter how many or few machines you have. It does effect the # of cpu tasks available. The maximum available is 100 per "machine". Wiki entry for a Beowulf cluster makes it sound like they are "loosely" connected. If so, there shouldn't be a problem. On the other hand, it mentions a central server and attached satellite systems. And it mentions paralleling libraries. To actually use it the way the cluster intended you may have to re-compile and/or re-code the source code. That code is available. So I think the only way you are going to know for sure is to "try it". Specifically, setup a stand alone Linux box and work with it until you get it working "right." Then attempt to transplant your experience into the Cluster. Or simply run all the machines as discrete machines. Since setting up a "repository" version of Linux can be tricky I think you should use the "Tbar All in One" with the instructions on how to install the dependencies. You need to not run a Linux release later than 18.4 if you use the "all in one." 18.10 depreciates some required libraries for the all in one. I am NOT a Linux guru. I dog paddle a lot. But I have installed the all-in-one on multiple machines and once you install the "repository" version of the Boinc Manager you either have to stay with it or re-install Linux before you can install the "all in one". I can't explain why but I can say I have watched threads with that kind of discussion. They just don't mix. Tom A proud member of the OFA (Old Farts Association). |
Dave Send message Joined: 9 Nov 15 Posts: 8 Credit: 364,873 RAC: 0 |
Thanks Tom, As I'm new to SETI I wasn't sure which OS's are supported. You're correct Bionic would have to be compiled to run on it. I'm no Linux expert either but enjoy playing around with it, plus I'm seeing a lot of these 1U rack servers being pulled out of production, loaded with RAM and the upper end of E5-2600 CPU family, functioning perfectly. I put three on a 10GB switch, loaded Ubuntu 17.XX server MPICH and got the thing to work. It was pretty impressive to see it execute across all the cores ! I was watching Bionic run on 56 core machine and was wondering how much faster it would run if written to run in parallel? Dave |
rob smith Send message Joined: 7 Mar 03 Posts: 22204 Credit: 416,307,556 RAC: 380 |
It would be HUGE re-write and re-validation of the code to get it work across parallel CPU cores - one would have to dig deep into the result synchronisation and sorting to produce correct results (those which validate against the existing single thread application). I'm not saying "impossible", just "very difficult", and probably wouldn't get you the x-fold increase in performance over running the existing applicaitons on x-number of independent threads. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
Dave Send message Joined: 9 Nov 15 Posts: 8 Credit: 364,873 RAC: 0 |
Thanks for the response Bob, I'm sure it's a big deal, not like porting excel to Linux. Somewhere I read SETI ran on a super computer and I confused it with BOINC, my bad. |
Tom M Send message Joined: 28 Nov 02 Posts: 5124 Credit: 276,046,078 RAC: 462 |
If I am understanding it right, the above is your fastest way to becoming productive on Seti or any other BOINC project. GPU's use parallelization of computation to out produce most cpu cores between 4 and 10+ times in terms of processing time. Even so, you should get some significant crunching done if you want to set them up as discrete machines. I am running dual e5-2690v2 on an ASROCK server motherboard. I have been pleased with the cpu crunching. Tom A proud member of the OFA (Old Farts Association). |
Dave Send message Joined: 9 Nov 15 Posts: 8 Credit: 364,873 RAC: 0 |
Thanks Tom ! |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.