How To Do A "Headless Linux CLI Multiple GPU Boinc Server" Semi-Pro Cruncher ;)

Message boards : Number crunching : How To Do A "Headless Linux CLI Multiple GPU Boinc Server" Semi-Pro Cruncher ;)
Message board moderation

To post messages, you must log in.

AuthorMessage
DanHansen@Denmark
Volunteer tester
Avatar

Send message
Joined: 14 Nov 12
Posts: 194
Credit: 5,881,465
RAC: 0
Denmark
Message 1611866 - Posted: 10 Dec 2014, 20:04:25 UTC
Last modified: 10 Dec 2014, 20:08:53 UTC

Hello friends,


I'm writing this to find out if anyone would like to build a pretty good cruncher (rack mounted, headless & linux based) for what I think is not too expensive. Or maybe just use my ToDo to setup his/her own system to run using the same software solution.

I've been working and testing several computers in this last year, trying to make a headless boinc cruncher. A semi professional computer running Linux in CLI using CPU & multiple GPU's to crunch data. With some help from a couple of helpful guys inhere and elsewhere I've been able to solve the task. A 2U rack mounted computer with low profile graphic cards running Ubuntu Server 14.04.1 (Kernel: 3.13.0-32-generic).
There has been a lot of "issues" along the way, but by testing several mobo's, graphic cards etc. I've been able to build a stable system. Another problem was the conflict that appeared after the Ubuntu Server 12.04.4 update. Since the update, the debian pkg CUDA5.0 wouldn't work with Ubuntu Server any more. But I solved these issues as well.
There were a couple of guys back then, that showed interest for the project and this is why I'm writing this little post. If you are still interested in hearing about the project and how it went on, please let me know and I'll show it to you. If there's somebody else who is maybe interested in building such a system, please don't hesitate to let me know. I'll show you how to do it and post my ToDo inhere. I've spent maybe 6-8 times the cost of 1 finished computer system, and there's no reason for "you" to do the same. Maybe there's a couple of "newbies" like myself who would love to build and support SETI@Home and others. This is my contribution to SETI ;)

Thanks especially to: Mr. Ageless, Zalster, Gundolf Jahn, Bil, OzzFan, Draco, Arkayn who helped me when it was a total blur for me, putting the pieces together. Thank you for showing interest and not giving up ;)







Kind Regards,
Dan

.
Project Headless CLI Linux Multiple GPU Boinc Servers
Ubuntu Server 14.04.1 64bit
Kernel 3.13.0-32-generic
CPU's i5-4690K
GPU's GT640/GTX750TI
Nvidia v.340.29
BOINC v.7.2.42

ID: 1611866 · Report as offensive
Profile Woodgie
Avatar

Send message
Joined: 6 Dec 99
Posts: 134
Credit: 89,630,417
RAC: 55
United Kingdom
Message 1612674 - Posted: 12 Dec 2014, 8:42:05 UTC - in response to Message 1611866.  

Well I for one am very interested in how to get this up and running, I have the possibility of landing a few older Dell servers which I was hoping to turn into crunchers by the addition of reasonable graphics cards. I would rather run Linux than Windows on them as I'm a Mac man and feel I'll get on better with Linux :)
~W

ID: 1612674 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1612884 - Posted: 12 Dec 2014, 18:06:39 UTC - in response to Message 1611866.  

Hi Dan,
Happy to have been of help. Not sure what I did, as usually when I help on Linux problems it's part guess, part gamble, part looked up info. :)
I always think that if I am typing bollocks, that someone will tell me I did that. And at times people do. :)

If you want to leave the info in a reliable place, you can post it over on the BOINC forums. I'll make sure to sticky it then, so we can point others looking for such information at it.
ID: 1612884 · Report as offensive
HighTech67

Send message
Joined: 22 Jun 03
Posts: 20
Credit: 5,936,475
RAC: 126
United States
Message 1612891 - Posted: 12 Dec 2014, 18:27:01 UTC - in response to Message 1611866.  

I'm also interested.

Faster graphics cards would of course be better. I know nothing about rack cases. Do they make them taller so you could put full height cards in them?
ID: 1612891 · Report as offensive
Profile ivan
Volunteer tester
Avatar

Send message
Joined: 5 Mar 01
Posts: 783
Credit: 348,560,338
RAC: 223
United Kingdom
Message 1613238 - Posted: 13 Dec 2014, 10:13:35 UTC - in response to Message 1612891.  
Last modified: 13 Dec 2014, 10:17:13 UTC

I'm also interested.

Faster graphics cards would of course be better. I know nothing about rack cases. Do they make them taller so you could put full height cards in them?

Yeah, there are a number of different configurations. One of my Supermicros is only 1U high but it has a C1060 and a GTX460 (sans fan!) in it -- they lie flat so height isn't really an issue. Similarly with my latest 2U chassis -- it has two slide-in dual-Xeon systems, each of which can take a full-size PCI-e card; we have the same chasses in our Grid cluster with four smaller slide-in dual-Xeons, but they can only take low-profile cards (like the GT640 with appropriate hardware). One of the other groups in our Department tested an Intel system a while ago that could take eight (IIRC) PCI-e cards, standing up, but I don't remember the chassis size or whether they were full-height or low-profile slots.
ID: 1613238 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1613876 - Posted: 14 Dec 2014, 15:01:09 UTC - in response to Message 1613238.  
Last modified: 14 Dec 2014, 15:04:25 UTC

I'm also interested.

Faster graphics cards would of course be better. I know nothing about rack cases. Do they make them taller so you could put full height cards in them?

Yeah, there are a number of different configurations. One of my Supermicros is only 1U high but it has a C1060 and a GTX460 (sans fan!) in it -- they lie flat so height isn't really an issue. Similarly with my latest 2U chassis -- it has two slide-in dual-Xeon systems, each of which can take a full-size PCI-e card; we have the same chasses in our Grid cluster with four smaller slide-in dual-Xeons, but they can only take low-profile cards (like the GT640 with appropriate hardware). One of the other groups in our Department tested an Intel system a while ago that could take eight (IIRC) PCI-e cards, standing up, but I don't remember the chassis size or whether they were full-height or low-profile slots.

You can use 4U rack cases that have the same volume as a normal PC case. However, they are unusual due to rack space usually being precious and expensive.

I happen to have only two such 4U beasts so far... One is a dev system to accommodate full size PCIe cards, the other is for taking a silly number of TBytes of data and backups.

My preferred format is 2U for general use. 1U is preferred for compute nodes or for such as hosting websites/cloudy.


And then you can go Phoronix style!

Happy crunch in
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1613876 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 1614059 - Posted: 15 Dec 2014, 0:42:29 UTC - in response to Message 1611866.  
Last modified: 15 Dec 2014, 0:42:57 UTC

I'm writing this to find out if anyone would like to build a pretty good cruncher (rack mounted, headless & linux based) for what I think is not too expensive. Or maybe just use my ToDo to setup his/her own system to run using the same software solution.

I've been working and testing several computers in this last year...

Nicely neat looking rackmount box...


Any hints/tips/observations for assembling the parts?

Does Boinc play nice (bad pun) across multiple GPGPUs when also used for some other primary task?

Have you tried using using the GPU crunching from VMs by any chance? (QEMU + KVM?)


Happy fast crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1614059 · Report as offensive
Profile Woodgie
Avatar

Send message
Joined: 6 Dec 99
Posts: 134
Credit: 89,630,417
RAC: 55
United Kingdom
Message 1615793 - Posted: 18 Dec 2014, 15:02:48 UTC

Don't leave us hanging buddy!
~W

ID: 1615793 · Report as offensive
Profile Woodgie
Avatar

Send message
Joined: 6 Dec 99
Posts: 134
Credit: 89,630,417
RAC: 55
United Kingdom
Message 1625333 - Posted: 8 Jan 2015, 8:18:56 UTC

Today or tomorrow I should be being sent 3 x Dell PowerEdge 1950 1U servers and I was hoping to set them up running Linux.

I have a thread going on over here discussing what kind of graphics cards I could fit in them (it looks as if it has to be a one slot thick card and probably with the shorter PCIe x8 connector, we'll see).

Any thoughts on the matter? Also, as I said in a PM my offer of hosting your instructions (with full accreditation to you and the people who actually DID the work) on my website still stands.
~W

ID: 1625333 · Report as offensive

Message boards : Number crunching : How To Do A "Headless Linux CLI Multiple GPU Boinc Server" Semi-Pro Cruncher ;)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.