Best Linux Distro To Crunch With

Message boards : Number crunching : Best Linux Distro To Crunch With

Author	Message
dcappello Send message Joined: 3 Apr 99 Posts: 249 Credit: 164,807,921 RAC: 16,498	Message 1121515 - Posted: 25 Jun 2011, 17:32:17 UTC
	Hello all, With so many choices for linux - what is the opinion of everybody for the "best" distro to use to Crunch with? I am looking for easy, small, fast - be "nice" if I could use hardware if it was available (ie graphic cards). I am moving off windows vista, server 2003 - so I'll be doing a few machines. regards, ____________
	ID: 1121515 ·

skildude Send message Joined: 4 Oct 00 Posts: 8940 Credit: 36,334,717 RAC: 52,717	Message 1121524 - Posted: 25 Jun 2011, 18:40:51 UTC - in response to Message 1121515.
	Ubuntu seems to be the easiest to use from the word of mouth I hear. I prefer Mandriva. It all depends on how Hardcore you intend to be with your OS ____________ Proud member of TSWB. End terrorism by building a school
	ID: 1121524 ·

phud Send message Joined: 12 Oct 99 Posts: 1319 Credit: 34,994,639 RAC: 0	Message 1121525 - Posted: 25 Jun 2011, 18:43:58 UTC Last modified: 25 Jun 2011, 18:45:04 UTC
	kubuntu is nice, i prefer the KDE setup. also there is a Ubuntu Studio now... lots of toys with that one... takes alot longer to install than regular ubuntu/kubuntu. i'm not sure if cuda cards are supported on Linux yet.... ???? ____________
	ID: 1121525 ·

Khangollo Send message Joined: 1 Aug 00 Posts: 240 Credit: 35,045,504 RAC: 75,881	Message 1121531 - Posted: 25 Jun 2011, 19:02:33 UTC - in response to Message 1121525. Last modified: 25 Jun 2011, 19:03:10 UTC
	i'm not sure if cuda cards are supported on Linux yet.... ???? nVidia CUDA is fully supported for a long time and it crunches like mad. ____________
	ID: 1121531 ·

Bernd Noessler Send message Joined: 15 Nov 09 Posts: 88 Credit: 39,685,856 RAC: 57,293	Message 1121644 - Posted: 26 Jun 2011, 7:45:50 UTC
	There is no support for linux/cuda by seti@home. But there are two 64bit clients available. You have to install them as anonymous platform. The clients are: 1) v6.08 compiled by Crunch3r for non-fermi cards 2) v6.11 compiled by aaronhaviland for fermi cards You need a 64 bit distro to install them. Dont't expect the same performance from your cuda devices compared to windows. One reason is the default behaviour of boinc. It runs the client with a priority of nice 10. Only root is allowed to change that. You can change this by running boinc as root and do a renice (not really an option) running boinc with --no_priority_change (I've never tried that) changing the boinc source and recompile it (that's my way) If you want to compile the manager you need a distro with wxWidgets. If you like to run BoincTasks a distro with a virtualbox would be a good idea.
	ID: 1121644 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7107 Credit: 3,680,809 RAC: 962	Message 1121963 - Posted: 27 Jun 2011, 13:17:00 UTC - in response to Message 1121644.
	... Dont't expect the same performance from your cuda devices compared to windows. One reason is the default behaviour of boinc. It runs the client with a priority of nice 10. ... The process priority shouldn't be a problem. I've not noticed any performance problems on my systems running Boinc under defaults. Any speed comparisons numbers you have between the windows versions and the linux versions will be interesting. The linux port at the moment is a few versions behind the windows experiments being done by the Lunatics. I'm sure Aaron will do another compile as soon as there is something significant added. As for which distro to try... Whichever you like the look of! Try a few and see which you like the best. Myself, I use a mix of Mandriva, Mageia, Kubuntu, Ubuntu, and Gentoo. Occasionally also DSL, and Puppy. Performance is similarly fast for all of them. The main differences are look and feel and installation footprint. Gentoo is ultimately customisable to very fine system detail, but the cost there is in extra time needed to configure for that fine detail. Gentoo runs fine on defaults to get you started, but beware the temptation to tweak and tinker and to ultimately go fully customised! Boinc is available already set up for you on Mandriva, Kubuntu, Ubuntu and Gentoo. Likely also on Mageia. Not looked for the others. Usually, Boinc is set up as a system service, so you'll need to add your username to the group for Boinc to read the Boinc remote access auth file. That's already done for you by some (all?) installs. (...But let me know if you get tripped up by that.) Let us know what you find! Happy fast crunchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 1121963 ·

Volunteer tester Send message Joined: 9 Apr 02 Posts: 11987 Credit: 17,831,638 RAC: 57,946	Message 1121975 - Posted: 27 Jun 2011, 13:39:14 UTC - in response to Message 1121963.
	... Dont't expect the same performance from your cuda devices compared to windows. One reason is the default behaviour of boinc. It runs the client with a priority of nice 10. ... The process priority shouldn't be a problem. I've not noticed any performance problems on my systems running Boinc under defaults. ...and to confirm what Martin is saying, the Windows client also runs at the lowest priority supported. The process priority should not affect the performance of the science app itself unless you're doing a bunch of intensive stuff on your system in the first place.
	ID: 1121975 ·

Bernd Noessler Send message Joined: 15 Nov 09 Posts: 88 Credit: 39,685,856 RAC: 57,293	Message 1122003 - Posted: 27 Jun 2011, 15:15:03 UTC - in response to Message 1121963.
	On my systems the RAC showed the performance gain. Before the change to prio 0 it was ~20K, after the change ~26.5K
	ID: 1122003 ·

Volunteer tester Send message Joined: 9 Apr 02 Posts: 11987 Credit: 17,831,638 RAC: 57,946	Message 1122012 - Posted: 27 Jun 2011, 15:26:29 UTC - in response to Message 1122003.
	On my systems the RAC showed the performance gain. Before the change to prio 0 it was ~20K, after the change ~26.5K There's so many different variables when it comes to a system's RAC, that I don't think you can attribute that gain to the process priority alone. Server stability, wingman reliability, fluctuations from CreditNew, solar flares, wind direction are all factors in a system's RAC. Ok all but the last two are, but that's still enough to question using RAC as a barometer for system performance even on the same system.
	ID: 1122012 ·

Bernd Noessler Send message Joined: 15 Nov 09 Posts: 88 Credit: 39,685,856 RAC: 57,293	Message 1122022 - Posted: 27 Jun 2011, 15:47:58 UTC - in response to Message 1122012.
	That was measured over weeks and not the snapshot of a single day.
	ID: 1122022 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7107 Credit: 3,680,809 RAC: 962	Message 1122033 - Posted: 27 Jun 2011, 16:16:58 UTC - in response to Message 1122022. Last modified: 27 Jun 2011, 16:21:50 UTC
	That was measured over weeks... If you change the priority, do you see a step-wise change in RAC? I have a lightly loaded system running under defaults, and looking back over the last few months, for s@h the RAC has varied between 200 and 270. The system is on 24 hours/day, and no changes from my side. So that's over a 30% variation for normal working... Are you running a multi-CPU farm or is your 20k+ RAC from using CUDA? Aside: I keep to the default 'nice' priority so that foreground tasks get their gulp of CPU quickly rather than lingering to still take the same CPU time but over a longer delay. The Boinc tasks get to use the same spare CPU time over a 24 hour period just the same. Happy fast crunchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 1122033 ·

Volunteer tester Send message Joined: 9 Apr 02 Posts: 11987 Credit: 17,831,638 RAC: 57,946	Message 1122044 - Posted: 27 Jun 2011, 16:57:25 UTC - in response to Message 1122022.
	Most RAC changes don't happen over the course of a single day anyway. Even when measuring in weeks, server outages and wingman performance can still have an affect overall.
	ID: 1122044 ·

Bernd Noessler Send message Joined: 15 Nov 09 Posts: 88 Credit: 39,685,856 RAC: 57,293	Message 1122045 - Posted: 27 Jun 2011, 16:58:01 UTC - in response to Message 1122033.
	I was running 7 boxes. 2 of them had a 240 GT. That's my total RAC. Nothing was changed except the prio for the cuda processes. The effect of running prio 0 is different on standard tasks and shorty's. On shorty's it is much higher. Before I had crunching times > 10 min. and after 300-360s. Standard tasks were crunched in ~35 min. and now ~22 min. The CPU load (for cuda) on my Phenom II X4 965 went from 1.5 % to 3% with a midrange tasks. With shorty's it is much higher. Everything has it's price. :-) I think the reason for this behaiviour are the CPU processes. There are running 4 processes with a prio of nice 19 and one with nice 10. And that makes the response times of the cuda process longer as it is with prio 0.
	ID: 1122045 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7107 Credit: 3,680,809 RAC: 962	Message 1122148 - Posted: 27 Jun 2011, 19:53:29 UTC - in response to Message 1122045.
	I was running 7 boxes. 2 of them had a 240 GT. That's my total RAC. Nothing was changed except the prio for the cuda processes. ... The CPU load (for cuda) on my Phenom II X4 965 went from 1.5 % to 3% with a midrange tasks. With shorty's it is much higher. Everything has it's price. :-) I think the reason for this behaiviour are the CPU processes. There are running 4 processes with a prio of nice 19 and one with nice 10. And that makes the response times of the cuda process longer as it is with prio 0. Possibly so, and quite interesting. Making a few wild guesses here... It could be that although the total CPU time needed remains unchanged, using a higher priority means that the GPU gets serviced sooner (lower latency) by the CPU when needed and so you get better GPU utilisation. Perhaps that is something that should be tweaked for Boinc CUDA-intensive tasks?... I'll try a test when I get a chance. Happy fast crunchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 1122148 ·

dcappello Send message Joined: 3 Apr 99 Posts: 249 Credit: 164,807,921 RAC: 16,498	Message 1122595 - Posted: 29 Jun 2011, 4:27:30 UTC
	Thanks for the all the info - I fired up Fedora 15 and Ubuntu - ended up on Ubuntu - most easy to setup and get running with BOINC host stats here: http://setiathome.berkeley.edu/show_host_detail.php?hostid=6067896 (yes I didn't put it inside a url) I think next I may try something with a GPU in it - that host only has 8 Cores. thanks again for the guidance. regards, ____________
	ID: 1122595 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7107 Credit: 3,680,809 RAC: 962	Message 1122674 - Posted: 29 Jun 2011, 12:52:34 UTC - in response to Message 1122595. Last modified: 29 Jun 2011, 12:55:21 UTC
	Thanks for the all the info - I fired up Fedora 15 and Ubuntu - ended up on Ubuntu - most easy to setup and get running with BOINC host stats here: hostid=6067896 Linux-2.6.38-8-generic (yes I didn't put it inside a url) Yes, but I did :-) Wow! Is that the first Linux machine you've tried?! I think next I may try something with a GPU in it - that host only has 8 Cores. thanks again for the guidance. Go for it! And you're most welcome. Happy fast crunchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 1122674 ·

S@NL - eFMer - www.efmer.eu/boinc Volunteer tester Send message Joined: 7 Jun 99 Posts: 497 Credit: 110,916,936 RAC: 170,944	Message 1123433 - Posted: 1 Jul 2011, 17:03:45 UTC - in response to Message 1121644.
	If you like to run BoincTasks a distro with a virtualbox would be a good idea. Ubuntu runs BoincTasks with Wine. Wine is easily installed and BoincTasks runs as good as on a Windows machine. ____________ TThrottle Control your temperatures. BoincTasks The best way to view BOINC. Anza Borrego Desert hiking.
	ID: 1123433 ·

BakCompat Send message Joined: 30 Jun 00 Posts: 7 Credit: 4,150,297 RAC: 3,227	Message 1123577 - Posted: 1 Jul 2011, 22:31:56 UTC
	I've got one running on Ubuntu, command line ony, and another on Fedora, with full gui. I think in general, I'd go with Ubuntu for a dedicated cruncher. ____________
	ID: 1123577 ·

Khangollo Send message Joined: 1 Aug 00 Posts: 240 Credit: 35,045,504 RAC: 75,881	Message 1123582 - Posted: 1 Jul 2011, 23:04:28 UTC
	Distribution doesn't really matter, as long as you know how to properly install and configure it, has fairly recent libraries and kernel, is 64 bit and you can install the latest official drivers from nVidia (for newer cards) - for that you may also need some rudimentary development stuff (kernel sources, gcc, ld...) - check that when you are installing the distro. On a dedicated cruncher, you might want to run it without X server running (runlevel 3). When you want to run X, you can always just startx it. nvidia-smi -pm 1 ensures that nVidia kernel module stays loaded (put it at the end of /etc/rc.d/rc.local). It is actually possible to run two Fermi GPU tasks on a 512MB card! (I tried it on GT 440. It worked, but the card is just too slow and there were no real gains.) I am also running a script in background (as root), which bumps all S@H (and also E@H) cuda processes to -10 niceness so they get the CPU cycles they deserve. #!/bin/sh while true do sleep 15 for arg in `ps -Lo lwp --no-headers -p $(pgrep -f "(CUDA\|cuda)") 2> /dev/null` do renice -10 -p ${arg} > /dev/null 2>&1 done done ____________
	ID: 1123582 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7107 Credit: 3,680,809 RAC: 962	Message 1123891 - Posted: 2 Jul 2011, 13:42:24 UTC - in response to Message 1123582. Last modified: 2 Jul 2011, 13:44:08 UTC
	... I am also running a script in background (as root), which bumps all S@H (and also E@H) cuda processes to -10 niceness so they get the CPU cycles they deserve. That is all rather extreme stuff to squeeze out perhaps a very small speed boost. A default install works surprisingly well for efficiency. Note that all tasks at a higher priority than "nice 0" are low-overhead system tasks that only briefly use the CPU. Encroaching into their territory means that you risk slowing down the system responsiveness. Thanks for some interesting tweaks there, but I would not recommend anything more than a "renice -5"... In my view, anything more than that just risks overall system operation for zero further gain for whatever it is you're trying to give highest priority. Then again, leaving Boinc to run at the default "nice 10" already grabs almost all of the CPU for an idle system. You can't use more CPU instruction cycles than exists, regardless of process priority! You can get better gains by minimising other system activity... But then again, a Linux system is pretty lean and mean to begin with... There's no continual anti-virus checking or file indexing or other "user experience" silliness to steal away the CPU time! To go really extreme is to go Gentoo customising... ;-) Happy fast crunchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 1123891 ·

1 · 2 · Next

Message boards : Number crunching : Best Linux Distro To Crunch With