Setting up Linux to crunch CUDA90 and above for Windows users

Author	Message
Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1893887 - Posted: 7 Oct 2017, 18:01:28 UTC Sound like you used the default repository or SETI version which dumps BOINC down in VAR/LIB or whatever. To get down to see those directories, use a Terminal as SuperUser or root. In Terminal gksu nautilus will get you the file manager which can move you into the special permission directories where you can find the BOINC client and manager. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1893887 · Reply Quote

Brent Norman Volunteer tester Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835	Message 1893891 - Posted: 7 Oct 2017, 18:09:02 UTC - in response to Message 1893881. I do remember that you installed some form of Cinnamon/Mint desktop on to of Ubuntu - I thought you had completely given up on that whole install because of some issues it was having??? If the clock is on the bottom, it's likely some form of Mint. You should be able to find the BOINC Manager in the Menu. Click "Menu" and type BOINC in the search box. You should be able to open it there. If you Right Click the icon, you can place shortcuts in your launcher or on desktop. ID: 1893891 · Reply Quote

Stephen "Heretic" Volunteer tester Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628	Message 1893967 - Posted: 7 Oct 2017, 23:33:03 UTC - in response to Message 1893881. Last modified: 7 Oct 2017, 23:35:49 UTC ... I had changed the desktop to a different version that what came with Mint, can't remember the name, but it was one that someone had mentioned here I believe, which I was told looked/acted even more like Windows. ... It came up, but I then just noticed that with this new desktop, it didn't have an Icon to bring up BOINC, nor any icon in the tray at the bottom near the clock, so I presumed that it wasn't running. . . ... this one appears to have been sending out results since I rebooted it, so it does start BOINC with the system. Soooo... Here's my problem. I'd like to bring up BOINC, but I can't find the executable, and don't have a desktop icon to do so. I did a search on BOINC in file manager, it came up with a number of things, but when I right clicked on the desktop to create a new 'launcher', and tried to drill down to those directories that I had found with BOINC in them, once I get past lib and try to get into boinc-projects for example, it tells me that I don't have permissions. sigh . . ... would really appreciate some assistance in getting past the permissions issues, and then to point me to the executable so I can create an icon on the desktop. I'd also love to set one up there for the Nvidia X server as well, because for whatever reason, it doesn't remember after the reboot that I had changed the power mizer and thermal settings, so I need to go back in each time and reset them. Good thing is, this thing should _rarely_ need a reboot, as it is a dedicated, set and forget cruncher. Thanks for any help guys. . . Hi there Al, . . As Keith said, it seems you are using the repository version of BOINC, if so the stuff you need for BOINC is in the folder /var/lib/boinc-client, and remember Linux is case sensitive in folder/file names. As Keith also said, gksu nautilus will allow you to navigate to that part of your drive and do what you need. But there is also a command which will change the permissions for that folder allowing your identity to access it. "sudo chmod -R 777 /var/lib/boinc-client". That will make it easier in future. You will also find (I suspect) as I have done that if you need to stop BOINC for any reason you cannot restart it without rebooting. There is a command which will restart the BOINC client and get you back in business. This must be run from within the boinc-client folder "sudo service boinc-client restart" and will do the job a treat. For general info, as someone else explained, sudo mean "super user do". I find it helpful understanding that :) . . I recall a message suggesting that desktops for Adapta and Vertex were most Windows like. I am not sure as I am using plain old Ubuntu 14.04 but if there is a "help" option that usually will have an "about" choice that might tell you the details of your setup. . . Also in Ubuntu there is an "app find" thingy. If you have a similar thing (not the file browser) open it and enter BOINC in the search box. It should give you a list and then select BOINC manager to run it. With it open you will have an icon on the launch bar (task bar) which you can right click on and then select "lock to launcher". This should also work for Nvidia x-server. But to avoid the tedious manual resetting of both Thermal and power-miser options there are scripts that can be set to run at startup to achieve this. I cannot paste them here as I am on my Windows machine and the scripts on the Linux boxes. Maybe Keith can post copies of his scripts. There is one to provide continuous monitoring and adjustment of the GPU temps/fans and two more to select the performance mode and clocking. Set and forget :) . . I hope this helps. Stephen :) ID: 1893967 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1893970 - Posted: 7 Oct 2017, 23:59:55 UTC - in response to Message 1893967. But to avoid the tedious manual resetting of both Thermal and power-miser options there are scripts that can be set to run at startup to achieve this. I cannot paste them here as I am on my Windows machine and the scripts on the Linux boxes. Maybe Keith can post copies of his scripts. There is one to provide continuous monitoring and adjustment of the GPU temps/fans and two more to select the performance mode and clocking. Set and forget :) . . I hope this helps. Stephen :) I dislike tedium. So I run some scripts before I start BOINC. Don't have to restart the computer or BOINC very often. These are the scripts I use. You can tailor them to your configuration. You will have to add in your extra gpus. unrestricted.sh #!/bin/bash /usr/bin/nvidia-smi -pm 1 /usr/bin/nvidia-smi -acp UNRESTRICTED gpuoverclock.sh #!/bin/bash /usr/bin/nvidia-settings -a "[gpu:0]/GPUFanControlState=1" /usr/bin/nvidia-settings -a "[gpu:1]/GPUFanControlState=1" /usr/bin/nvidia-settings -a "[gpu:2]/GPUFanControlState=1" /usr/bin/nvidia-settings -a "[gpu:0]/GPUPowerMizerMode=1" /usr/bin/nvidia-settings -a "[gpu:1]/GPUPowerMizerMode=1" /usr/bin/nvidia-settings -a "[gpu:2]/GPUPowerMizerMode=1" /usr/bin/nvidia-settings -a "[gpu:0]/GPUMemoryTransferRateOffset[3]=0" -a "[gpu:0]/GPUGraphicsClockOffset[3]=40" /usr/bin/nvidia-settings -a "[gpu:1]/GPUMemoryTransferRateOffset[3]=0" -a "[gpu:1]/GPUGraphicsClockOffset[3]=40" /usr/bin/nvidia-settings -a "[gpu:2]/GPUMemoryTransferRateOffset[3]=0" -a "[gpu:2]/GPUGraphicsClockOffset[3]=40" /usr/bin/nvidia-smi -i 0 -ac 3505,1394 /usr/bin/nvidia-smi -i 1 -ac 3505,1394 /usr/bin/nvidia-smi -i 2 -ac 3505,1394 It looks like you have four 950 and one 1050. You can overclock the 950s but you can't do anything with the 1050 since it is a Pascal card. Maybe someday someone will figure out a way to get it into Performance Mode 3 in Linux. You will have to modify the Transfer rate clocks and the graphics clock overclocks to suit your cards. You can run this command in Terminal to see what your cards are capable of: "nvidia-smi -q -d SUPPORTED_CLOCKS" Pick a matching pair of clocks speeds for both video core clock and memory clock and plug them into the gpuoverclock.sh script. And finally, there is: GPUPriority.sh script. Courtesy of Petri and Brent. It sets the nice level and priority of both the CPU and GPU tasks. Main one is to run the special app in High Priority mode. #Run in root terminal, NOT sudo nvidia-smi -pm 1 for (( ; ; )) do # Assign CPU Priority (19=Nice/LowPriority, 0=Normal, -20=HighPriority) # This was code Petri gave out # GPU Tasks get high Priority schedtool -n -20 `pidof setiathome_x41p_zi3v_x86_64-pc-linux-gnu_cuda80` schedtool -n -20 `pidof astropulse_7.08_x86_64-pc-linux-gnu__opencl_nvidia_100` # CPU Tasks get (a little) Below Normal Priority (0 being normal) to make sure it doesn't choke the OS schedtool -n 5 `pidof ap_7.05r2728_bdver1_linux64` schedtool -n 5 `pidof MBv8_8.05r3345_avx_linux64` # Assign CPU Usage Threads (0-7) # Brent added this to Petri's code # Keep GPU tasks on threads 1 3 5 7 schedtool -a 1,3,5,7 `pidof setiathome_x41p_zi3v_x86_64-pc-linux-gnu_cuda80` schedtool -a 1,3,5,7 `pidof astropulse_7.08_x86_64-pc-linux-gnu__opencl_nvidia_100` # Keep CPU tasks on threads 0 2 4 6 schedtool -a 0,2,4,6 `pidof MBv8_8.05r3345_avx_linux64` schedtool -a 0,2,4,6 `pidof ap_7.05r2728_bdver1_linux64` # CPU Priority Assignment Script date # lscpu \| grep MHz sleep 5 echo " CPU Priority and Assignment Script (8 Threads)" done Just drop into Terminal as root and execute all the scripts in order. unrestricted, then gpuoverclock and finally GPUPriority. And leave the terminal open at the end to keep the GPUPriority script constantly running. Just minimize it to the Launcher. You need to keep it running so every new task is set to the desired run priority as it starts. Hope these scripts are found useful. Let us know how they work out if you use them. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1893970 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1893972 - Posted: 8 Oct 2017, 0:03:31 UTC Last modified: 8 Oct 2017, 0:06:39 UTC Forgot to mention you will have to adjust the app names in the GPUPriority.sh script to match your actual apps for MB and AP that you are using. [Edit] Also forgot to mention that the script also set affinity for both CPU and GPU tasks. Adjust accordingly to your processor. I have a AMD FX processor so I keep CPU tasks only on the physical cores and let the virtual cores handle the feeding of work to the GPUs. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1893972 · Reply Quote

Stephen "Heretic" Volunteer tester Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628	Message 1893980 - Posted: 8 Oct 2017, 0:38:21 UTC - in response to Message 1893972. Last modified: 8 Oct 2017, 0:43:41 UTC Forgot to mention you will have to adjust the app names in the GPUPriority.sh script to match your actual apps for MB and AP that you are using. [Edit] Also forgot to mention that the script also set affinity for both CPU and GPU tasks. Adjust accordingly to your processor. I have a AMD FX processor so I keep CPU tasks only on the physical cores and let the virtual cores handle the feeding of work to the GPUs. . . Hi Keith, . . You also forgot to include the fan control script ... I am back on the Linux box now ... #!/bin/bash nvidia-settings -a "[gpu:0]/GPUFanControlState=1" nvidia-settings -a "[gpu:1]/GPUFanControlState=1" #nvidia-settings -a "[gpu:2]/GPUFanControlState=1" interval=10 while true; do current_temp=$(nvidia-settings -t -q [gpu:0]/GPUCoreTemp) if (("$current_temp" < 41)); then nvidia-settings -a [fan:0]/GPUCurrentFanSpeed=00 elif (("$current_temp" > 40)) && (("$current_temp" < 49)); then nvidia-settings -a [fan:0]/GPUCurrentFanSpeed=40 elif (("$current_temp" > 48)) && (("$current_temp" < 55)); then nvidia-settings -a [fan:0]/GPUCurrentFanSpeed=50 elif (("$current_temp" > 54)) && (("$current_temp" < 58)); then nvidia-settings -a [fan:0]/GPUCurrentFanSpeed=60 elif (("$current_temp" > 57)) && (("$current_temp" < 64)); then nvidia-settings -a [fan:0]/GPUCurrentFanSpeed=75 elif (("$current_temp" > 63)) && (("$current_temp" < 68)); then nvidia-settings -a [fan:0]/GPUCurrentFanSpeed=90 elif (("$current_temp" > 67)); then nvidia-settings -a [fan:0]/GPUCurrentFanSpeed=100 fi current_temp=$(nvidia-settings -t -q [gpu:1]/GPUCoreTemp) if (("$current_temp" < 41)); then nvidia-settings -a [fan:1]/GPUCurrentFanSpeed=00 elif (("$current_temp" > 40)) && (("$current_temp" < 49)); then nvidia-settings -a [fan:1]/GPUCurrentFanSpeed=40 elif (("$current_temp" > 48)) && (("$current_temp" < 55)); then nvidia-settings -a [fan:1]/GPUCurrentFanSpeed=50 elif (("$current_temp" > 54)) && (("$current_temp" < 58)); then nvidia-settings -a [fan:1]/GPUCurrentFanSpeed=60 elif (("$current_temp" > 57)) && (("$current_temp" < 64)); then nvidia-settings -a [fan:1]/GPUCurrentFanSpeed=75 elif (("$current_temp" > 63)) && (("$current_temp" < 68)); then nvidia-settings -a [fan:1]/GPUCurrentFanSpeed=90 elif (("$current_temp" > 67)); then nvidia-settings -a [fan:1]/GPUCurrentFanSpeed=100 fi #current_temp=$(nvidia-settings -t -q [gpu:2]/GPUCoreTemp) #if (("$current_temp" < 41)); then # nvidia-settings -a [fan:2]/GPUCurrentFanSpeed=00 #elif (("$current_temp" > 40)) && (("$current_temp" < 49)); then # nvidia-settings -a [fan:2]/GPUCurrentFanSpeed=40 #elif (("$current_temp" > 48)) && (("$current_temp" < 55)); then # nvidia-settings -a [fan:2]/GPUCurrentFanSpeed=50 #elif (("$current_temp" > 54)) && (("$current_temp" < 58)); then # nvidia-settings -a [fan:2]/GPUCurrentFanSpeed=60 #elif (("$current_temp" > 57)) && (("$current_temp" < 64)); then # nvidia-settings -a [fan:2]/GPUCurrentFanSpeed=75 #elif (("$current_temp" > 63)) && (("$current_temp" < 68)); then # nvidia-settings -a [fan:2]/GPUCurrentFanSpeed=90 #elif (("$current_temp" > 67)); then # nvidia-settings -a [fan:2]/GPUCurrentFanSpeed=100 #fi sleep "$interval" done . . Again, I hope all this is useful for you Al and anyone else who is interested. Now all I have to do is remember how you add them to your startup options. OK, go to app find and search for startup manager. Open it and lock it to your launch bar. It will allow you to add the script files so they automatically start with each reboot. Stephen :) ID: 1893980 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1893987 - Posted: 8 Oct 2017, 1:14:45 UTC Sorry, yes I did. Didn't think of it because I use Jeff's app now for fan control. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1893987 · Reply Quote

Stephen "Heretic" Volunteer tester Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628	Message 1893996 - Posted: 8 Oct 2017, 1:29:38 UTC - in response to Message 1893987. Sorry, yes I did. Didn't think of it because I use Jeff's app now for fan control. . . Fair enough. That reminds me that I had intended to try out his rescheduler as well. With the little controversy I sparked over rescheduling and the run time timeouts I forgot about it. . . How do you find his fan control app? Stephen ?? ID: 1893996 · Reply Quote

Jeff Buck Volunteer tester Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0	Message 1893998 - Posted: 8 Oct 2017, 1:31:08 UTC - in response to Message 1893987. Sorry, yes I did. Didn't think of it because I use Jeff's app now for fan control. As long as you're using the GUI app for fan control, you should be able to omit the "GPUFanControlState=1" lines from your gpuoverclock script, as that's the first thing the GUI app does for each GPU that it's controlling. No harm in having that control state set multiple times, obviously, but should be unnecessary. ID: 1893998 · Reply Quote

Jeff Buck Volunteer tester Send message Joined: 11 Feb 00 Posts: 1441 Credit: 148,764,870 RAC: 0	Message 1894000 - Posted: 8 Oct 2017, 1:35:12 UTC - in response to Message 1893996. . . How do you find his fan control app? Stephen ?? NVIDIA GPU Fan Control using GUI in Linux ID: 1894000 · Reply Quote

Stephen "Heretic" Volunteer tester Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628	Message 1894002 - Posted: 8 Oct 2017, 1:47:22 UTC - in response to Message 1893970. Last modified: 8 Oct 2017, 1:49:43 UTC It looks like you have four 950 and one 1050. You can overclock the 950s but you can't do anything with the 1050 since it is a Pascal card. Maybe someday someone will figure out a way to get it into Performance Mode 3 in Linux. You will have to modify the Transfer rate clocks and the graphics clock overclocks to suit your cards. . . Actually, with the 1050 there are less performance levels than other 10xx cards and it will run in P0 by default, no tweaking necessary. The downside of being in P0 though is very high power usage when the card is idle. You can check the performance level when crunching by running "nvidia-smi -l". In case your font is as bad as mine that is -{lower case L}. . . So for Al, he only needs to include the 950s in those scripts to get them up to P0. Stephen .. ID: 1894002 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1894003 - Posted: 8 Oct 2017, 1:47:29 UTC - in response to Message 1893996. Both excellent. I've used his rescheduling app for the last couple of weeks. Said goodbye to Mr. Kevvy's app. I only use Jimbocous' app for intraday rescheduling < > of Arecibo < > GBT work on the Windows machines since you can leave BOINC running. Don't bother with the Linux machine as it could care less what antenna the work comes from. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1894003 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1894005 - Posted: 8 Oct 2017, 1:51:46 UTC - in response to Message 1893998. Sorry, yes I did. Didn't think of it because I use Jeff's app now for fan control. As long as you're using the GUI app for fan control, you should be able to omit the "GPUFanControlState=1" lines from your gpuoverclock script, as that's the first thing the GUI app does for each GPU that it's controlling. No harm in having that control state set multiple times, obviously, but should be unnecessary. Thanks for clarifying that Jeff. But Stephen suggested my base scripts for Al. If he gets adventurous in the future, he can give your apps a look. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1894005 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1894006 - Posted: 8 Oct 2017, 1:53:25 UTC - in response to Message 1894002. It looks like you have four 950 and one 1050. You can overclock the 950s but you can't do anything with the 1050 since it is a Pascal card. Maybe someday someone will figure out a way to get it into Performance Mode 3 in Linux. You will have to modify the Transfer rate clocks and the graphics clock overclocks to suit your cards. . . Actually, with the 1050 there are less performance levels than other 10xx cards and it will run in P0 by default, no tweaking necessary. The downside of being in P0 though is very high power usage when the card is idle. You can check the performance level when crunching by running "nvidia-smi -l". In case your font is as bad as mine that is -{lower case L}. . . So for Al, he only needs to include the 950s in those scripts to get them up to P0. Stephen .. Hi Stephen, thanks for pointing out that peculiarity of the 1050. I forgot you mentioned that fact before. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1894006 · Reply Quote

MarkJ Volunteer tester Send message Joined: 17 Feb 08 Posts: 1139 Credit: 80,854,192 RAC: 5	Message 1894042 - Posted: 8 Oct 2017, 6:05:54 UTC Last modified: 8 Oct 2017, 6:06:36 UTC Word of warning for those of you running Debian. They did a point release to Stretch (9.2) yesterday which includes Nvidia drivers 375.82. The driver gets stuck in P8 mode when running compute tasks so they take forever to complete. BOINC blog ID: 1894042 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1894044 - Posted: 8 Oct 2017, 6:24:28 UTC - in response to Message 1894042. Thanks for the head's up, Mark. Perfectly happy with the 375.66 drivers. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1894044 · Reply Quote

MarkJ Volunteer tester Send message Joined: 17 Feb 08 Posts: 1139 Credit: 80,854,192 RAC: 5	Message 1894048 - Posted: 8 Oct 2017, 7:07:41 UTC - in response to Message 1894044. Thanks for the head's up, Mark. Perfectly happy with the 375.66 drivers. I was too. Now I have to work out how to get them back. It seems after a reboot its goes in P2 state and stays that way for about 15 minutes then drops into P8 (idle mode) and one has to reboot the machine to get it going again. I will raise a Debian bug for it but I'm sure its Nvidia's code. BOINC blog ID: 1894048 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1894058 - Posted: 8 Oct 2017, 9:20:49 UTC - in response to Message 1894048. Looked at your systems. You seem to be running Pascal cards. Petri has explained that there is no way to get the Pascal cards to run in anything but P2 state when the drivers detect a compute workload. I am happy with my Maxwell cards and the 375.66 drivers to be able to move them and hold them at P0 state for compute workloads. I don't want that to change. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1894058 · Reply Quote

TBar Volunteer tester Send message Joined: 22 May 99 Posts: 5204 Credit: 840,779,836 RAC: 2,768	Message 1894083 - Posted: 8 Oct 2017, 13:25:12 UTC - in response to Message 1894058. I am happy with my Maxwell cards and the 375.66 drivers... Enjoy it while you can. Petri's new App requires driver 384 and above. It also requires Ubuntu 14.04.4 with kernel 4.2 and above. Lower systems don't have a new enough libstdc++.so which apparently needs to be libstdc++.so.6.0.20 or above. Add to that, my older Core2Quads don't work with driver 384 and kernel 4.10. I had the same experience with another machine, the things won't even boot with driver 384 & Kernel 4.10. They work fine with 4.8 & 4.4 with driver 384. So, it looks as though these older machines are restricted to Ubuntu 14.04.4 to 16.04.1. Anything above the 16.04.1 install image will eventually update to kernel 4.10. Within that narrow range, CUDA App zi3xs2 works fine on my machines, and the Arecibo tasks are a bit faster. ID: 1894083 · Reply Quote

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1894218 - Posted: 9 Oct 2017, 2:24:37 UTC How will things change once Nvidia starts shipping drivers with CUDA 9.0? Won't that make Petri's statically linked executable a moot point? Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1894218 · Reply Quote

©2024 University of California

SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.