Setting up Linux to crunch CUDA90 and above for Windows users

Message boards : Number crunching : Setting up Linux to crunch CUDA90 and above for Windows users
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 23 · 24 · 25 · 26 · 27 · 28 · 29 . . . 162 · Next

AuthorMessage
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1880356 - Posted: 26 Jul 2017, 9:38:17 UTC - in response to Message 1880353.  

Those sound like Questions, not Errors. Now that the Settings are being accepted, try Petri's script again. Hopefully it will work this time.
BTW, I've found that if I set the settings too high it will run a little while and then automatically downclock. The Graphics clock will Not take much before it downclocks, I don't even bother with that anymore. On the cards I run I've found the vendor has already wrung about as much as you can wring, not much more can be gained. I usually don't mess with it anymore.
ID: 1880356 · Report as offensive     Reply Quote
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 1880359 - Posted: 26 Jul 2017, 10:16:35 UTC
Last modified: 26 Jul 2017, 10:34:52 UTC

Hi,

What value did you set for the coolbits?
Mine is set with:
nvidia-xconfig --enable-all-gpus --cool-bits=28
and reboot.

Look 'Enabling overclocking' from this link https://wiki.archlinux.org/index.php/NVIDIA/Tips_and_tricks for more options, like setting max perf level (Option "RegistryDwords" "PerfLevelSrc=0x2222").
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1880359 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880361 - Posted: 26 Jul 2017, 10:43:57 UTC - in response to Message 1880356.  

I purged the nvidia stuff and then fought as usual to get it to boot a readable screen, just locks up with torn video gibberish. Tried to get through the Recovery option a dozen times. Finally, somehow it got back to the desktop. Rebooted and back to gibberish. Got back to the desktop again. Scared to try shutting it down again. I am on the Nvidia proprietary driver now.

Nvidia Settings still won't let me stay in P3 with gpu tasks running. FYI, it doesn't matter whether there is any overclock, it still will not stay in P3 even at stock clocks.

I don't know why I have issues getting to a readable screen after configuration changes. Once it is stable at least it will come back after shutting down or reboot.

I thought my trouble was the lack of --nomodeset in the grub file but surprisingly it was there and I still had video gibberish.

I am going to run my script at root to get the P3 memory clocks back and call it a night. Its 4AM. Thanks for the help.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880361 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880362 - Posted: 26 Jul 2017, 10:50:40 UTC - in response to Message 1880359.  

Yes, I have coolbits set with that same command. I have fan and thermal control as expected. Everything works EXCEPT keeping the memory clocks in P3 when the gpus are crunching.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880362 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880364 - Posted: 26 Jul 2017, 10:59:20 UTC - in response to Message 1880359.  

Petri, what do make of this?
The current user does not have permission to change clocks for GPU 00000000:01:00.0.
 Run 'nvidia-smi -acp UNRESTRICTED' as root/admin to enable this option for all users.
Terminating early due to previous errors.
The current user does not have permission to change clocks for GPU 00000000:02:00.0.
 Run 'nvidia-smi -acp UNRESTRICTED' as root/admin to enable this option for all users.
Terminating early due to previous errors.
The current user does not have permission to change clocks for GPU 00000000:03:00.0.
 Run 'nvidia-smi -acp UNRESTRICTED' as root/admin to enable this option for all users.
Terminating early due to previous errors.

This is my script currently:
#!/bin/bash

/usr/bin/nvidia-smi -acp UNRESTRICTED
/usr/bin/nvidia-smi -pm 1


/usr/bin/nvidia-settings -a "[gpu:0]/GPUFanControlState=1" -a [fan:0]/GPUTargetFanSpeed=100
/usr/bin/nvidia-settings -a "[gpu:1]/GPUFanControlState=1" -a [fan:1]/GPUTargetFanSpeed=100
/usr/bin/nvidia-settings -a "[gpu:2]/GPUFanControlState=1" -a [fan:2]/GPUTargetFanSpeed=100

/usr/bin/nvidia-settings -a "[gpu:0]/GPUPowerMizerMode=1"
/usr/bin/nvidia-settings -a "[gpu:1]/GPUPowerMizerMode=1"
/usr/bin/nvidia-settings -a "[gpu:2]/GPUPowerMizerMode=1"

/usr/bin/nvidia-settings -a "[gpu:0]/GPUMemoryTransferRateOffset[3]=200" -a "[gpu:0]/GPUGraphicsClockOffset[3]=40"
/usr/bin/nvidia-settings -a "[gpu:1]/GPUMemoryTransferRateOffset[3]=200" -a "[gpu:1]/GPUGraphicsClockOffset[3]=40"
/usr/bin/nvidia-settings -a "[gpu:2]/GPUMemoryTransferRateOffset[3]=200" -a "[gpu:2]/GPUGraphicsClockOffset[3]=40"


/usr/bin/nvidia-smi -i 0 -ac 3605,1394
/usr/bin/nvidia-smi -i 1 -ac 3605,1384
/usr/bin/nvidia-smi -i 2 -ac 3605,1394

It runs fine if run as root and all my clocks are set and I stay in P3. It just can't be run in anything but root and have the memory clocks stay in P3.

P3 memory clocks with BOINC
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880364 · Report as offensive     Reply Quote
Profile tazzduke
Volunteer tester

Send message
Joined: 15 Sep 07
Posts: 190
Credit: 28,269,068
RAC: 5
Australia
Message 1880365 - Posted: 26 Jul 2017, 11:05:24 UTC

Greetings All

Things that make me happy when I have moved to Linux.

I finally figured out how to use BOINCTASKS to monitor my two Linux machines from my Windows machine.

Yeah I know its not much, but to me its a small accomplishment :-)

Cheers
ID: 1880365 · Report as offensive     Reply Quote
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 1880366 - Posted: 26 Jul 2017, 11:09:12 UTC

From command prompt run as root or sudo
nvidia-smi -acp UNRESTRICTED

After that all users can set the -ac parameter.
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1880366 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880428 - Posted: 26 Jul 2017, 16:59:17 UTC - in response to Message 1880365.  

Greetings All

Things that make me happy when I have moved to Linux.

I finally figured out how to use BOINCTASKS to monitor my two Linux machines from my Windows machine.

Yeah I know its not much, but to me its a small accomplishment :-)

Cheers

Oh, Please Do Tell!! I looked into that on the BoincTasks forums and Fred said it wasn't possible without installing WINE.

Please post the instructions on how you were able to accomplish this feat. I would really like to monitor my new linux machine from my Windows machine BoincTasks server.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880428 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880429 - Posted: 26 Jul 2017, 17:00:44 UTC - in response to Message 1880366.  

From command prompt run as root or sudo
nvidia-smi -acp UNRESTRICTED

After that all users can set the -ac parameter.

So it is not possible to do that from a script at startup?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880429 · Report as offensive     Reply Quote
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1880439 - Posted: 26 Jul 2017, 18:04:36 UTC - in response to Message 1880428.  

Please post the instructions on how you were able to accomplish this feat. I would really like to monitor my new linux machine from my Windows machine BoincTasks server.
Monitoring A Linux cruncher from Windows is exactly the same as for a Windows client. Except that you add an IP entry in the /etc/hosts file. To run BoincTasks on Linux would require using Wine.
ID: 1880439 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880447 - Posted: 26 Jul 2017, 19:12:02 UTC - in response to Message 1880439.  

Please post the instructions on how you were able to accomplish this feat. I would really like to monitor my new linux machine from my Windows machine BoincTasks server.
Monitoring A Linux cruncher from Windows is exactly the same as for a Windows client. Except that you add an IP entry in the /etc/hosts file. To run BoincTasks on Linux would require using Wine.

Thanks for the information Brent. I guess I misinterpreted the posts from Fred. I have now added Darksider to my BoincTasks Computers. Wonderful!
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880447 · Report as offensive     Reply Quote
Profile tazzduke
Volunteer tester

Send message
Joined: 15 Sep 07
Posts: 190
Credit: 28,269,068
RAC: 5
Australia
Message 1880464 - Posted: 26 Jul 2017, 20:55:35 UTC

Greetings

I found the information on the BOINCTASKS website. Which gives you a step by step instruction on how to do it.

Regards
ID: 1880464 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1880513 - Posted: 27 Jul 2017, 3:48:30 UTC - in response to Message 1880335.  

Well I have learned a few things. First I can't get the script to accept my power levels settings. Second, and most important is what I learned from executing the script in Terminal and reading the output.

Most important fact is that the script has to be run as root to set the permissions for the power level command to recognized. I still haven't figured out why my commands are not valid. I am copying everything exactly from Petri's posts.

My core clocks are being set correctly by the script after booting and BOINC running, It is just that the memory clocks are still falling back to P2 at 6008 Mhz.


. . A question Keith. With the memory clock running at the Performance level 2 settings what GPU usage do you see if you execute the command nvidia-smi -l ??

Stephen

??
ID: 1880513 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880517 - Posted: 27 Jul 2017, 4:48:45 UTC - in response to Message 1880513.  

Well I have learned a few things. First I can't get the script to accept my power levels settings. Second, and most important is what I learned from executing the script in Terminal and reading the output.

Most important fact is that the script has to be run as root to set the permissions for the power level command to recognized. I still haven't figured out why my commands are not valid. I am copying everything exactly from Petri's posts.

My core clocks are being set correctly by the script after booting and BOINC running, It is just that the memory clocks are still falling back to P2 at 6008 Mhz.


. . A question Keith. With the memory clock running at the Performance level 2 settings what GPU usage do you see if you execute the command nvidia-smi -l ??

Stephen

??

Hi Stephen, I don't know. I would have to figure out how to shove the cards back into P2 mode. I have them running at P3 now. It is has been another couple of ball-buster days with this system again. I had to start from scratch and reinstall everything. I just could not get the system to give me readable video output upon booting with the direct Nvidia sourced driver 384.59. Boots just hung with unreadable video across the top 1" of screen. Recovery modes wouldn't work either with failsafe video. Couldn't even get to console. I finally was able to install the normal Ubuntu 375.66 drivers and fixed the dependency issues with Boinc Manager.

I learned you cannot set acp permissions or nvidia-smi commands without being root. And you cannot run a script from Startup Applications and have them run as root. So I am manually starting the commands in my script file. That sets the fan speeds, the performance levels and the overclocks.

I am seeing generally 98-100% usage in P3. I'll post the output.

root@Darksider:/home/keith# nvidia-smi -l
Wed Jul 26 21:30:56 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.66                 Driver Version: 375.66                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 970     On   | 0000:01:00.0      On |                  N/A |
|100%   73C    P0   143W / 170W |   2268MiB /  4034MiB |     98%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 970     On   | 0000:02:00.0     Off |                  N/A |
|100%   64C    P0   132W / 170W |   2077MiB /  4037MiB |     99%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 970     On   | 0000:03:00.0     Off |                  N/A |
|100%   77C    P0   143W / 170W |   2077MiB /  4037MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1092    G   /usr/lib/xorg/Xorg                             110MiB |
|    0      1689    G   compiz                                          80MiB |
|    0     11721    C   ...home_x41p_zi3v_x86_64-pc-linux-gnu_cuda80  2073MiB |
|    1     11604    C   ...home_x41p_zi3v_x86_64-pc-linux-gnu_cuda80  2073MiB |
|    2     11691    C   ...home_x41p_zi3v_x86_64-pc-linux-gnu_cuda80  2073MiB |
+-----------------------------------------------------------------------------+
Wed Jul 26 21:31:01 2017       

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880517 · Report as offensive     Reply Quote
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 1880544 - Posted: 27 Jul 2017, 7:31:18 UTC - in response to Message 1880517.  

Hi,
This is the best state to run. P-state 0 (nvidia-smi reports it P0 in the picture) . i.e. perf level 3 (as reported by nvidia settings). You have everything ok. I can not get my cards to run at anything but P2 (nvidia-smi says).

And to the other issue 'running a startup script as root' you find help with a little googling: linux startup script root

Petri
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1880544 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880549 - Posted: 27 Jul 2017, 8:05:00 UTC - in response to Message 1880544.  

Hi Petri, yes I spend a good couple of hours googling through all the results trying to match what I am attempting to do. Nothing matched up 100% in desired outcome to what I want. The closest was a post about getting network drivers to load at boot as root.

I have things working as how I want for now. Next task is to figure out how to run those root commands at boot. I need to take a break from this system for a while and get on with normal everyday duties.

Computers .................. the 'black-hole' of time.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880549 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880552 - Posted: 27 Jul 2017, 8:42:00 UTC

What was the consensus reached about which flavor of CPU app was fastest under Linux. I know there are SSE4.1 and AVX varieties I think over at Lunatics developed by Urs. Is the SSSE3 CPU app that is in the special sauce package put together by TBar considered the best nowadays?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880552 · Report as offensive     Reply Quote
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1880557 - Posted: 27 Jul 2017, 11:43:40 UTC - in response to Message 1880552.  

I can't remember what my numbers were, but I tried all 3 and settled on AVX.
It's easy enough to change executables. Just complete any 'active' tasks, and change the executable name in app_info (2 places) and DON'T change the version number. Then run another batch through.
ID: 1880557 · Report as offensive     Reply Quote
dallasdawg

Send message
Joined: 19 Aug 99
Posts: 49
Credit: 142,692,438
RAC: 2
United States
Message 1880579 - Posted: 27 Jul 2017, 14:10:19 UTC - in response to Message 1880517.  

I learned you cannot set acp permissions or nvidia-smi commands without being root. And you cannot run a script from Startup Applications and have them run as root. So I am manually starting the commands in my script file.


If you want to run a script as root that will run every time your machine boots, you can add the script to the init daemon.

Copy your script file to /etc/init.d/ using "sudo cp <script file> /etc/init.d/"
Then make sure the script file permission is 755 with "sudo chmod 755 <script file>" and the owner is root with "sudo chown root:root <script file>".
Next, in that directory execute this "sudo update-rc.d <script file> default". This will add your script to the init daemon and it will execute at the default run level.
Reboot and then verify your script has run by checking your video card settings.
ID: 1880579 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880596 - Posted: 27 Jul 2017, 16:35:55 UTC - in response to Message 1880579.  

Thanks Dallasdawg, that's the kind of explicit directions I need and handholding for this linux noobie. I saw the reference in init.d in my googling but no exact instructions on how to do so.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880596 · Report as offensive     Reply Quote
Previous · 1 . . . 23 · 24 · 25 · 26 · 27 · 28 · 29 . . . 162 · Next

Message boards : Number crunching : Setting up Linux to crunch CUDA90 and above for Windows users


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.