Setting up Linux to crunch CUDA90 and above for Windows users

Message boards : Number crunching : Setting up Linux to crunch CUDA90 and above for Windows users
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 162 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880645 - Posted: 27 Jul 2017, 22:04:20 UTC - in response to Message 1880579.  

I learned you cannot set acp permissions or nvidia-smi commands without being root. And you cannot run a script from Startup Applications and have them run as root. So I am manually starting the commands in my script file.


If you want to run a script as root that will run every time your machine boots, you can add the script to the init daemon.

Copy your script file to /etc/init.d/ using "sudo cp <script file> /etc/init.d/"
Then make sure the script file permission is 755 with "sudo chmod 755 <script file>" and the owner is root with "sudo chown root:root <script file>".
Next, in that directory execute this "sudo update-rc.d <script file> default". This will add your script to the init daemon and it will execute at the default run level.
Reboot and then verify your script has run by checking your video card settings.

Hi Dallasdawg, I put my scripts into etc/init.d/ and rebooted. It looks like they didn't run. I can run them manually from root Terminal without error messages. Is there a way to look at any log of the scripts run from etc /init.d/ to see if they are run correctly?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880645 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880656 - Posted: 27 Jul 2017, 23:32:57 UTC
Last modified: 27 Jul 2017, 23:34:38 UTC

Dallasdawg, are these messages something I should worry about?

root@Darksider:/home/keith/Desktop# sudo cp UNRESTRICTED.sh /etc/init.d/
root@Darksider:/home/keith/Desktop# sudo cp gpufanspeed.sh /etc/init.d/
root@Darksider:/home/keith/Desktop# cd /etc/init.d
root@Darksider:/etc/init.d# sudo update-rc.d UNRESTRICTED.sh default
usage: update-rc.d [-n] [-f] <basename> remove
       update-rc.d [-n] <basename> disable|enable [S|2|3|4|5]
		-n: not really
		-f: force

The disable|enable API is not stable and might change in the future.
root@Darksider:/etc/init.d# sudo update-rc.d gpufanspeed.sh default
usage: update-rc.d [-n] [-f] <basename> remove
       update-rc.d [-n] <basename> disable|enable [S|2|3|4|5]
		-n: not really
		-f: force

The disable|enable API is not stable and might change in the future.
root@Darksider:/etc/init.d# exit
exit
keith@Darksider:~/Desktop$ 


Oh, BTW, I added this command to the script files.
exec 1> >(logger -s -t $(basename $0)) 2>&1

It's supposed to log to the syslog the output of the scripts.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880656 · Report as offensive     Reply Quote
dallasdawg

Send message
Joined: 19 Aug 99
Posts: 49
Credit: 142,692,438
RAC: 2
United States
Message 1880661 - Posted: 27 Jul 2017, 23:47:39 UTC - in response to Message 1880645.  
Last modified: 27 Jul 2017, 23:55:03 UTC

Hi Dallasdawg, I put my scripts into etc/init.d/ and rebooted. It looks like they didn't run. I can run them manually from root Terminal without error messages. Is there a way to look at any log of the scripts run from etc /init.d/ to see if they are run correctly?


Keith,

I steered you down the wrong path. My bad. Not sure what I was thinking.

1) If you are in the root account, you do not need to use sudo. sudo is a command that is short for "superuser do". It grants temporary root privileges to a command. If you are in the root account, then the command is not needed because you are already root.

2) Your script is not going to work for init.d (also know as SysVinit). That init daemon is really meant for starting and stopping services. The script you wrote isn't a service. It is just a group of commands to manipulate video card settings. Further, you script is really a Bash script and not really an init script (too much to write to discuss the differences). Bash is the shell and command language you use in the terminal. You put " #! /bin/bash "at the top of the script to tell the system that this is a Bash shell script (as opposed to a python script or other).

You can remove my error from your system with this command: sudo update-rc.d -f <script name> remove
Then you can remove the actual file from the /etc/init.d/ directory with: sudo rm <script name>

So, I am thinking the best way for you to execute this script at boot is to make a cron job. It has been a while since I set up one, so I need to go brush up on it before I give any instructions.
ID: 1880661 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880662 - Posted: 27 Jul 2017, 23:52:55 UTC - in response to Message 1880661.  

Thanks Dallasdawg, I will do as you instruct. I ran some upgrades and noticed this in the details. So, yes the system has troubles with my scripts.
insserv: warning: script 'UNRESTRICTED.sh' missing LSB tags and overrides
insserv: warning: script 'gpufanspeed.sh' missing LSB tags and overrides

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880662 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880664 - Posted: 28 Jul 2017, 0:10:38 UTC - in response to Message 1880661.  

Hi Dallasdawg, I put my scripts into etc/init.d/ and rebooted. It looks like they didn't run. I can run them manually from root Terminal without error messages. Is there a way to look at any log of the scripts run from etc /init.d/ to see if they are run correctly?


Keith,

I steered you down the wrong path. My bad. Not sure what I was thinking.

1) If you are in the root account, you do not need to use sudo. sudo is a command that is short for "superuser do". It grants temporary root privileges to a command. If you are in the root account, then the command is not needed because you are already root.

2) Your script is not going to work for init.d (also know as SysVinit). That init daemon is really meant for starting and stopping services. The script you wrote isn't a service. It is just a group of commands to manipulate video card settings. Further, you script is really a Bash script and not really an init script (too much to write to discuss the differences). Bash is the shell and command language you use in the terminal. You put " #! /bin/bash "at the top of the script to tell the system that this is a Bash shell script (as opposed to a python script or other).

You can remove my error from your system with this command: sudo update-rc.d -f <script name> remove
Then you can remove the actual file from the /etc/init.d/ directory with: sudo rm <script name>

So, I am thinking the best way for you to execute this script at boot is to make a cron job. It has been a while since I set up one, so I need to go brush up on it before I give any instructions.

The update of rc.d was not liked.
root@Darksider:/etc/init.d# sudo update-rc.d -f gpufanspeed.sh remove
insserv: warning: script 'gpufanspeed.sh' missing LSB tags and overrides
insserv: warning: script 'UNRESTRICTED.sh' missing LSB tags and overrides
root@Darksider:/etc/init.d# sudo update-rc.d -f UNRESTRICTED.sh remove
insserv: warning: script 'UNRESTRICTED.sh' missing LSB tags and overrides
insserv: warning: script 'gpufanspeed.sh' missing LSB tags and overrides


Should I be worried about this insserv warning?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880664 · Report as offensive     Reply Quote
dallasdawg

Send message
Joined: 19 Aug 99
Posts: 49
Credit: 142,692,438
RAC: 2
United States
Message 1880676 - Posted: 28 Jul 2017, 0:51:42 UTC - in response to Message 1880664.  
Last modified: 28 Jul 2017, 0:52:47 UTC

Reading the man page for update-rc.d, you need to delete the actual script file first, then do the sudo update-rc.d <filename> remove. According to the man page, if the actual file is still present in /etc/init.d/ then the remove function aborts without doing anything. I learned something new today.

Hopefully, that will take care of it.
ID: 1880676 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880680 - Posted: 28 Jul 2017, 1:09:39 UTC - in response to Message 1880676.  

Reading the man page for update-rc.d, you need to delete the actual script file first, then do the sudo update-rc.d <filename> remove. According to the man page, if the actual file is still present in /etc/init.d/ then the remove function aborts without doing anything. I learned something new today.

Hopefully, that will take care of it.

Thanks again Dallasdawg. The removal of the script files again said the files weren't there. So my previous removal seems to have worked.

The update-rc.d remove seems to have run without errors as the command simply brought me back to root.

So back to square one. I will run the scripts manually at each boot. That is infrequent when the system is running fine and I am not tinkering. I will research how to run a cron job next.

Thanks for your assistance ...... it is greatly appreciated.

Cheers, Keith
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880680 · Report as offensive     Reply Quote
dallasdawg

Send message
Joined: 19 Aug 99
Posts: 49
Credit: 142,692,438
RAC: 2
United States
Message 1880689 - Posted: 28 Jul 2017, 1:39:00 UTC - in response to Message 1880680.  
Last modified: 28 Jul 2017, 1:46:03 UTC

Thanks again Dallasdawg. The removal of the script files again said the files weren't there. So my previous removal seems to have worked.

The update-rc.d remove seems to have run without errors as the command simply brought me back to root.

So back to square one. I will run the scripts manually at each boot. That is infrequent when the system is running fine and I am not tinkering. I will research how to run a cron job next.

Thanks for your assistance ...... it is greatly appreciated.

Cheers, Keith


Good to hear. So, I have been looking into the cron job for you and have something for you to try. If you want to try it, see below.

As for adding a cron job, here is what you do:

I would recommend doing this in the root account. We want root to be the owner of this cron job, so that it will execute properly....just like when you do manually. I believe you can keep the script anywhere including your home folder, but it will still need root ownership and the execute flag set.

In the Terminal:
1) To enter root account, type: sudo -i, then enter password.
2) Type: crontab -e (crontab is the the admin command, -e option is to edit to create the new job)
3) If it asks for a viewer, select Nano (should be #2; it usually asks the first time you ever use it). If you select something different, then the keystrokes in #5 won't work.
4) Once in Nano, 1st line below the comments, type: @reboot /path/to/file/<script file>. You need to explicitly write out the path, because it is possible that some environment variables may not be set during the boot phase and this script running. So, if you keep the file in your home directory, then the path will be something like /home/<username>/<remainder of path, if any>
5) Now do a CTRL O to save and CTRL X to exit.
6) Verify the cronjob took by typing: crontab -l (this will list all cron jobs for the root account; this job will probably be the only one)
6) Reboot and see if your video card settings change.

If this does not work, then I'll try to mimic this on my machine and work it out....then resend instructions. I have not tried anything as root and not as an "only run after boot", so I am basing this on what is in the man pages. I run several cron jobs on my webserver to execute python scripts to pull data from other websites (mainly weather related). These are set up to run every day at midnight.
ID: 1880689 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880695 - Posted: 28 Jul 2017, 2:10:42 UTC - in response to Message 1880689.  

Thanks so much again for the assistance, Dallasdawg. I will attempt to make a cron job tomorrow. I was waiting on a Amazon delivery all day via USPS. Finally delivered at 5:22 PM. So I will tackle that after I transition to a SSD. Crossing my fingers. That will mean taking the system down for installation and a good time to try out the cron job of my scripts.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880695 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880697 - Posted: 28 Jul 2017, 2:12:41 UTC - in response to Message 1880557.  

I can't remember what my numbers were, but I tried all 3 and settled on AVX.
It's easy enough to change executables. Just complete any 'active' tasks, and change the executable name in app_info (2 places) and DON'T change the version number. Then run another batch through.

Brent, the shift to the AVX app went fine with no issues. Can already tell it is the faster compared to the SSSE3 app. The AMD FX processors as WELL as the Ryzen 1700X REALLY like the AVX apps.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880697 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880701 - Posted: 28 Jul 2017, 2:19:11 UTC

OK, another question for the Linux users. Does linux handle CPU allocations more intelligently than Windows? I don't have ProcessLasso as a tool in the Darksider system. Is it needed? I think there is an app called TOP or something that can change CPU priorities and such as the equivalent to ProcessLasso.

Do the linux app and host system users just let their systems run? Or do they tweak?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880701 · Report as offensive     Reply Quote
dallasdawg

Send message
Joined: 19 Aug 99
Posts: 49
Credit: 142,692,438
RAC: 2
United States
Message 1880708 - Posted: 28 Jul 2017, 3:07:34 UTC - in response to Message 1880701.  
Last modified: 28 Jul 2017, 3:07:56 UTC

OK, another question for the Linux users. Does linux handle CPU allocations more intelligently than Windows? I don't have ProcessLasso as a tool in the Darksider system. Is it needed? I think there is an app called TOP or something that can change CPU priorities and such as the equivalent to ProcessLasso.

Do the linux app and host system users just let their systems run? Or do they tweak?


In one word......YES....to all of it. There are many tools for tweaking the kernel for whatever you want. The question will be...How far down the rabbit hole do you want to go?

TOP is only a CPU, memory and process lister. It does not have any controls to change allocation. Here is a blog that discusses some of the options you can use to control processes and CPU limits.

http://blog.scoutapp.com/articles/2014/11/04/restricting-process-cpu-usage-using-nice-cpulimit-and-cgroups

The only tool I have really ever used is nice. The other tools I have never used. I have never really tried to tweak out my machine to the Nth degree, but there are many who do especially linux server admins. I find, in general, Linux out of the box is more efficient that Windows and is not near the resource hog. I run BOINC on this machine and my 4 other linux boxes with no special settings. I do use Mate for my desktop, and it is lighter on the RAM than Unity, Gnome, and KDE.

Just a warning.....linux is a very powerful OS, and it will let you do all kinds of things to customize, streamline, and make it your own. But, it will also be more than happy to feed you all the rope it can to hang yourself. It can definitely be a very unforgiving OS when you do something wrong.
ID: 1880708 · Report as offensive     Reply Quote
dallasdawg

Send message
Joined: 19 Aug 99
Posts: 49
Credit: 142,692,438
RAC: 2
United States
Message 1880710 - Posted: 28 Jul 2017, 3:13:46 UTC - in response to Message 1880697.  

I can't remember what my numbers were, but I tried all 3 and settled on AVX.
It's easy enough to change executables. Just complete any 'active' tasks, and change the executable name in app_info (2 places) and DON'T change the version number. Then run another batch through.

Brent, the shift to the AVX app went fine with no issues. Can already tell it is the faster compared to the SSSE3 app. The AMD FX processors as WELL as the Ryzen 1700X REALLY like the AVX apps.


On my machine with an I7-4790K, I am using the AVX version. Tasks are generally completing in 45-50 minutes.
ID: 1880710 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880712 - Posted: 28 Jul 2017, 3:20:43 UTC - in response to Message 1880708.  


Just a warning.....linux is a very powerful OS, and it will let you do all kinds of things to customize, streamline, and make it your own. But, it will also be more than happy to feed you all the rope it can to hang yourself. It can definitely be a very unforgiving OS when you do something wrong.


Oh, I have already verified that fact {grin}. I just would like to lock CPU tasks to a physical core since the AMD FX cpus are a special case.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880712 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880713 - Posted: 28 Jul 2017, 3:23:35 UTC - in response to Message 1880710.  


On my machine with an I7-4790K, I am using the AVX version. Tasks are generally completing in 45-50 minutes.

Yes, I was getting discouraged with the 2 1/2 hour runtimes with the CPU tasks using the SSSE3 app. I am used to 50-75 minutes with the AVX app on my other FX and Ryzen systems. I was just using the default app_info supplied in TBar's package. Decided it was time for a tweak.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880713 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1880962 - Posted: 29 Jul 2017, 15:43:02 UTC
Last modified: 29 Jul 2017, 15:45:58 UTC

Well, after I sold my Z600 and got my "monster" more or less up and running I installed Lubuntu (Light Ubuntu) on my Z400 as a dual boot.

Got Nvidia drivers installed, got boinc-manager, and boinc-client installed. Used the software update gui and Synatic package manager for all this so I have no idea where anything is.

Got registered with SetiBeta and have cpu and gpu tasks downloaded and processing. I have both nividia_SOG and nividia_sah waiting to process. It appears to have processed a Cuda60.

What is the catch? I can't find app_config.xml so I can dedicate a core to the gpu.

As I posted in the messages area for Unix/Linux. Where is the Linux equivalent of "ProgramData/BOINC/projects/setiathome.berkeley.edu"?

I have looked and found some of the .xml files I recognize. The boinc manager log claims to have downloaded some text files that I would surely recognize and use, if I could find them?

I suspect that Boinc installs "uniquely" under each distro of Linux? And I am confident things are spread all over the place. But where are the app_config.xml, app_info.xml, mb*sog.txt and the mcuda*.cfg files?

Thank you,
Tom
A proud member of the OFA (Old Farts Association).
ID: 1880962 · Report as offensive     Reply Quote
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1880970 - Posted: 29 Jul 2017, 16:20:19 UTC - in response to Message 1880962.  
Last modified: 29 Jul 2017, 16:24:59 UTC

What is the catch? I can't find app_config.xml so I can dedicate a core to the gpu.

As I posted in the messages area for Unix/Linux. Where is the Linux equivalent of "ProgramData/BOINC/projects/setiathome.berkeley.edu"?

I have looked and found some of the .xml files I recognize. The boinc manager log claims to have downloaded some text files that I would surely recognize and use, if I could find them?

I suspect that Boinc installs "uniquely" under each distro of Linux? And I am confident things are spread all over the place. But where are the app_config.xml, app_info.xml, mb*sog.txt and the mcuda*.cfg files?

Thank you,
Tom
If you take a look at the BOINC Event log, about the 4th line from the beginning should identify the "Data directory" location. That folder should include the mb* configuration files. However, the app_config.xml and app_info.xml are files that you need to create from scratch. They're totally optional files that are not distributed by the project.

EDIT: And the app_config.xml file goes in the project's directory, which is subordinate to the data directory.
ID: 1880970 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1880975 - Posted: 29 Jul 2017, 16:32:34 UTC - in response to Message 1880962.  
Last modified: 29 Jul 2017, 16:32:47 UTC


I suspect that Boinc installs "uniquely" under each distro of Linux? And I am confident things are spread all over the place. But where are the app_config.xml, app_info.xml, mb*sog.txt and the mcuda*.cfg files?

Thank you,
Tom

The Ubuntu default is /var/lib/boinc-client
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1880975 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1880979 - Posted: 29 Jul 2017, 16:49:34 UTC - in response to Message 1880975.  

The Ubuntu default is /var/lib/boinc-client


Thank you Jeff and Keith.

I am aware of my "selective" blindness. But once I got into that directory, things looked a WHOLE LOT more familiar :D

Once I get done rattling around in there, then I get to go back to the "original" question of how to get the Cuda80 install and patched in there.

Then I will be about as "expert" as I ever am. It sounds like I am going to have to start working through a newbie tutorial on Ubuntu after I get all the rest of this working.

Tom
A proud member of the OFA (Old Farts Association).
ID: 1880979 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1880982 - Posted: 29 Jul 2017, 17:00:14 UTC - in response to Message 1880979.  

The Ubuntu default is /var/lib/boinc-client


Opps, now I have to figure out how to access the project directory. Even though I am the only user on the system it says I don't have "permission". Mumble. I can open a terminal and I clearly have a path. The question is this simply a SUDO cd? or a "change permissions" on the file type of issue?

I am, of course, trying to access it through the File Manager. I tried changing my user id to system admin but that didn't work. So I am trying a reboot to see if that will synchronize it.

Hmmmm....

Tom
A proud member of the OFA (Old Farts Association).
ID: 1880982 · Report as offensive     Reply Quote
Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 162 · Next

Message boards : Number crunching : Setting up Linux to crunch CUDA90 and above for Windows users


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.