Setting up Linux to crunch CUDA90 and above for Windows users

Message boards : Number crunching : Setting up Linux to crunch CUDA90 and above for Windows users
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 73 · 74 · 75 · 76 · 77 · 78 · 79 . . . 162 · Next

AuthorMessage
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1955069 - Posted: 12 Sep 2018, 22:34:42 UTC - in response to Message 1955066.  

...machine boots, browser works. Seti says "nothing".
Can you be a little more descriptive?
Like, does the boincmgr start and show Window?
Have you checked to see if boincmgr & boinc are running? If you don't have System Monitor see if you can install it, otherwise enter top into a Terminal window and see if those two boinc Apps are running.
ID: 1955069 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955081 - Posted: 12 Sep 2018, 23:51:15 UTC - in response to Message 1955069.  

...machine boots, browser works. Seti says "nothing".
Can you be a little more descriptive?
Like, does the boincmgr start and show Window?
Have you checked to see if boincmgr & boinc are running? If you don't have System Monitor see if you can install it, otherwise enter top into a Terminal window and see if those two boinc Apps are running.


I left it off for 45 minutes and then tried again. It took a little longer than I expected but it came up again. Strange.

The TaskManager was showing no load but I didn't hunt to see if boncmgr & boinc were running. So I don't know.

I am reluctant to touch it, now that it seems to be running again. :(

I don't know a "System Monitor" in Linux. Can you give me a hint besides "system monitor" so I can try to find it?

Tom
A proud member of the OFA (Old Farts Association).
ID: 1955081 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1955084 - Posted: 12 Sep 2018, 23:56:59 UTC - in response to Message 1955081.  

Gnome System Monitor in the Ubuntu Software Catalog.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1955084 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1955098 - Posted: 13 Sep 2018, 1:07:57 UTC - in response to Message 1954832.  

my 2 systems have been running over 12 hours with Firefox open and refreshing the SETI main page every 5 mins with a tab reloader.

These ones:
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8390155
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8432395

feel free to scrub through the recent tasks, or check back on them.

so far nothing.


now has been 48hrs. no signs of any missed best pulse tasks due to Firefox activity.

the experiment continues. i suppose i'll let it run like this over a week.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1955098 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1955099 - Posted: 13 Sep 2018, 1:09:07 UTC - in response to Message 1955066.  

My dual e5-2670 seti won't start up. Apparently it won't even clear the Seti log (if I found the right file in BOINC). I shut it down without stopping BOINC manager and the tasks, accidently, a while ago, machine boots, browser works. Seti says "nothing".

As a Linux newbie, is there a "log" file I can be pointed to that might tell me something more?

Thanks,
Tom


. . If you just closed the manager, in Linux the client would still be running. You need to shut down the client and restart it. I think you said you were running the repository version of BOINC. So open a terminal window, change to the /var/lib/boinc-client folder to be sure, and type "sudo service boinc-client restart". It should work from anywhere but ... Then re-open the Boinc Manager.

Stephen

:)
ID: 1955099 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1955102 - Posted: 13 Sep 2018, 1:17:12 UTC - in response to Message 1955084.  

Gnome System Monitor in the Ubuntu Software Catalog.


. . that is one of the shortcomings with Lubuntu. It doesn't have the nice link to the Ubuntu repository apps on the launch bar like full strength Ubuntu does.

Stephen

:(
ID: 1955102 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1955104 - Posted: 13 Sep 2018, 1:29:49 UTC - in response to Message 1955099.  

My dual e5-2670 seti won't start up. Apparently it won't even clear the Seti log (if I found the right file in BOINC). I shut it down without stopping BOINC manager and the tasks, accidently, a while ago, machine boots, browser works. Seti says "nothing".

As a Linux newbie, is there a "log" file I can be pointed to that might tell me something more?

Thanks,
Tom


. . If you just closed the manager, in Linux the client would still be running. You need to shut down the client and restart it. I think you said you were running the repository version of BOINC. So open a terminal window, change to the /var/lib/boinc-client folder to be sure, and type "sudo service boinc-client restart". It should work from anywhere but ... Then re-open the Boinc Manager.

Stephen

:)

Not sure about repository versions, would think it doesn't matter, but when you exit the Manager the first time you run it, it asks you whether you want to quit the client and all tasks when you exit or quit the Manager. And remembers from then on. So always cleanly shuts down both the Client and Manager and all running tasks from the Exit dialog of the Manager.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1955104 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1955108 - Posted: 13 Sep 2018, 2:02:26 UTC - in response to Message 1955098.  

my 2 systems have been running over 12 hours with Firefox open and refreshing the SETI main page every 5 mins with a tab reloader.

These ones:
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8390155
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8432395

feel free to scrub through the recent tasks, or check back on them.
so far nothing.
now has been 48hrs. no signs of any missed best pulse tasks due to Firefox activity.
the experiment continues. i suppose i'll let it run like this over a week.
You can keep going, but, I've already given it up as some rare problem in Linux that turns into a PITA on the Mac. I've seen a few of those in the years working with Petri's App. It seems to be completely random, sorta like the Bad Best Pulse problem that's been around for over a year, and sometimes misses pulses without a Browser ever being run. I have a couple of tasks that will run in the benchmark App fine one test, and fail the next. Then the next run will only be One failure while the next are both successes. Seems the only way to avoid it on the Mac is to not run BOINC on the GPU you use for Browsing, it's a shame I got so use to running zi3v for over a year without having to worry about it. With zi3v I would leave FireFox open for days without having any trouble. Can't do that with this App. Hopefully the next version will change that, or at least maybe the Bad Best Pulse fix will also solve this problem, if the Bad Best Pulse problem is ever solved...
Until then I'll just keep telling BOINC to ignore the 750Ti I have running the monitor. The new 1070 meanwhile is tearing it up, around 90 secs for a BLC12 and 2.5 minutes for an Arecibo VLAR. It'd be real nice if I had two of them in the machine to make up that that lost slot.
ID: 1955108 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955148 - Posted: 13 Sep 2018, 10:44:19 UTC - in response to Message 1955104.  

. . If you just closed the manager, in Linux the client would still be running. You need to shut down the client and restart it. I think you said you were running the repository version of BOINC. So open a terminal window, change to the /var/lib/boinc-client folder to be sure, and type "sudo service boinc-client restart". It should work from anywhere but ... Then re-open the Boinc Manager.

Stephen

:)
Not sure about repository versions, would think it doesn't matter, but when you exit the Manager the first time you run it, it asks you whether you want to quit the client and all tasks when you exit or quit the Manager. And remembers from then on. So always cleanly shuts down both the Client and Manager and all running tasks from the Exit dialog of the Manager.


For whatever reason, Lubuntu running the Boinc that Tbar posted, doesn't seem to "remember" on either system, that I said "yes, I want to shut the tasks down".

Tom
A proud member of the OFA (Old Farts Association).
ID: 1955148 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955149 - Posted: 13 Sep 2018, 10:46:20 UTC - in response to Message 1955084.  

Gnome System Monitor in the Ubuntu Software Catalog.



Thank you for the pointer.

I got up this morning after adding a 2nd gpu to my single e5-2670 and that system is off, not responding to power button. No time to troubleshoot. Hope it is a remove a gpu restart :)

Tom
A proud member of the OFA (Old Farts Association).
ID: 1955149 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955489 - Posted: 15 Sep 2018, 12:49:58 UTC - in response to Message 1955149.  

Gnome System Monitor in the Ubuntu Software Catalog.



Thank you for the pointer.

I got up this morning after adding a 2nd gpu to my single e5-2670 and that system is off, not responding to power button. No time to troubleshoot. Hope it is a remove a gpu restart :)

Tom


It appears I over loaded the PSU. Its up and crunching overnight with an 850 watt PSU.

Tom
A proud member of the OFA (Old Farts Association).
ID: 1955489 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955490 - Posted: 15 Sep 2018, 12:51:46 UTC

To -nobs or not to -nobs?

The latest setup I am running. The "Multi-Gpu" CUDA91 doesn't have the -nobs in the command line. And seems to be running very fast.

Should I have the -nobs on there or not?

Tom
A proud member of the OFA (Old Farts Association).
ID: 1955490 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955492 - Posted: 15 Sep 2018, 12:52:53 UTC

Question about GTX 1080Ti's.

I may have an opportunity to upgrade to a 1080Ti.

Does it run single task or 3 tasks at a time?

Thanks,

Tom
A proud member of the OFA (Old Farts Association).
ID: 1955492 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1955521 - Posted: 15 Sep 2018, 15:01:05 UTC - in response to Message 1955490.  

To -nobs or not to -nobs?

The latest setup I am running. The "Multi-Gpu" CUDA91 doesn't have the -nobs in the command line. And seems to be running very fast.

Should I have the -nobs on there or not?

Tom


. . I run my rigs with -nobs. Just be sure you have enough free CPU power to support it, that is one core per Task(/GPU) and one for the pot.

. . The consensus for running singles or doubles as best I know is to run singles. It is not like SoG in that respect.

Stephen

. .
ID: 1955521 · Report as offensive     Reply Quote
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1955526 - Posted: 15 Sep 2018, 15:12:35 UTC - in response to Message 1955492.  

Question about GTX 1080Ti's.

I may have an opportunity to upgrade to a 1080Ti.

Does it run single task or 3 tasks at a time?

Thanks,

Tom

3 tasks with SoG on Windows

1 task with cuda in linux
ID: 1955526 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1955528 - Posted: 15 Sep 2018, 15:22:09 UTC - in response to Message 1955521.  
Last modified: 15 Sep 2018, 15:23:52 UTC


. . The consensus for running singles or doubles as best I know is to run singles. It is not like SoG in that respect.


Look at the #2 line of the supplied README in the docs folder (this is for the CUDA9.2 builds)):

Seti@home Special Cuda Multibeam V8


This App will only work on GPUs that are Compute Capability 6.1 or higher, in Ubuntu equivalent 16.04 or higher.
See list of supported GPUs here, https://en.wikipedia.org/wiki/CUDA#GPUs_supported

For best use;
1) Use a Newer CUDA 9.2 driver, testing was with 396.51 
2) Run One Task per GPU
3) The commandline for -unroll should be set for Your GPU using Compute Units as reference.
A GPU with 28 CUs should be set to -unroll 28 in the app_info.xml, raise or lower the number to match Your GPU. With this version -unroll autotune and Blocking Sync is set by default, this will automatically set the unroll to match your compute units. Override Autotune with the cmdline -unroll N, Blocking Sync override is -nobs
4) If you wish to use 100% CPU per task, add the command -nobs to the app_info.xml.
<cmdline>-nobs</cmdline>
5) The App may give Incorrect results on a restarted task. One way to avoid restarted tasks is to set the checkpoint higher than the task's estimated run-time, and also avoid suspending/resuming a task.
6) If you have an AMD CPU move the CPU App in the folder 'For AMD CPUs' to the root level and see if that works better.
7) Petri says the cmdline -pfb 32 does give better results on his machine.
<cmdline>-nobs -pfb 32</cmdline>

ID: 1955528 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955546 - Posted: 15 Sep 2018, 17:18:59 UTC - in response to Message 1955521.  

To -nobs or not to -nobs?

The latest setup I am running. The "Multi-Gpu" CUDA91 doesn't have the -nobs in the command line. And seems to be running very fast.

Should I have the -nobs on there or not?

Tom


. . I run my rigs with -nobs. Just be sure you have enough free CPU power to support it, that is one core per Task(/GPU) and one for the pot.

. . The consensus for running singles or doubles as best I know is to run singles. It is not like SoG in that respect.

Stephen

. .


I don't know if I want to laugh or cry. I have enough cpu cores/power available but both of my e5- machines are plugged into one UPS. After a little while it start screaming, "I'm overloaded, I'm overloaded.... etc." I have had to drop the -nobs from both machines.

Tom :(
A proud member of the OFA (Old Farts Association).
ID: 1955546 · Report as offensive     Reply Quote
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1955547 - Posted: 15 Sep 2018, 17:21:56 UTC - in response to Message 1955528.  

Look at the #2 line of the supplied README in the docs folder (this is for the CUDA9.2 builds)):

Seti@home Special Cuda Multibeam V8


This App will only work on GPUs that are Compute Capability 6.1 or higher, in Ubuntu equivalent 16.04 or higher.
See list of supported GPUs here, https://en.wikipedia.org/wiki/CUDA#GPUs_supported

For best use;
1) Use a Newer CUDA 9.2 driver, testing was with 396.51 
2) Run One Task per GPU
3) The commandline for -unroll should be set for Your GPU using Compute Units as reference.
A GPU with 28 CUs should be set to -unroll 28 in the app_info.xml, raise or lower the number to match Your GPU. With this version -unroll autotune and Blocking Sync is set by default, this will automatically set the unroll to match your compute units. Override Autotune with the cmdline -unroll N, Blocking Sync override is -nobs
4) If you wish to use 100% CPU per task, add the command -nobs to the app_info.xml.
<cmdline>-nobs</cmdline>
5) The App may give Incorrect results on a restarted task. One way to avoid restarted tasks is to set the checkpoint higher than the task's estimated run-time, and also avoid suspending/resuming a task.
6) If you have an AMD CPU move the CPU App in the folder 'For AMD CPUs' to the root level and see if that works better.
7) Petri says the cmdline -pfb 32 does give better results on his machine.
<cmdline>-nobs -pfb 32</cmdline>


Thank you Juan.
Like way to many people I don't read the docs as often as I should.

Tom
A proud member of the OFA (Old Farts Association).
ID: 1955547 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1955625 - Posted: 16 Sep 2018, 0:15:33 UTC - in response to Message 1955546.  

I just upgraded my 4 gpu host with a new UPS because the original one was constantly screaming it was overloaded. Now the new one only occasionally screams it is overloaded when it goes over 1080 Watts.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1955625 · Report as offensive     Reply Quote
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1955629 - Posted: 16 Sep 2018, 0:24:48 UTC - in response to Message 1955625.  

I just upgraded my 4 gpu host with a new UPS because the original one was constantly screaming it was overloaded. Now the new one only occasionally screams it is overloaded when it goes over 1080 Watts.

What UPS you use? I'm thinking to buy one here too.
ID: 1955629 · Report as offensive     Reply Quote
Previous · 1 . . . 73 · 74 · 75 · 76 · 77 · 78 · 79 . . . 162 · Next

Message boards : Number crunching : Setting up Linux to crunch CUDA90 and above for Windows users


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.