Running out of WU on Tuesdays

Message boards : Number crunching : Running out of WU on Tuesdays
Message board moderation

To post messages, you must log in.

AuthorMessage
Phoenix
Volunteer tester

Send message
Joined: 11 Sep 99
Posts: 6
Credit: 31,871,602
RAC: 77,714
United Kingdom
Message 1880121 - Posted: 24 Jul 2017, 20:16:06 UTC

I am aware the project is under maintenance on Tuesdays, but despite having the preference set to hold 10days plus an extra 3 days I run out on my main Number crunching rig (Xeon 44Core with Nvidia 980 doing an addition 7GPU work units)

Would this be because I have the some other devices such as a couple of laptops that are much slower still working through units as well, so its classes them as 10days worth of stored WU's across my whole account?

24/07/2017 21:07:26 | SETI@home | No tasks sent
24/07/2017 21:07:26 | SETI@home | No tasks are available for AstroPulse v7
24/07/2017 21:07:26 | SETI@home | No tasks are available for SETI@home v8
24/07/2017 21:07:26 | SETI@home | This computer has reached a limit on tasks in progress


Thanks
ID: 1880121 · Report as offensive
AlProject Donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1387
Credit: 260,755,621
RAC: 470,956
United States
Message 1880124 - Posted: 24 Jul 2017, 20:34:27 UTC - in response to Message 1880121.  
Last modified: 24 Jul 2017, 20:37:39 UTC

Hey Phoenix, this has been hashed over in a number of threads, but the gist of it is, that in your instance with that machine, the 10 day setting is pretty much irrelevant. You get 100 tasks per GPU (I believe there is a total limit on these too, others more knowledgeable will hopefully chime in), and 100 CPU tasks, period. Regardless of the number of CPUs in your system, or the number of cores. There has been some opaque discussion as to how these numbers might not be cast in stone, if you do some searching. I have the same problem with a few of my rigs, the ones with 48-56 cores and 3-6 GPU's in them. They are pretty much empty within 2-4 hours, but I haven't taken the time to research it further. It hasn't rated high enough on my To Do list to deal with it right now, as I've got a couple much bigger non-SETI fish to fry. Maybe when things (hopefully) settle down in the late fall/early winter. Hope this helps a little.

ID: 1880124 · Report as offensive
Profile betregerProject Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 6446
Credit: 15,562,827
RAC: 10,158
United States
Message 1880125 - Posted: 24 Jul 2017, 20:36:43 UTC

To avoid overloading the servers the project limits each host to max 100 tasks per CPU and max 100 tasks per GPU. That beast of yours chews thru that in less than 10 hrs obviously.
ID: 1880125 · Report as offensive
Profile Brent Norman
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1824
Credit: 108,053,034
RAC: 457,853
Canada
Message 1880126 - Posted: 24 Jul 2017, 20:39:32 UTC - in response to Message 1880125.  

0.23 days actually on his CPU.
ID: 1880126 · Report as offensive
Profile Wiggo "Socialist"
Avatar

Send message
Joined: 24 Jan 00
Posts: 12611
Credit: 169,598,224
RAC: 87,158
Australia
Message 1880132 - Posted: 24 Jul 2017, 20:50:17 UTC

There is a hard limit of 100 CPU tasks and 100 tasks per GPU per PC so asking for 10 days worth of tasks isn't going to get you anymore work than that.

...I run out on my main Number crunching rig (Xeon 44Core with Nvidia 980 doing an addition 7GPU work units)

Now I know why your GPU tasks take 10x longer to do than what my 1060's can do them in.

Running that old Cuda 32 app isn't helping either, Cuda 50 would've been better and running a max of 3 tasks which was the sweet spot for a 980.

But you would do so much better using the much newer SoG app instead of those old Cuda apps and just 1 task at a time (2 at the most).

I generally run out of work on my 1060's just before the end of the weekly outrage though my other hardware is fine to make it through, but then I only 4 core rigs and not a 44 core rig.

Cheers.
ID: 1880132 · Report as offensive
Profile DarrellProject Donor
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,374,701
RAC: 498
United States
Message 1880133 - Posted: 24 Jul 2017, 20:52:03 UTC
Last modified: 24 Jul 2017, 20:54:03 UTC

http://boinc.berkeley.edu/projects.php

This is a listing of projects from the Boinc website. The Boincstats website listing of projects is three times larger. Surely there is at least one project that you wouldn't mind giving some time to while the Seti project is shutdown for maintenance.

That was the whole purpose behind developing Boinc.
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1880133 · Report as offensive
Profile Tom Miller
Volunteer tester
Avatar

Send message
Joined: 28 Nov 02
Posts: 768
Credit: 18,732,445
RAC: 14,660
United States
Message 1880226 - Posted: 25 Jul 2017, 3:10:59 UTC - in response to Message 1880133.  

http://boinc.berkeley.edu/projects.php

This is a listing of projects from the Boinc website. The Boincstats website listing of projects is three times larger. Surely there is at least one project that you wouldn't mind giving some time to while the Seti project is shutdown for maintenance.

That was the whole purpose behind developing Boinc.


I have added GPUGRID (Nvidia gpu's only) as a Zero level project. If/when my gpu runs out of tasks, it downloads a "monster" gpu task that will keep it busy for the next 12-36 hours or so (runs 12-14 hours on a Gtx 1080).

I also run SetiBeta (cpu only), Rosseta, World Community Grid, Citizen Science Grid and Climateprediction. All are set at a resource level of 1 while Seti is either at 100 or 1,000 depending on what I reset it lately.

While I am chiefly a Seti cruncher, all the other projects are worthy because of either research on Cancer/Zika/etc or other medical reserch or are working on modeling the global warming issues by trying to make better climate models.

If you have enough cores there are parameters available on a project by project basis that will allow you to limit say WCG to no more than 4 cores. The same ideas can be applied on all the rest of the projects.

HTH,
Tom
"You are entitled to your own opinion but not to your own facts." Senator and Professor Patrick Moynihan
---
https://GalensonConsulting.WordPress.com
ID: 1880226 · Report as offensive
rob smithProject Donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 15221
Credit: 252,874,900
RAC: 322,928
United Kingdom
Message 1880238 - Posted: 25 Jul 2017, 4:57:43 UTC

I have various "zero" projects, and as most of my crunchers run out during the weekly outrage I either do heavy dust bug evictions or allow the which ever other project a chance to have my services.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1880238 · Report as offensive
Phoenix
Volunteer tester

Send message
Joined: 11 Sep 99
Posts: 6
Credit: 31,871,602
RAC: 77,714
United Kingdom
Message 1880394 - Posted: 26 Jul 2017, 14:28:16 UTC - in response to Message 1880132.  

Wiggo - How'd you find out what Cuda I was running? My install was from Lunatics Seti so it created an Appdata.xml file so I could modify such settings as Cuda core counts. Have to let us know about this Cuda 50.

I settled on 7WU as I ran GPUz for monitoring purposes and reach 90% util, whilst on default I was finding it was only utilizing the GPU to a much lesser amount. 30% if I recall correctly.

Thanks for all the advice guys, glad I'm not the only one, and I'll have to look into setting up a zero project to fill in the gap.
ID: 1880394 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 8894
Credit: 115,323,607
RAC: 70,427
Australia
Message 1880536 - Posted: 27 Jul 2017, 6:32:52 UTC - in response to Message 1880394.  

How'd you find out what Cuda I was running?

When the system is working better than it is now, you can look at your validated WU results, and it shows in there what application processed that particular WU.
It appears you're using CUDAA32, which is even worse than using CUDA50.
If you want to process as much work as possible, you would be better off with SoG, and running only 1 WU at a time.

I settled on 7WU as I ran GPUz for monitoring purposes and reach 90% util, whilst on default I was finding it was only utilizing the GPU to a much lesser amount. 30% if I recall correctly.

How much (or how little) the GPU and Memory controller loads are isn't important.
What is important is how many WUs you can process each hour. The more you can do each hour, the more efficient your system is.
And running more than 2 WUs at a time, except on really high end hardware, using any of the the CUDA applications will result in very poor performance.
Your GTX980 is processing work only slightly faster than my GTX 750Tis.

My GTX 750Ti does a shorty in roughly 20 min, crunching 2 at a time.
So 3 per hour *2 gives 6 shorties per hour.
Your GTX980 does a shorty in roughly 50 min, crunching 7 at a time.
So 1.25 per hour *7 gives 8.75 shorties per hour.
With the right application and settings, that video card is capable of doing more than 15 shorties per hour (probably close to 18).

Your settings have maxed out your GPUs power consumption, however it's work output is extremely low.
Grant
Darwin NT
ID: 1880536 · Report as offensive
Profile Wiggo "Socialist"
Avatar

Send message
Joined: 24 Jan 00
Posts: 12611
Credit: 169,598,224
RAC: 87,158
Australia
Message 1880542 - Posted: 27 Jul 2017, 7:08:23 UTC

Wiggo - How'd you find out what Cuda I was running? My install was from Lunatics Seti so it created an Appdata.xml file so I could modify such settings as Cuda core counts. Have to let us know about this Cuda 50.

I settled on 7WU as I ran GPUz for monitoring purposes and reach 90% util, whilst on default I was finding it was only utilizing the GPU to a much lesser amount. 30% if I recall correctly.

Thanks for all the advice guys, glad I'm not the only one, and I'll have to look into setting up a zero project to fill in the gap.

"Elementary my dear Watson", I just checked your task results. ;-)

Quite frankly if you reserving a CPU core for each GPU task and running the SOG app from the Lunatics 0.45 Beta 6, using the installer you must scroll down to select the best app for your GPU yourself (unlike the CPU, but then there's now a better CPU app for certain older 64-bit non AVX CPU's and an even later version SoG app, but no installer for those yet, good enough though).

If you were running 1 SOG task on that 980 I'd expect that you'll get much closer a 50K RAC and running 2 would get you even further (whether you can run 3 tasks profitably only long term testing will tell), but you'd do a lot better than what you're doing now. ;-)

Cheers.
ID: 1880542 · Report as offensive
Phoenix
Volunteer tester

Send message
Joined: 11 Sep 99
Posts: 6
Credit: 31,871,602
RAC: 77,714
United Kingdom
Message 1880618 - Posted: 27 Jul 2017, 19:28:30 UTC - in response to Message 1880542.  

Thanks Wiggo - Downloaded latest Beta and now receiving SoG GPU tasks :)
ID: 1880618 · Report as offensive

Message boards : Number crunching : Running out of WU on Tuesdays


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.