BOINC scheduling of CUDA tasks /GPU memory issues

Questions and Answers : GPU applications : BOINC scheduling of CUDA tasks /GPU memory issues
Message board moderation

To post messages, you must log in.

AuthorMessage
Chris

Send message
Joined: 14 Jun 99
Posts: 12
Credit: 10,574,206
RAC: 6
United Kingdom
Message 888207 - Posted: 25 Apr 2009, 10:11:09 UTC

Some observations and thoughts...

The observation.
Like others, I have a number of the CUDA WU's drop back to cpu processing.
If I am around and notice it, I tend to re-boot the system, and gpu processing is usually resumed. However if I'm not there, the task runs for hours rather than minutes, and it tends NOT to revert to gpu processing for the next "CUDA" WU. So some sort of hangover in the GPU memory is a problem. This seems to be exacerbated when BOINC "queue jumps" newly downloaded tasks and puts a running task into "Waiting to run" mode. I have had up to four or five tasks started and then put into that state concurrently. After that the only way I've found to get gpu processing to go again is a re-boot.

BOINC re-schedules the tasks even when the deadlines are weeks away - and they may only be minutes apart.

The thought.
This is pointless - if the projected run time is a few hours and the deadline three weeks, why suspend a task for another where the run time is similar but the deadline a few minutes earlier. This creates the queue of waiting to run tasks, and the potential problem with dropping out of gpu processing. So, a rethink on the BOINC task scheduling to take account of the benefit to be gained to let a cuda task run to its conclusion (where deadlines are more than hours away) would be worth considering.
ID: 888207 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 888216 - Posted: 25 Apr 2009, 11:16:14 UTC

The developers are already busy with that. But do know that when we speak of "the developers", that we're talking about 3 to 4 people. It won't happen overnight and if you think you can change your words into runnable C++ code, then by all means go do it and send it to the BOINC developers for evaluation.

In the mean time, newer CUDA code is being tested in versions after 6.6.20, so they're not sitting still either.
ID: 888216 · Report as offensive
Fred W
Volunteer tester

Send message
Joined: 13 Jun 99
Posts: 2524
Credit: 11,954,210
RAC: 0
United Kingdom
Message 888218 - Posted: 25 Apr 2009, 11:36:54 UTC

And one of the improvements being tested in the 6.6.2x builds is the elimination of the "queue-jumping" of CUDA tasks i.e. getting away from permanent EDF mode.

F.
ID: 888218 · Report as offensive
Chris

Send message
Joined: 14 Jun 99
Posts: 12
Credit: 10,574,206
RAC: 6
United Kingdom
Message 888236 - Posted: 25 Apr 2009, 12:48:03 UTC

Thanks for positive responses - I'm afraid my coding days ended in 1972 with PLAN 3 (who remembers that now), so the C++ option isn't viable!
ID: 888236 · Report as offensive
SmartWombat
Avatar

Send message
Joined: 9 Jan 04
Posts: 64
Credit: 6,577,011
RAC: 0
United Kingdom
Message 890006 - Posted: 30 Apr 2009, 23:53:21 UTC - in response to Message 888236.  

Oh yes, I remember PLAN.
My second assembly language (I learned Z80 first).
PAul

[IMG][/IMG]
ID: 890006 · Report as offensive

Questions and Answers : GPU applications : BOINC scheduling of CUDA tasks /GPU memory issues


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.