GPU task not starting after one finishes.

Questions and Answers : Windows : GPU task not starting after one finishes.
Message board moderation

To post messages, you must log in.

AuthorMessage
Jem

Send message
Joined: 3 Apr 99
Posts: 2
Credit: 26,888,908
RAC: 0
United Kingdom
Message 991158 - Posted: 22 Apr 2010, 18:44:04 UTC

Hi
Just upgraded my machine to Win 7 x64 and installed Boinc 6.10.43 and everthing seemed to be working ok. But once a GPU (cuda23) has finished it does not start crunching the next WU instead it downloads another 7 WUs. I've currently got about 42 cuda23 with a status of "Ready to Start" I can restart the machine and it will complete one GPU task and then download another 7 units... Any Ideas?? Message log below

22/04/2010 18:41:03 Starting BOINC client version 6.10.43 for windows_x86_64
22/04/2010 18:41:03 log flags: file_xfer, sched_ops, task
22/04/2010 18:41:03 Libraries: libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
22/04/2010 18:41:03 Data directory: C:\ProgramData\BOINC
22/04/2010 18:41:03 Running under account Jem
22/04/2010 18:41:03 Processor: 4 GenuineIntel Intel(R) Core(TM)2 Quad CPU Q8300 @ 2.50GHz [Family 6 Model 23 Stepping 10]
22/04/2010 18:41:03 Processor: 2.00 MB cache
22/04/2010 18:41:03 Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 syscall nx lm vmx tm2 pbe
22/04/2010 18:41:03 OS: Microsoft Windows 7: x64 Edition, (06.01.7600.00)
22/04/2010 18:41:03 Memory: 4.00 GB physical, 8.00 GB virtual
22/04/2010 18:41:03 Disk: 29.72 GB total, 9.85 GB free
22/04/2010 18:41:03 Local time is UTC +1 hours
22/04/2010 18:41:03 NVIDIA GPU 0: GeForce GT 240 (driver version 19745, CUDA version 3000, compute capability 1.2, 475MB, 257 GFLOPS peak)
22/04/2010 18:41:03 SETI@home URL http://setiathome.berkeley.edu/; Computer ID 5379978; resource share 100
22/04/2010 18:41:03 SETI@home General prefs: from SETI@home (last modified 21-Apr-2010 20:15:35)
22/04/2010 18:41:03 SETI@home Computer location: home
22/04/2010 18:41:03 SETI@home General prefs: no separate prefs for home; using your defaults
22/04/2010 18:41:03 Reading preferences override file
22/04/2010 18:41:03 Preferences:
22/04/2010 18:41:03 max memory usage when active: 2047.62MB
22/04/2010 18:41:03 max memory usage when idle: 3685.72MB
22/04/2010 18:41:03 max disk usage: 9.88GB
22/04/2010 18:41:03 suspend work if non-BOINC CPU load exceeds 70 %
22/04/2010 18:41:03 (to change, visit the web site of an attached project,
22/04/2010 18:41:03 or click on Preferences)
22/04/2010 18:41:03 Not using a proxy
22/04/2010 18:41:03 SETI@home Restarting task 29dc06af.21201.890.5.10.183_1 using setiathome_enhanced version 603
22/04/2010 18:41:03 SETI@home Restarting task 29dc06af.21201.890.5.10.196_1 using setiathome_enhanced version 603
22/04/2010 18:41:03 SETI@home Restarting task 29dc06af.21201.890.5.10.199_0 using setiathome_enhanced version 603
22/04/2010 18:41:03 SETI@home Restarting task 29dc06af.21201.890.5.10.217_0 using setiathome_enhanced version 603
22/04/2010 18:41:03 SETI@home Starting 28dc06aa.20281.890.7.10.246_0
22/04/2010 18:41:04 SETI@home Starting task 28dc06aa.20281.890.7.10.246_0 using setiathome_enhanced version 609
22/04/2010 18:52:05 SETI@home Computation for task 28dc06aa.20281.890.7.10.246_0 finished
22/04/2010 18:52:06 SETI@home Starting 27ja07ag.21227.481.9.10.62_0
22/04/2010 18:52:06 SETI@home Starting task 27ja07ag.21227.481.9.10.62_0 using setiathome_enhanced version 609
22/04/2010 18:52:08 SETI@home Started upload of 28dc06aa.20281.890.7.10.246_0_0
22/04/2010 18:52:13 SETI@home Finished upload of 28dc06aa.20281.890.7.10.246_0_0
22/04/2010 19:16:28 SETI@home Computation for task 27ja07ag.21227.481.9.10.62_0 finished
22/04/2010 19:16:31 SETI@home Started upload of 27ja07ag.21227.481.9.10.62_0_0
22/04/2010 19:16:33 SETI@home Sending scheduler request: To fetch work.
22/04/2010 19:16:33 SETI@home Reporting 1 completed tasks, requesting new tasks for GPU
22/04/2010 19:16:36 SETI@home Finished upload of 27ja07ag.21227.481.9.10.62_0_0
22/04/2010 19:16:38 SETI@home Scheduler request completed: got 7 new tasks
22/04/2010 19:16:40 SETI@home Started download of 24ja07af.16290.88411.13.10.148
22/04/2010 19:16:40 SETI@home Started download of 24ja07af.18937.12751.14.10.146
22/04/2010 19:16:45 SETI@home Finished download of 24ja07af.16290.88411.13.10.148
22/04/2010 19:16:45 SETI@home Finished download of 24ja07af.18937.12751.14.10.146
22/04/2010 19:16:45 SETI@home Started download of 24ja07af.16290.88411.13.10.152
22/04/2010 19:16:45 SETI@home Started download of 24ja07af.16290.88411.13.10.165
22/04/2010 19:16:49 SETI@home Finished download of 24ja07af.16290.88411.13.10.152
22/04/2010 19:16:49 SETI@home Started download of 24ja07af.16290.88411.13.10.150
22/04/2010 19:16:52 SETI@home Finished download of 24ja07af.16290.88411.13.10.165
22/04/2010 19:16:52 SETI@home Started download of 24ja07af.16290.88411.13.10.175
22/04/2010 19:16:53 SETI@home Finished download of 24ja07af.16290.88411.13.10.150
22/04/2010 19:16:53 SETI@home Started download of 24ja07af.18937.12751.14.10.135
22/04/2010 19:16:55 SETI@home Finished download of 24ja07af.16290.88411.13.10.175
22/04/2010 19:16:57 SETI@home Finished download of 24ja07af.18937.12751.14.10.135
22/04/2010 19:17:54 SETI@home Sending scheduler request: To fetch work.
22/04/2010 19:17:54 SETI@home Reporting 1 completed tasks, requesting new tasks for GPU
22/04/2010 19:17:59 SETI@home Scheduler request completed: got 0 new tasks
22/04/2010 19:17:59 SETI@home Message from server: (Project has no jobs available)
22/04/2010 19:18:14 SETI@home Sending scheduler request: To fetch work.
22/04/2010 19:18:14 SETI@home Requesting new tasks for GPU
22/04/2010 19:18:19 SETI@home Scheduler request completed: got 0 new tasks
22/04/2010 19:18:19 SETI@home Message from server: (Project has no jobs available)
22/04/2010 19:19:34 SETI@home Sending scheduler request: To fetch work.
22/04/2010 19:19:34 SETI@home Requesting new tasks for GPU
22/04/2010 19:19:39 SETI@home Scheduler request completed: got 3 new tasks
22/04/2010 19:19:41 SETI@home Started download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:19:41 SETI@home Started download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:19:43 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.10: HTTP error
22/04/2010 19:19:43 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:19:43 SETI@home Temporarily failed download of 24ja07af.20875.17250.15.10.130: HTTP error
22/04/2010 19:19:43 SETI@home Backing off 1 min 0 sec on download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:19:43 SETI@home Started download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:19:45 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.2: HTTP error
22/04/2010 19:19:45 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:20:15 SETI@home update requested by user
22/04/2010 19:20:19 SETI@home Sending scheduler request: Requested by user.
22/04/2010 19:20:19 SETI@home Not reporting or requesting tasks
22/04/2010 19:20:24 SETI@home Scheduler request completed
22/04/2010 19:20:45 SETI@home Started download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:20:45 SETI@home Started download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:20:46 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.10: HTTP error
22/04/2010 19:20:46 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:20:46 SETI@home Temporarily failed download of 24ja07af.20875.17250.15.10.130: HTTP error
22/04/2010 19:20:46 SETI@home Backing off 1 min 0 sec on download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:22:59 SETI@home Started download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:22:59 SETI@home Started download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:23:00 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.10: HTTP error
22/04/2010 19:23:00 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:23:00 SETI@home Temporarily failed download of 24ja07af.20875.17250.15.10.130: HTTP error
22/04/2010 19:23:00 SETI@home Backing off 1 min 0 sec on download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:23:30 SETI@home Started download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:23:30 SETI@home Started download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:23:32 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.10: HTTP error
22/04/2010 19:23:32 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:23:32 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.2: HTTP error
22/04/2010 19:23:32 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:23:41 SETI@home Started download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:23:42 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.10: HTTP error
22/04/2010 19:23:42 SETI@home Backing off 1 min 44 sec on download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:24:01 SETI@home Started download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:24:02 SETI@home Temporarily failed download of 24ja07af.20875.17250.15.10.130: HTTP error
22/04/2010 19:24:02 SETI@home Backing off 1 min 0 sec on download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:24:32 SETI@home Started download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:24:34 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.2: HTTP error
22/04/2010 19:24:34 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:25:34 SETI@home Started download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:25:34 SETI@home Started download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:25:35 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.10: HTTP error
22/04/2010 19:25:35 SETI@home Backing off 5 min 35 sec on download of 24ja07af.18937.13160.14.10.10
22/04/2010 19:25:35 SETI@home Temporarily failed download of 24ja07af.20875.17250.15.10.130: HTTP error
22/04/2010 19:25:35 SETI@home Backing off 1 min 50 sec on download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:26:45 SETI@home Started download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:26:46 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.2: HTTP error
22/04/2010 19:26:46 SETI@home Backing off 1 min 0 sec on download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:30:27 SETI@home Started download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:30:27 SETI@home Started download of 24ja07af.18937.13160.14.10.2
22/04/2010 19:30:28 SETI@home Temporarily failed download of 24ja07af.20875.17250.15.10.130: HTTP error
22/04/2010 19:30:28 SETI@home Backing off 6 min 15 sec on download of 24ja07af.20875.17250.15.10.130
22/04/2010 19:30:28 SETI@home Temporarily failed download of 24ja07af.18937.13160.14.10.2: HTTP error
22/04/2010 19:30:28 SETI@home Backing off 1 min 22 sec on download of 24ja07af.18937.13160.14.10.2


Thanks
ID: 991158 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 991187 - Posted: 22 Apr 2010, 20:19:51 UTC - in response to Message 991158.  

Perhaps there's not enough GPU memory available after some tasks, until a reboot clears it again. To see if that's the case, you could enable <cpu_sched_debug> logging (with cc_config.xml).

Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)

SETI@home classic workunits 3,758
SETI@home classic CPU time 66,520 hours
ID: 991187 · Report as offensive
Calphor

Send message
Joined: 28 Jul 00
Posts: 2
Credit: 459,428
RAC: 0
United States
Message 991347 - Posted: 23 Apr 2010, 11:45:23 UTC - in response to Message 991158.  

I'm getting the same error on a WinXP 32bit box. Sometimes it will download, but most of the time it backs off with an HTTP error.
ID: 991347 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 991349 - Posted: 23 Apr 2010, 12:21:44 UTC - in response to Message 991347.  

That is not the same error, unless your CUDA tasks don't start either after downloading.

Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)

SETI@home classic workunits 3,758
SETI@home classic CPU time 66,520 hours
ID: 991349 · Report as offensive
Calphor

Send message
Joined: 28 Jul 00
Posts: 2
Credit: 459,428
RAC: 0
United States
Message 991486 - Posted: 24 Apr 2010, 0:22:49 UTC

You are correct. I saw the HTTP error in the log and made an assumption. Mea Culpa!
ID: 991486 · Report as offensive
Jem

Send message
Joined: 3 Apr 99
Posts: 2
Credit: 26,888,908
RAC: 0
United Kingdom
Message 992773 - Posted: 29 Apr 2010, 20:24:52 UTC - in response to Message 991187.  

Hi, sorry about the slow reply.
I've turned on the <cpu_sched_debug> (see log below) but not sure what it is telling me? When you say GPU memory do you mean my graphics card memory (Which is now 512mb) I've set my Colour Depth to "medium 16 bit" and the cuda WU's will keep running ok. But I don't really want to do that as media centre whinges... I've just upgrade my machine to a 512mb card from a 256mb card that use to run the cuda WU's ok. I'm just a bit confused as to what is going on and how I can free up some memory? Also is it a bug in the code that keeps downloading the cuda WU's when it runs out of memory as I have over 100 to crunch now.

Thanks

29/04/2010 21:23:00 Starting BOINC client version 6.10.43 for windows_x86_64
29/04/2010 21:23:00 log flags: file_xfer, sched_ops, task, coproc_debug, cpu_sched_debug
29/04/2010 21:23:00 Libraries: libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
29/04/2010 21:23:00 Data directory: C:\ProgramData\BOINC
29/04/2010 21:23:00 Running under account Jem
29/04/2010 21:23:00 Processor: 4 GenuineIntel Intel(R) Core(TM)2 Quad CPU Q8300 @ 2.50GHz [Family 6 Model 23 Stepping 10]
29/04/2010 21:23:00 Processor: 2.00 MB cache
29/04/2010 21:23:00 Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss htt tm pni ssse3 cx16 sse4_1 syscall nx lm vmx tm2 pbe
29/04/2010 21:23:00 OS: Microsoft Windows 7: x64 Edition, (06.01.7600.00)
29/04/2010 21:23:00 Memory: 4.00 GB physical, 8.00 GB virtual
29/04/2010 21:23:00 Disk: 29.72 GB total, 5.77 GB free
29/04/2010 21:23:00 Local time is UTC +1 hours
29/04/2010 21:23:00 NVIDIA GPU 0: GeForce GT 240 (driver version 19745, CUDA version 3000, compute capability 1.2, 475MB, 257 GFLOPS peak)
29/04/2010 21:23:00 NVIDIA library reports 1 GPU
29/04/2010 21:23:00 No ATI library found.
29/04/2010 21:23:00 SETI@home URL http://setiathome.berkeley.edu/; Computer ID 5379978; resource share 100
29/04/2010 21:23:00 SETI@home General prefs: from SETI@home (last modified 21-Apr-2010 20:15:35)
29/04/2010 21:23:00 SETI@home Computer location: home
29/04/2010 21:23:00 SETI@home General prefs: no separate prefs for home; using your defaults
29/04/2010 21:23:00 Reading preferences override file
29/04/2010 21:23:00 Preferences:
29/04/2010 21:23:00 max memory usage when active: 2047.62MB
29/04/2010 21:23:00 max memory usage when idle: 3685.72MB
29/04/2010 21:23:00 max disk usage: 5.00GB
29/04/2010 21:23:00 suspend work if non-BOINC CPU load exceeds 95 %
29/04/2010 21:23:00 (to change, visit the web site of an attached project,
29/04/2010 21:23:00 or click on Preferences)
29/04/2010 21:23:00 [cpu_sched_debug] Request CPU reschedule: Prefs update
29/04/2010 21:23:00 [cpu_sched_debug] Request CPU reschedule: Startup
29/04/2010 21:23:00 Not using a proxy
29/04/2010 21:23:00 [cpu_sched_debug] Request CPU reschedule: Idle state change
29/04/2010 21:23:00 [cpu_sched_debug] Request CPU reschedule: Scheduling period elapsed.
29/04/2010 21:23:00 [cpu_sched_debug] schedule_cpus(): start
29/04/2010 21:23:00 SETI@home [cpu_sched_debug] scheduling 28dc06aa.20281.17973.7.10.247_1 (coprocessor job, FIFO)
29/04/2010 21:23:00 [cpu_sched_debug] reserving 1.000000 of coproc CUDA
29/04/2010 21:23:00 [cpu_sched_debug] Request enforce CPU schedule: schedule_cpus
29/04/2010 21:23:00 [cpu_sched_debug] enforce_schedule(): start
29/04/2010 21:23:00 [cpu_sched_debug] preliminary job list:
29/04/2010 21:23:00 SETI@home [cpu_sched_debug] 0: 28dc06aa.20281.17973.7.10.247_1 (MD: no; UTS: no)
29/04/2010 21:23:00 [cpu_sched_debug] final job list:
29/04/2010 21:23:00 SETI@home [cpu_sched_debug] 0: 28dc06aa.20281.17973.7.10.247_1 (MD: no; UTS: no)
29/04/2010 21:23:00 SETI@home [coproc_debug] Assigning CUDA instance 0 to 28dc06aa.20281.17973.7.10.247_1
29/04/2010 21:23:01 SETI@home [cpu_sched_debug] scheduling 28dc06aa.20281.17973.7.10.247_1
29/04/2010 21:23:01 [cpu_sched_debug] using 0.17 out of 4 CPUs
29/04/2010 21:23:01 SETI@home [cpu_sched_debug] 28dc06aa.20281.17973.7.10.247_1 sched state 1 next 2 task state 0
29/04/2010 21:23:01 SETI@home Restarting task 28dc06aa.20281.17973.7.10.247_1 using setiathome_enhanced version 609
29/04/2010 21:23:01 [cpu_sched_debug] enforce_schedule: end
ID: 992773 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 992780 - Posted: 29 Apr 2010, 20:53:18 UTC - in response to Message 992773.  
Last modified: 29 Apr 2010, 20:55:13 UTC

I've turned on the <cpu_sched_debug> (see log below) but not sure what it is telling me.

It should be telling you needed versus available GPU memory, but in the meantime, the developers have changed that to <coproc_debug>. [edit]You should enable both logging options if you still suspect problems.[/edit]

When you say GPU memory do you mean my graphics card memory (Which is now 512mb)?

Yes, since GPU stands for Graphics Processing Unit. ;-)

I've set my Colour Depth to "medium 16 bit" and the cuda WU's will keep running ok. But I don't really want to do that as media centre whinges... I've just upgrade my machine to a 512mb card from a 256mb card that use to run the cuda WU's ok. I'm just a bit confused as to what is going on and how I can free up some memory?

Since you now have more total memory (reported), I don't think it's necessary to free up even more.

Also is it a bug in the code that keeps downloading the cuda WU's when it runs out of memory as I have over 100 to crunch now.

Yes, that's a bug, which is currently addressed by the developers.

Gruß,
Gundolf
ID: 992780 · Report as offensive

Questions and Answers : Windows : GPU task not starting after one finishes.


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.