CUDA units hogging CPU

Questions and Answers : GPU applications : CUDA units hogging CPU
Message board moderation

To post messages, you must log in.

AuthorMessage
Julio Montero

Send message
Joined: 28 Apr 04
Posts: 1
Credit: 335,009
RAC: 0
United States
Message 939741 - Posted: 14 Oct 2009, 4:24:20 UTC

I've noticed that though BOINC (6.6.36) states in the Status section that CUDA units are "Running (.04 CPUS, 1 CUDA)", a check on Windows Task Manager reveals that "setiathome_6.08_windows_intelx86__cuda.exe" is using 50% of all CPU cycles.

Furthermore, BOINC is reporting that 3 tasks are running. Task Manager reports that the CUDA unit uses 50% CPU always, the other two tasks are either at 25% CPU each, or one will get 50% CPU and the other will run at 0% CPU, even though BOINC says that the workunit is running, and the time elapsed for that unit increases.

Is something wrong here? Or is this normal? I thought that CUDA units would run on the GPU, and show very little CPU time, but the current CUDA unit has run only 16% of the way so far and has 1hr 13min clocked on the time elapsed.

I am running Vista Premium 32-bit, an Intel Core 2 Duo at 2.4Ghz, no overclock. The GPU is a Nvidia GeForce 8600 GTS, 256MB VRAM, no overclock.
ID: 939741 · Report as offensive
PP

Send message
Joined: 3 Jul 99
Posts: 42
Credit: 10,012,664
RAC: 0
Sweden
Message 939928 - Posted: 14 Oct 2009, 20:02:05 UTC - in response to Message 939741.  

Are you still using CUDA 2.1? That is a known bug. Upgrade to CUDA 2.3 and Nvidia 190.x and enjoy a 30% increase in speed.
ID: 939928 · Report as offensive
LP

Send message
Joined: 4 May 03
Posts: 10
Credit: 44,775,622
RAC: 61
United States
Message 945020 - Posted: 4 Nov 2009, 12:14:16 UTC - in response to Message 939928.  

Are you still using CUDA 2.1? That is a known bug. Upgrade to CUDA 2.3 and Nvidia 190.x and enjoy a 30% increase in speed.


I experience the exact same problem. It occurs occasionally. I try to identify the cause, but none is conclusive so far.

My display card is GTS 250. Display card drivers are NVIDIA's ver 6.14.11.9038 which includes CUDA 2.3. Please kindly help to advise

1/ what NVIDIA 190.x is, and where it can be checked; and
2/ where I can look up information and resolution to this known bug.

Many thanks.

ID: 945020 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 945023 - Posted: 4 Nov 2009, 12:28:05 UTC - in response to Message 945020.  

Your driver version is apparently 190.38 (just set the dots right:-).

If your video memory suffers from "congestion" (sometimes caused by BOINC scheduling) the GPU tasks "fall back" to the CPU. That condition can only be resolved by a reboot (full power cicle).

Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)

SETI@home classic workunits 3,758
SETI@home classic CPU time 66,520 hours
ID: 945023 · Report as offensive
Profile Loter

Send message
Joined: 18 Jul 01
Posts: 23
Credit: 368,078
RAC: 0
Moldova
Message 945027 - Posted: 4 Nov 2009, 12:46:26 UTC
Last modified: 4 Nov 2009, 13:26:12 UTC

ID: 945027 · Report as offensive
LP

Send message
Joined: 4 May 03
Posts: 10
Credit: 44,775,622
RAC: 61
United States
Message 945061 - Posted: 4 Nov 2009, 18:28:44 UTC - in response to Message 945027.  

hey, take a look at my thread

http://setiathome.berkeley.edu/forum_thread.php?id=56074


Good link. Very informative.

Besides rebooting the system, I have found another getting around -- manually suspend the CUDA task in trouble, so processing continues with the next one. Wait till this next task is finished, resume the one in trouble, and it will continue processing normally.

Gundolf mentioned BOINC scheduling. I was really troubled by BOINC Manager 6.6.41, and now fall back to 6.6.38. Will 6.10.17 help?

Suggestion. I wonder if CUDA architecture supports task priority settings. If BOINC CUDA task can be set to a lower priority (just like BOINC CPU task does), it may help solving part of the issue. Correct?
ID: 945061 · Report as offensive
Wembley
Volunteer tester
Avatar

Send message
Joined: 16 Sep 09
Posts: 429
Credit: 1,844,293
RAC: 0
United States
Message 945151 - Posted: 5 Nov 2009, 0:32:28 UTC

If your graphics card is low on memory, disabling fancy windows themes and such crap does more to help than anything else.

ID: 945151 · Report as offensive
LP

Send message
Joined: 4 May 03
Posts: 10
Credit: 44,775,622
RAC: 61
United States
Message 945221 - Posted: 5 Nov 2009, 9:07:39 UTC - in response to Message 945151.  

If your graphics card is low on memory, disabling fancy windows themes and such crap does more to help than anything else.


Graphic card has 512MB memory. PC has 2GB memory running on XP SP2, with basic "Windows XP" theme, no background, and no screen saver. It seems to be more of a memory congestion problem than low on memory. Question is what can be done to make it not happen again.

One more observation. When the BOINC Service is stopped then restarted, it fails to detect the CUDA device and takes a drastic action to wipe out all CUDA tasks stored in the data directory. But it forgets to inform BOINC Master Database of such action, and as a result, there is a long list of task got stuck in "Your Results".
ID: 945221 · Report as offensive
Wembley
Volunteer tester
Avatar

Send message
Joined: 16 Sep 09
Posts: 429
Credit: 1,844,293
RAC: 0
United States
Message 945283 - Posted: 5 Nov 2009, 21:14:17 UTC - in response to Message 945221.  

One more observation. When the BOINC Service is stopped then restarted, it fails to detect the CUDA device and takes a drastic action to wipe out all CUDA tasks stored in the data directory. But it forgets to inform BOINC Master Database of such action, and as a result, there is a long list of task got stuck in "Your Results".

Use the latest 6.10.17 version of Boinc to prevent the tasks from getting wiped when the cuda device disappears.

Are you using remote desktop?
ID: 945283 · Report as offensive
LP

Send message
Joined: 4 May 03
Posts: 10
Credit: 44,775,622
RAC: 61
United States
Message 945440 - Posted: 6 Nov 2009, 15:22:00 UTC - in response to Message 945283.  

One more observation. When the BOINC Service is stopped then restarted, it fails to detect the CUDA device and takes a drastic action to wipe out all CUDA tasks stored in the data directory. But it forgets to inform BOINC Master Database of such action, and as a result, there is a long list of task got stuck in "Your Results".

Use the latest 6.10.17 version of Boinc to prevent the tasks from getting wiped when the cuda device disappears.

Are you using remote desktop?


Thanks. I will switch to 6.10.17.

No, I am not using remote desktop. Which one is having problem: when this PC is local or remote system?
ID: 945440 · Report as offensive
Wembley
Volunteer tester
Avatar

Send message
Joined: 16 Sep 09
Posts: 429
Credit: 1,844,293
RAC: 0
United States
Message 945471 - Posted: 6 Nov 2009, 19:09:20 UTC - in response to Message 945440.  

No, I am not using remote desktop. Which one is having problem: when this PC is local or remote system?


remote desktop is bad as it uses it's own drivers and disables the cuda drivers when you log in remotely.

Boinc also has problems seeing the cuda device if you are trying to run it in Vista or 7 as a service (protected mode in the install)
ID: 945471 · Report as offensive
Mike, Crunching for Physics

Send message
Joined: 7 Jan 09
Posts: 1
Credit: 7,727,787
RAC: 0
Canada
Message 949902 - Posted: 26 Nov 2009, 14:29:00 UTC

Hello

I am having a similar problem after I upgraded Boinc and my video driver. Currently I am running Boinc 6.10.18. From the Boinc messages, all my hardware is recognized fine

NVIDIA GPU 0: GeForce 9600 GT (driver version 19562, CUDA version 3000, compute capability 1.1, 512MB, 243 GFLOPS peak)

I have run through all the suggestions on this thread to try to correct this problem. Here's what I have noticed on my machine, I have a dual screen setup and when Boinc manager is open on one screen, the cuda task as seen from the task manager is using 20-23%. The GPU temp is also at the idle temperature. If i open a web browser, I'm using Chrome, then the cuda task immediately drops to 2-3% of cpu use and the GPU temp increases. As soon as I close Chrome, the cuda task returns to using 20-23% and the GPU temp drops. Sounds strange, but it happens every time. Anyone heard of this problem, and possible solutions?

In the mean time I guess i'll be leaving Chrome open if i want my GPU to crunch
ID: 949902 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 949905 - Posted: 26 Nov 2009, 15:17:59 UTC - in response to Message 949902.  

Sounds more like you have some problem with that card and the tasks are falling back to the CPU to be crunched on that.
ID: 949905 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 949917 - Posted: 26 Nov 2009, 16:04:44 UTC - in response to Message 949902.  

Ageless seems to be right. See tasks 1430761436 and 1430761387. Though, not all of your CUDA tasks error out.

Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)

SETI@home classic workunits 3,758
SETI@home classic CPU time 66,520 hours
ID: 949917 · Report as offensive

Questions and Answers : GPU applications : CUDA units hogging CPU


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.