Large increase of computation duration

Questions and Answers : GPU applications : Large increase of computation duration
Message board moderation

To post messages, you must log in.

AuthorMessage
Gutierrez

Send message
Joined: 1 Apr 00
Posts: 5
Credit: 2,355,656
RAC: 0
France
Message 998891 - Posted: 26 May 2010, 12:08:18 UTC

Hello,

I discover recently that time to complete units on my GPU has increased drastically. Usualy, a unit was completed in less than 1 hour, and for few days now, it takes often around 3 or 4 hours to complete a unit, sometimes until 10 hours.

Before trying to investigate if a software change on my PC can be responsible of this situation, I prefer to check if someone has already had the same problem ?
perhaps the size of units changed ?


Thanks in advance for any answer.
Jean.
ID: 998891 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 998896 - Posted: 26 May 2010, 13:23:48 UTC - in response to Message 998891.  

I only looked at a couple of your work units but from what I saw it looks like you have hit a patch of VLARs. (Very Low Angle Range) These tend to be pretty hard on your GPU and slow it down a lot. A couple of things you could do is to try running an optimized package from Lunatic's or try the rescheduler tool. There are links and discussions on both over in the numbers crunching forum if you would like to learn more about them.


PROUD MEMBER OF Team Starfire World BOINC
ID: 998896 · Report as offensive
Gutierrez

Send message
Joined: 1 Apr 00
Posts: 5
Credit: 2,355,656
RAC: 0
France
Message 998959 - Posted: 26 May 2010, 18:27:54 UTC - in response to Message 998896.  

Thanks a lot.
Jean.
ID: 998959 · Report as offensive
Profile BilBg
Volunteer tester
Avatar

Send message
Joined: 27 May 07
Posts: 3720
Credit: 9,385,827
RAC: 0
Bulgaria
Message 998983 - Posted: 26 May 2010, 19:58:05 UTC - in response to Message 998959.  


Lunatics' Unified Installer info
http://setiathome.berkeley.edu/forum_thread.php?id=54288&nowrap=true#994550

ReSchedule 1.9 info
http://setiathome.berkeley.edu/forum_thread.php?id=59291



 


- ALF - "Find out what you don't do well ..... then don't do it!" :)
 
ID: 998983 · Report as offensive
artki

Send message
Joined: 15 May 99
Posts: 15
Credit: 446,116
RAC: 0
United States
Message 1002473 - Posted: 10 Jun 2010, 11:04:11 UTC
Last modified: 10 Jun 2010, 11:16:50 UTC

I wonder if something similar is going on with my computer. I've been running seti with the GPU on for about a month. (Ran without the GPU before that due to overheating problems. GPU cooling fan fixed that) Most of the time I even let it run while I'm using the computer since the graphics slowdown is trivial - most of the time.

Just in the last few hours things have changed radically. The estimated completion time for regular workunits (the 6.03 cpu ones, not the 6.08 GPUs) changed from 6-7 hours to 54 hours. These are mostly the same WUs I had yesterday. A gigantic batch of GPU WUs was downloaded a few hours ago. I normally only process 3 or 4 GPU WUs a day - I just got 100! (I ask for an 8 day supply)

The CPU WUs are actually processing in about the same time they usually take - it's the estimated time that has shot upwards.

(I have a quad-core cpu so I process 4 CPU WUs and one GPU WU)

The GPU WU that's being processed runs incredibly slowly now. 41 minutes in and it's only done 0.108%. And when it is running it noticeably impacts the computers graphic performance. The browser is very jumpy. If I play a video the picture is still while the audio plays. Before the there was a mild impact on videos while the GPU was running - sometimes I didn't even bother to disable it. Now it would be hard to type this message if I hadn't disabled the GPU processing.

Nothing has changed on my coomputer in the last few days.

So. Have I got a weird batch of low-angle range GPU WUs? If I have, why'd they send so many? I'll never be able to process them all - even if it was running at the normal speed.

I did rerun the benchmarks. I got the same values I usually get but the estimated time to complete the CPU WUs is still way off.

Also, before today I would see this message displayed in the STATUS column of the TASKS view on the active GPU WU.

Running (0.01 CPUs + 1.00 NVIDIA GPUs)

Now I get this message

Running (0.07 CPUs + 1.00 NVIDIA GPUs)

No idea why it changed from 0.01 to 0.07 but that might be clue to what's going on.
ID: 1002473 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 1002517 - Posted: 10 Jun 2010, 14:07:29 UTC - in response to Message 1002473.  
Last modified: 10 Jun 2010, 14:08:09 UTC

No idea why it changed from 0.01 to 0.07 but that might be clue to what's going on.

I don't think so, it's a change at the project's side.

Your problem probably is a (batch of) Very Low Angle Range GPU task(s), as you already suspected.

Those are especially havy on the GPU, as you have experienced. And since there's only one Duration Correction Factor for CPU and GPU tasks, the longer time to completion of these VLAR tasks pushes the estimate for the CPU tasks way up.

Your computers are hidden, but I assume that you are running stock applications. I'm not quite sure if the lunatics rescheduler program works for those (it moves VLAR tasks to the CPU), but you could look through the Number crunching forum for hints.

Gruß,
Gundolf
Computer sind nicht alles im Leben. (Kleiner Scherz)

SETI@home classic workunits 3,758
SETI@home classic CPU time 66,520 hours
ID: 1002517 · Report as offensive
artki

Send message
Joined: 15 May 99
Posts: 15
Credit: 446,116
RAC: 0
United States
Message 1002669 - Posted: 10 Jun 2010, 21:23:05 UTC - in response to Message 1002517.  

I don't think so, it's a change at the project's side.

Your problem probably is a (batch of) Very Low Angle Range GPU task(s), as you already suspected.


After 7 hours (and about 1 or 2% completion) the GPU WU terminated with a COMPUTATION ERROR message. The next GPU in the queue started up and is running normally - that is, there is a slight effect on the computers graphics but no big deal.

Your computers are hidden, but I assume that you are running stock applications.

Yeah, I noticed they were hidden but I couldn't find the switch to make it public again. I'm running the stock application.

The estimated time on the CPU WUs in the queue are gradually coming back to "normal". From 56 hours they're down to around 23 hours now.

I guess everything will fix itself over the next few weeks as the GPU WUs I can't finish get reassigned.

Thank You.

ID: 1002669 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 1002682 - Posted: 10 Jun 2010, 22:04:36 UTC - in response to Message 1002669.  

Yeah, I noticed they were hidden but I couldn't find the switch to make it public again.

It's in your SETI@home preferences:
Should SETI@home show your computers on its web site? yes

Gruß,
Gundolf
ID: 1002682 · Report as offensive

Questions and Answers : GPU applications : Large increase of computation duration


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.