'to completion' times grossly overestimated.

Message boards : Number crunching : 'to completion' times grossly overestimated.
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Kaylie

Send message
Joined: 26 Jul 08
Posts: 39
Credit: 333,106
RAC: 0
United States
Message 954789 - Posted: 14 Dec 2009, 19:56:36 UTC

estimated completion times for WU's are being estimated at 35 hours when the tasks actually finish in seven. This seems to limit the amount of work BOINC will accept.

Any ideas how to fix this?
ID: 954789 · Report as offensive
Fred W
Volunteer tester

Send message
Joined: 13 Jun 99
Posts: 2524
Credit: 11,954,210
RAC: 0
United Kingdom
Message 954793 - Posted: 14 Dec 2009, 20:00:12 UTC - in response to Message 954789.  

estimated completion times for WU's are being estimated at 35 hours when the tasks actually finish in seven. This seems to limit the amount of work BOINC will accept.

Any ideas how to fix this?

Jut wait. Boinc learns from experience so the estimates will improve quite quickly.

F.
ID: 954793 · Report as offensive
Luke
Volunteer developer
Avatar

Send message
Joined: 31 Dec 06
Posts: 2546
Credit: 817,560
RAC: 0
New Zealand
Message 954794 - Posted: 14 Dec 2009, 20:08:17 UTC - in response to Message 954789.  
Last modified: 14 Dec 2009, 20:09:46 UTC

estimated completion times for WU's are being estimated at 35 hours when the tasks actually finish in seven. This seems to limit the amount of work BOINC will accept.

Any ideas how to fix this?


It'll fix itself after time, you might just have to give it a few days.

It's called the Task Duration Correction Factor, basically it's a calculation of the difference between how long BOINC thinks it will take to complte a task and how long it actually does take. To check exactly what your Task Duration Correction Factor is, go to Your Account > Your Computers > Click "Details" on one of them > And scroll down to the bottom and above "Location" and "Merge This Computer" is your TDCF.

I think the optimum is 1.00?
My two computers are showing 0.55 and 0.11 respectively.

- Luke.
- Luke.
ID: 954794 · Report as offensive
Kaylie

Send message
Joined: 26 Jul 08
Posts: 39
Credit: 333,106
RAC: 0
United States
Message 954796 - Posted: 14 Dec 2009, 20:09:05 UTC - in response to Message 954793.  

for quite some time the estimates have been fairly acurate. now they drop slightly when tasks finish but go right back up again a short time later.
ID: 954796 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 954815 - Posted: 14 Dec 2009, 21:47:05 UTC - in response to Message 954796.  

for quite some time the estimates have been fairly acurate. now they drop slightly when tasks finish but go right back up again a short time later.

Before you decide this is a huge problem, you need to look at how BOINC uses the time estimates, and how inaccurate estimates could cause problems.

Let's say you want a 7 day cache, and each work unit takes about ten hours (just for easy math).

168 hours in a week, BOINC rounds up, so 17 work units in your queue.

If BOINC over estimates, it will carry less work. If BOINC estimates 100 hours per work unit, you'll have two in your queue, and while the queue isn't very full, the risk of missing deadlines is small, because that 200 hours of queued work is really 20.

On the other hand, let's say that BOINC under estimated, and thought the work would take 1 hour when it takes ten. Now your queue is 168 work units, and instead of being one week, it is ten weeks.

Now you're in trouble.

So the estimates are intentionally high.

As others have been pointing out, there is a correction factor that BOINC uses to bring estimates into line. It is intentionally intended to adjust downward very slowly, and adjust upward quickly, because under-estimates are really bad, while over-estimates are mostly cosmetic.
ID: 954815 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 954821 - Posted: 14 Dec 2009, 22:07:09 UTC - in response to Message 954796.  

Since your computers are hidden it's kinda hard to tell but if you are running a CUDA card that could be messing with your DCF. We have been getting a lot of shorties lately and if they have been running on your GPU they will drag the DCF way down. Then when you turn in a couple of midrange WUs with your CPU DCF will go crazy trying to figure out what happened.


PROUD MEMBER OF Team Starfire World BOINC
ID: 954821 · Report as offensive
RB

Send message
Joined: 7 Mar 00
Posts: 103
Credit: 1,084,436
RAC: 0
Canada
Message 954847 - Posted: 14 Dec 2009, 23:38:32 UTC - in response to Message 954794.  

I think the optimum is 1.00?
My two computers are showing 0.55 and 0.11 respectively.
- Luke.



My TDCF is 0.168309

The WU's complete in almost the exact time that is shown in the estimate. (within a minute or so, if not exact)

Shouldn't my number be closer to 1.00 then, for being so close?

Just wondering...
ID: 954847 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 954852 - Posted: 14 Dec 2009, 23:49:32 UTC - in response to Message 954847.  
Last modified: 14 Dec 2009, 23:51:22 UTC

I think the optimum is 1.00?
My two computers are showing 0.55 and 0.11 respectively.
- Luke.



My TDCF is 0.168309

The WU's complete in almost the exact time that is shown in the estimate. (within a minute or so, if not exact)

Shouldn't my number be closer to 1.00 then, for being so close?

Just wondering...


1.00 is the default value. The more work your machine can do the lower boinc will make this value.

0.15488 is the value for my 3.0GHz E8400 C2D
0.684242 is the value for my triple Pentium II 400MHz server
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 954852 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 954860 - Posted: 15 Dec 2009, 0:12:20 UTC - in response to Message 954794.  

estimated completion times for WU's are being estimated at 35 hours when the tasks actually finish in seven. This seems to limit the amount of work BOINC will accept.

Any ideas how to fix this?


It'll fix itself after time, you might just have to give it a few days.

It's called the Task Duration Correction Factor, basically it's a calculation of the difference between how long BOINC thinks it will take to complte a task and how long it actually does take. To check exactly what your Task Duration Correction Factor is, go to Your Account > Your Computers > Click "Details" on one of them > And scroll down to the bottom and above "Location" and "Merge This Computer" is your TDCF.

I think the optimum is 1.00?
My two computers are showing 0.55 and 0.11 respectively.

- Luke.

On my i7 it is Task duration correction factor 0.08962 and it has lots of units.
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 954860 · Report as offensive
Kaylie

Send message
Joined: 26 Jul 08
Posts: 39
Credit: 333,106
RAC: 0
United States
Message 954871 - Posted: 15 Dec 2009, 0:39:30 UTC - in response to Message 954860.  
Last modified: 15 Dec 2009, 0:43:53 UTC

Still over 30 hours for 7 hour tasks on a quad core with two GPU's (there have been some short CUDA's) and

the DCF for that computer is an outragous 2.9 I understand the need to insure meeting the deadlines.
ID: 954871 · Report as offensive
Luke
Volunteer developer
Avatar

Send message
Joined: 31 Dec 06
Posts: 2546
Credit: 817,560
RAC: 0
New Zealand
Message 954872 - Posted: 15 Dec 2009, 0:41:24 UTC - in response to Message 954871.  

Still over 30 hours for 7 hour tasks on a quad core with two GPU's (there have been some short CUDA's) and

the DCF for that computer is an outragous 2.9


Yep. Give it time to get lower. You might need a week or more for it to fully stabilize, as it is slow to come down and fast to rise up.

- Luke.
- Luke.
ID: 954872 · Report as offensive
Ianab
Volunteer tester

Send message
Joined: 11 Jun 08
Posts: 732
Credit: 20,635,586
RAC: 5
New Zealand
Message 954877 - Posted: 15 Dec 2009, 1:32:56 UTC

Another thing I have noticed is that if a PC is put into "Suspend" mode it seems to mess with the correction factor. Not sure of the details, but one of my P4s got suspended for a couple of days and the DCF went from it's normal 0.25 to 0.8 So although it's set to 3 days work, it was only requesting about 1 day.

As a work around, increase the work unit cache temporarily, that will fool it into downloading some more work units and it will correct itself as work units are returned with the correct time calculations.

Ian
ID: 954877 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 954918 - Posted: 15 Dec 2009, 5:35:15 UTC - in response to Message 954871.  

Still over 30 hours for 7 hour tasks on a quad core with two GPU's (there have been some short CUDA's) and

the DCF for that computer is an outragous 2.9 I understand the need to insure meeting the deadlines.

Not sure about the outrageous part.

You can go and find the <duration_correction_factor> tag in client_state.xml (stop BOINC before editing) and change it if you wish (0.5 is probably a safe number), or as others have suggested, you can wait.

Each time you finish a WU, it'll go down a bit.
ID: 954918 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 955143 - Posted: 16 Dec 2009, 5:12:34 UTC - in response to Message 954918.  

Still over 30 hours for 7 hour tasks on a quad core with two GPU's (there have been some short CUDA's) and

the DCF for that computer is an outragous 2.9 I understand the need to insure meeting the deadlines.

Not sure about the outrageous part.

You can go and find the <duration_correction_factor> tag in client_state.xml (stop BOINC before editing) and change it if you wish (0.5 is probably a safe number), or as others have suggested, you can wait.

Each time you finish a WU, it'll go down a bit.

Outrageous was the gross underestimate of time that some projects have had. I had a DCF on one projct of 1E30 (Not a typo that is a 1 with 30 zeros after it). Problems like this got the DCF capped at 100, and any DCF over 90 would automatically put ALL tasks for that project into Earliest Deadline First (at this point, since BOINC has capped the DCF, it has NO idea what is happening with time for that project).


BOINC WIKI
ID: 955143 · Report as offensive
Kaylie

Send message
Joined: 26 Jul 08
Posts: 39
Credit: 333,106
RAC: 0
United States
Message 960601 - Posted: 3 Jan 2010, 16:10:15 UTC - in response to Message 954789.  

It's happening again, just when I want to load up with tasks for the outage. I editied the XML file and set TDCF to .5 and the task estimates droped to 8 hrs, when they take 5. That is entirely acceptable but as soon as one 5 hour task finished the estimates on all tasks junped to 27 hours. This sure is annoying...

Running BOINC 6.10.17 on a quad core with 2- GPU's.
ID: 960601 · Report as offensive
Profile Pappa
Volunteer tester
Avatar

Send message
Joined: 9 Jan 00
Posts: 2562
Credit: 12,301,681
RAC: 0
United States
Message 960610 - Posted: 3 Jan 2010, 17:08:12 UTC - in response to Message 960601.  

It's happening again, just when I want to load up with tasks for the outage. I editied the XML file and set TDCF to .5 and the task estimates droped to 8 hrs, when they take 5. That is entirely acceptable but as soon as one 5 hour task finished the estimates on all tasks junped to 27 hours. This sure is annoying...

Running BOINC 6.10.17 on a quad core with 2- GPU's.


It sounds like you got a couple of VLARs which drive DCF UP almost instantly. While other more normal WU's return it more slowly...

You have two choices:

Wait it out.
or
The only real way to correct that before the outage is do a Project Detach and then Attach... It will clean the information in that is in the client_state.xml file.

Regards

Please consider a Donation to the Seti Project.

ID: 960610 · Report as offensive
Profile S@NL - eFMer - efmer.com/boinc
Volunteer tester
Avatar

Send message
Joined: 7 Jun 99
Posts: 512
Credit: 148,746,305
RAC: 0
United States
Message 960632 - Posted: 3 Jan 2010, 18:07:16 UTC - in response to Message 960601.  
Last modified: 3 Jan 2010, 18:09:06 UTC

It's happening again, just when I want to load up with tasks for the outage. I editied the XML file and set TDCF to .5 and the task estimates droped to 8 hrs, when they take 5. That is entirely acceptable but as soon as one 5 hour task finished the estimates on all tasks junped to 27 hours. This sure is annoying...

Running BOINC 6.10.17 on a quad core with 2- GPU's.

BOINC can't handle CPU and GPU right, at least not the latest stable release.
Setting the FLOPS statement solved this problem on my computers.
Is goes wrong when the GPU is a lot faster than the CPU.
Make the FLOPS value larger when the GPU tasks take shorter than predicted.

(GTX 295) one half.

<app_version>
<app_name>setiathome_enhanced</app_name>
<version_num>608</version_num>
<flops>22000000000</flops>
<plan_class>cuda</plan_class>
<avg_ncpus>0.040000</avg_ncpus>
<max_ncpus>0.040000</max_ncpus>
- <coproc>
<type>CUDA</type>
<count>1</count>
</coproc>
- <file_ref>
<file_name>MB_6.08_CUDA_V12_VLARKill_FPLim2048.exe</file_name>
<main_program />
</file_ref>
- <file_ref>
<file_name>cudart.dll</file_name>
</file_ref>
- <file_ref>
<file_name>cufft.dll</file_name>
</file_ref>
- <file_ref>
<file_name>libfftw3f-3-1-1a_upx.dll</file_name>
</file_ref>
</app_version>
TThrottle Control your temperatures. BoincTasks The best way to view BOINC. Anza Borrego Desert hiking.
ID: 960632 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 960642 - Posted: 3 Jan 2010, 18:39:06 UTC - in response to Message 960632.  

It's happening again, just when I want to load up with tasks for the outage. I editied the XML file and set TDCF to .5 and the task estimates droped to 8 hrs, when they take 5. That is entirely acceptable but as soon as one 5 hour task finished the estimates on all tasks junped to 27 hours. This sure is annoying...

Running BOINC 6.10.17 on a quad core with 2- GPU's.

BOINC can't handle CPU and GPU right, at least not the latest stable release.
Setting the FLOPS statement solved this problem on my computers.
Is goes wrong when the GPU is a lot faster than the CPU.
Make the FLOPS value larger when the GPU tasks take shorter than predicted

For CPU work, BOINC keeps a <duration_correction_factor> which adjusts for the difference between the benchmark prediction, and reality.

BOINC needs a DCF for the GPU as well, but the real problem is that there is no suitable GPU benchmark.

... and as Fred points out, GPUs vary wildly.

In the grand scheme of things, this is mostly cosmetic. As long as BOINC doesn't grossly overestimate performance, work will be completed on-time.
ID: 960642 · Report as offensive
Profile S@NL - eFMer - efmer.com/boinc
Volunteer tester
Avatar

Send message
Joined: 7 Jun 99
Posts: 512
Credit: 148,746,305
RAC: 0
United States
Message 960654 - Posted: 3 Jan 2010, 19:01:09 UTC - in response to Message 960642.  

It's happening again, just when I want to load up with tasks for the outage. I editied the XML file and set TDCF to .5 and the task estimates droped to 8 hrs, when they take 5. That is entirely acceptable but as soon as one 5 hour task finished the estimates on all tasks junped to 27 hours. This sure is annoying...

Running BOINC 6.10.17 on a quad core with 2- GPU's.

BOINC can't handle CPU and GPU right, at least not the latest stable release.
Setting the FLOPS statement solved this problem on my computers.
Is goes wrong when the GPU is a lot faster than the CPU.
Make the FLOPS value larger when the GPU tasks take shorter than predicted

For CPU work, BOINC keeps a <duration_correction_factor> which adjusts for the difference between the benchmark prediction, and reality.

BOINC needs a DCF for the GPU as well, but the real problem is that there is no suitable GPU benchmark.

... and as Fred points out, GPUs vary wildly.

In the grand scheme of things, this is mostly cosmetic. As long as BOINC doesn't grossly overestimate performance, work will be completed on-time.

I know the Lunatics guys are working on this.
TThrottle Control your temperatures. BoincTasks The best way to view BOINC. Anza Borrego Desert hiking.
ID: 960654 · Report as offensive
Kaylie

Send message
Joined: 26 Jul 08
Posts: 39
Credit: 333,106
RAC: 0
United States
Message 960744 - Posted: 5 Jan 2010, 1:40:48 UTC

Think I see now what is happening. My GPU's run slower than estimated and, scince there is only one TDCF for both CPU and GPU, the CPU task length estimates suffer.

This make sense?
ID: 960744 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : 'to completion' times grossly overestimated.


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.