Further to the discussion on detection of AP 30/30 tasks, I have just had such a task, it had 16.5% blanking which on my system would have normally taken ~13 hrs to complete. It bailed out under the 30/30 rule at 12h23m.
In a case like this it would have made only a small change to the correct APR.
It is the tasks that bail out before a minute that cause the problem.

While following this thread a thought has been brewing in 'the back of my mind' regarding the reporting of download requests...

It would be very nice if the 'NO Work Available for...' message were not, in effect, a LIE!.

Something like Scheduler Unable To Send New Work At This Time, would be truthful, and more polite, especially when there are tens of thousands of work-units shown as available.

With that said... I'll try to wait patiently for BOINC to regain its sanity...

---

Further to the discussion on detection of AP 30/30 tasks, I have just had such a task, it had 16.5% blanking which on my system would have normally taken ~13 hrs to complete. It bailed out under the 30/30 rule at 12h23m.
In a case like this it would have made only a small change to the correct APR.
It is the tasks that bail out before a minute that cause the problem.

I'm nervous about using an absolute runtime, rather than one relative to the speed of the host, or the size of the task, to determine 'outlier'. That's exactly the mistake David made in introducing the faulty -177 'cure'.

My GTX 470 is already running tasks in around 90 seconds - that's good, full-length, validating, VHAR tasks. Not many more interations of Moore's Law, or improvements in the optimisation of the application, before your 1-minute test starts catching live ones. Whereas a relative test should be good for a while yet.

If that is the case then it will have to be a percentage, presumably on flops count against est_flops (or whatever its called)

Talking of which do you reckon there would be much gained from improving the estimated flops. It would probably improve the APR calculation.

Unfortunately that thought might have to be on hold for a long time as it is a Seti responsibility. But maybe not David is head of Seti he can do so that he can check the APR figures are accurate. LOL.

I think it might have been helpful it a partial correction had been made here, not just on Beta....
I know that once the code was botched, backing off wholesale might have precipitated a real mess (like this isn't already).
I have rigs in all manner of Boinc hell....not asking for work, asking and getting none, getting 1 at a time, getting only GPU whilst the CPU idles, you name it.

But, most have not gone totally dry yet.

And I will admit, at least it has not been a total disaster for the Seti project itself...much work IS going out and being returned at one of the highest rates ever for an extended period of time.

But it's gonna be a tough hole to dig out of, given the bandwidth limitation.
All that pent up AP work has gotta go out sooner or later....and once it starts, things are gonna bottleneck real fast.

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Perhaps the new plan is to alternate projects. MB one week AP the next.

---

Boinc....Boinc....Boinc....Boinc....

Perhaps the new plan is to alternate projects. MB one week AP the next.

LOL...
Unfortunately, I don't think anything is going according to plan right now.

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Plan? We have a plan? We don't need no stinkin" plan! Full speed ahead, we'll find the pesky aliens yet!! :-)

---

PROUD MEMBER OF Team Starfire World BOINC

My GTX 470 is already running tasks in around 90 seconds - that's good, full-length, validating, VHAR tasks. Not many more interations of Moore's Law, or improvements in the optimisation of the application, before your 1-minute test starts catching live ones. Whereas a relative test should be good for a while yet.

Even my GTX 560Ti when running 1 WU at a time does shorties in just under 2min.
I can see the next generation of video cards & drivers knocking them out in well under 1min.

---

Grant
Darwin NT

My GTX 470 is already running tasks in around 90 seconds - that's good, full-length, validating, VHAR tasks. Not many more interations of Moore's Law, or improvements in the optimisation of the application, before your 1-minute test starts catching live ones. Whereas a relative test should be good for a while yet.

Even my GTX 560Ti when running 1 WU at a time does shorties in just under 2min.
I can see the next generation of video cards & drivers knocking them out in well under 1min.

Seti will soon have to increase the size of a WU, or increase the depth of the analysis, just to keep the processing time longer than the download time.

---

it's been suggested before (and been done before) ... Do more science per WU and a lot of bottlenecks go away ... (but the suggestion was ignored)

Ed F

V7 is slated to be released soon. I haven't any data on the speeds of the Lunatics apps on V7 WU's, but the stock app runs about 5 hours per CPU WU on my i7-950 @ 4Ghz.

Perhaps that will relieve some strain soon.

---

V7 is slated to be released soon. I haven't any data on the speeds of the Lunatics apps on V7 WU's, but the stock app runs about 5 hours per CPU WU on my i7-950 @ 4Ghz.

Perhaps that will relieve some strain soon.

Off topic, but probably related, as we see the next fiasko on the horizon.

'Soon' is a relative term. At least another month, I'd say a good bit more.
Cuda apps need to be tested at beta first, and even v6.97 is still under scrutiny.

We are in extended alpha/beta, when it comes to optimised apps. We aren't looking at speeds just yet.

The extra analysis doesn't take that much time. So from that point of view not much less strain.

The rollout? Chaos and mayhem.

On topic:

Getting back to 'normal' requires 3 steps.
Step one, BOINC server code for outlier removal has been done but isn't much use without
Step two, SETI code to mark tasks as outliers.
Step three, remove the capping code again.

I have a horrible feeling, that 3 isn't going to be staggered as advised...

At that point, hosts that have adapted with incredibly low DCFs will suddenly be seeing APR estimated tasks again - estimated under the assumption that host DCF is near 1. So, coming in underestimated 10,20,50x... Workfetch frenzy ensues - bloating actual cache to what? extactly - 10,20,50x... - Actually taking into consideration the usual problems with workfetch it's probably not going to get that high. Then again, maybe yes. Take a host set with a 10 day cache. It maybe has tasks for 5. It will take 5 days for the first heavily underestimated tasks to work their way to the head of the queue. 5 days of neigh unlimted workfetch... THEN the first tasks gets processed, requiering 10,20,50x times the estimate. But no, after an elapsed time of more than 10x the estimate the task errors with a -177.

Anybody knows if DCF would go up?
IIRC tasks that end in errors don't impact on DCF.
On a host processing on both CPU and GPU, CPU is probably going to pull DCF back to 1, at which time the GPU starts breathing again.
On a GPU only host there is nothing whatsoever that can reset DCF apart from the user. An unsupervised GPU only host will be dead in the water.
An unsupervised mixed host will eventually recover - after trashing countless tasks.

I'm with Sartre on this one - 'L'infer c'est les autres'.

Based on the assumption that we aren't going to get the gradual release of the cap, I would suggest:

Depending on cache left, set NNT and/or reduce cache settings to an absolute minimum.
As soon as underestimated tasks appear (but preferably before) set NNT.
Wait for cache to empty of 'old' tasks.
Edit DCF to 1 - alternatively detach/reattach (you'll lose the few underestimated tasks that crept in, but they would just error anyway - see above) - second alternative, use the anti -177 option of Fred's rescheduler to be able to process the underestimated tasks and wait for DCF to return to 1 with those.
Resume workfetch. If it everything looks alright, you can increase the cache settings again.

Apologies, I seem to have run out of black paint.

Thanks for the input, LadyL.

Even though the kitties are not fond of black.

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Isn't the detach/reattach option a no goer, effectively what caused all this in the first place.

Because the quick to APR, MB tasks, and slow to APR, AP tasks will still apply.

For hosts which have both CPU and GPU applications installed (or which could easily have them installed by re-running the installer), one option would be to set "Use NVIDIA GPU" to 'no', and "Use CPU" to 'yes', in the website preferences until things are under control.

That allows GPU processing of any cached tasks to continue, but will inhibit the fetching of any new ones.

By allowing CPU tasks to be downloaded and crunched, DCF would be kept 'reasonable'.

You might need to keep toggling "Use NVIDIA GPU" for a little while, because of the known tendency of the server to preferentially tag for GPU on mixed requests. You would want to avoid getting too may GPU tasks at once, without CPU tasks to leaven the mix, until you see that new tasks for both resources are being issued with sane estimates.

Isn't the detach/reattach option a no goer, effectively what caused all this in the first place.

Because the quick to APR, MB tasks, and slow to APR, AP tasks will still apply.

Well a detach/reattach is always the last straw and the worst option.

However to reset DCF on a MB only GPU only host, where DCF is not going to be pulled back to 1 by CPU tasks, you have no choice but to either manually edit DCF in client_state.xml to 1 or to get it reset with a reattach - I'm not positive simply resetting the project resets DCF as well.

[as Richard says, you could of course enable CPU processing to steer DCF]

As I have mentioned before, this is a worst case szenario - I've got a small hope remaining that the implications of just removing the cap again will reach the attention of the decision making minds before they unleash hell by doing so.

Isn't the detach/reattach option a no goer, effectively what caused all this in the first place.

Because the quick to APR, MB tasks, and slow to APR, AP tasks will still apply.

Well a detach/reattach is always the last straw and the worst option.

However to reset DCF on a MB only GPU only host, where DCF is not going to be pulled back to 1 by CPU tasks, you have no choice but to either manually edit DCF in client_state.xml to 1 or to get it reset with a reattach - I'm not positive simply resetting the project resets DCF as well.

[as Richard says, you could of course enable CPU processing to steer DCF]

As I have mentioned before, this is a worst case szenario - I've got a small hope remaining that the implications of just removing the cap again will reach the attention of the decision making minds before they unleash hell by doing so.

And I wonder what this is going to do to all of the 'set and forget' hosts...
Folks that attached to the project on a whim and have long since forgotten it is running in the background. There are probably many thousands of them.

Might the project have to issue some kind of DCF reset command to coax them back into action? Is that even possible?

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Detaching/reattaching to the project physically deletes the setiathome project folder - so all application files and configuration setting are lost. But you usually get the same HostID, and host APR etc., back again. That's the worst possible outcome, even if you remembered to back up your applications so you can put them back again.

Resetting the project does set DCF to the default 1.0000 (and, I think, in general preserves optimised applications) - but I think the behaviour varies between different versions of BOINC - best to let someone else test it first.

The other thing that resetting the project certainly does it to throw away, without reporting, all current tasks - including those completed but not yet uploaded.

The best sequence, if you're adventurous enough to try, is:

NNT - crunch all cached tasks - upload - report - backup folder contents - reset - check folder contents are still intact - allow new work.

Detaching/reattaching to the project physically deletes the setiathome project folder - so all application files and configuration setting are lost. But you usually get the same HostID, and host APR etc., back again. That's the worst possible outcome, even if you remembered to back up your applications so you can put them back again.

Resetting the project does set DCF to the default 1.0000 (and, I think, in general preserves optimised applications) - but I think the behaviour varies between different versions of BOINC - best to let someone else test it first.

The other thing that resetting the project certainly does it to throw away, without reporting, all current tasks - including those completed but not yet uploaded.

The best sequence, if you're adventurous enough to try, is:

NNT - crunch all cached tasks - upload - report - backup folder contents - reset - check folder contents are still intact - allow new work.

It's been such a long time.......
I don't recall if a reset will chuck out the baby with the bathwater...IE, the optimized apps and app_info file. A detach certainly does.

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster