The 100 limit per device is reaching its own limit thanks to new hardware.
I would prefer to see it increased slightly incrementally every 2 months and have the deadline be reduced slightly (also in increments) in order for the impact on the database to be nil. Any thoughts on the feasibility or variants of such a scenario?

I refer to the rule of unintended consequences for such a scenario.

Raising the limit will allow the more powerful machines to keep running through an 8 hour outage such as Outage Tuesdays, but the impact is more outstanding results with a higher possibility of database corruption.

You suggested reducing the deadline to deal with the problem of outstanding results, but the consequence of this is that low-powered devices won't be able to return work on time. Who cares, right? [Not meant as sarcasm, but as what would be a legit response from someone whom hasn't thought it through.]

The consequence of that would that the shorter the deadline, the less chance a lower powered device or a device that isn't on 24/7 will be able to complete the work on time. The Project Scientists have stated they don't want to exclude anyone from participating in the project.

So in a scenario where someone can't afford to run their computer 24/7 and can only donate about an hour per day (maybe due to financial constraints, for example), without GPGPU capable of processing work, and they still use their trusty old single-core machine from ~2003 because they like it/can't afford an upgrade/see no reason to upgrade. If we assume that the average workunit completes on a 1.8GHz Pentium 4 (~2003 class CPU) in about 18 hours, at one hour of crunch time per day would be 18 days before they could return their work. Obviously if workunits get larger (as the Scientists have been talking about), then it will take longer for this fictitious person to return work.

So what? We can easily look at the average turn around time and see that most people have powerful machines and return work much faster and therefore do more work for the project, thus these people should be satisfied, right? Well, it still goes back to the last time we visited this topic and Dr. Korpela said he didn't want people being rejected from the project because their hardware wasn't good enough or because they couldn't afford to be a part of the search.


All that being said, I'm sure the Project Scientists are contemplating and re-evaluating these limits and, given enough data or reasoned arguments, they wouldn't be completely opposed to increasing the limits as needed. I'm just laying out thoughts and responses as you requested, and explaining where we were at last time this topic came up. :-)

Thanks for the long and detailed reply OzzFan.

I have a few Qs/response:

1. Is this not a hard-coded constraints?
- Boinc Client won't download more work than can be processed within 10 days
...even if the S@h 100 limit hasn't been reached.

If I understand how it works, this is based on past processing rate projected into the future.
So even if the host is shutdown for 2 weeks while the volunteer goes on vacation, I don't see a need for anything more than a deadline of 1 month into the future, and considering the current max deadline is almost 8 weeks in the future, this seems to me like it is excessive.
Am I missing something in my analysis?

2. I see at the bottom of the CPU tally page:
~35 Pentium III processing work for S@H
At some point, there is need to identify a cutoff based on something like credit/watt...especially if the host is in a country where electricity used is known to produce high levels of CO2.
Otherwise, it becomes almost unethical (scientifically) for such a situation to go unaddressed.

3. do you have a link to a previous discussion on the subject?
I haven't seen much during the last 3.5 months.

Cheers,
Rob ;-)

1. Is this not a hard-coded constraints?
- Boinc Client won't download more work than can be processed within 10 days
...even if the S@h 100 limit hasn't been reached.

That's correct, it is a hard-coded limit set by the project staff. But 10 days worth of work is still relative to the machine requesting more work, and can be several thousand workunits "in the field" for a single host that a database would have to keep track of.

If I understand how it works, this is based on past processing rate projected into the future.
So even if the host is shutdown for 2 weeks while the volunteer goes on vacation, I don't see a need for anything more than a deadline of 1 month into the future, and considering the current max deadline is almost 8 weeks in the future, this seems to me like it is excessive.
Am I missing something in my analysis?

That sounds about right. But six+ years ago when we last had this discussion, 8 weeks was considered reasonable for the slowest hosts of that period. Though again, if the staff increases the processing length of workunits as they've been discussing, slower hosts will be right back at struggling to return work on time.

2. I see at the bottom of the CPU tally page:
~35 Pentium III processing work for S@H
At some point, there is need to identify a cutoff based on something like credit/watt...especially if the host is in a country where electricity used is known to produce high levels of CO2.
Otherwise, it becomes almost unethical (scientifically) for such a situation to go unaddressed.

The counter-argument to that, in favor of ethical environmental responsibility would be that the Project shouldn't encourage people to build "waste" cycles just for higher RACs and credits. The original mission of the project was only to use "spare" cycles. Now I understand that modern hardware can turn off parts not in use to save energy, thus any crunching could be considered "waste"... the balance would be back to: only using the spare cycles you aren't using, that weren't put to sleep, while using your computer during normal operations. Even leaving the system on 24/7 just to crunch is more wasteful, and produces more CO2, than the person only running a Pentium 4 1.8GHz for one hour per day. (Full disclosure: I leave my main system on 24/7 just to crunch.)

3. do you have a link to a previous discussion on the subject?
I haven't seen much during the last 3.5 months.

My apologies, but my search-foo isn't what it used to be, and given my personal situation with my life, finding old topics just doesn't rate very high on my priority list. Not trying to be a jerk or anything. Just being honest. It was about 5 or 6 years ago though, if that helps.

According to the WUprop project, most projects use a dealine of less than 15 days:
http://wuprop.boinc-af.org/results/delai.py
and I couldn't find one with many datapoints above a deadline of 4 weeks that do not have tasks requiring weeks to crunch on a powerful PC.

I'm just not getting something because it seems to me that S@h's deadline of about
40-60 days is illogical.

correct me if I'm wrong:
Fact #1: Only the slower PCs (processing <=10 tasks/day) can accumulate 10 days worth of work.

The only scenarios I could come up with requiring a bit more than a month are:

1. Someone who
- has a slow PC that allows them to cache 10 days worth of tasks; and
- shuts it down for 3 weeks while on vacation.
The last task downloaded before going on vacation would get returned about 35 days later (3 weeks + a long weekend + 10 days of cache to burn + 1 day of grace since there's also an extra night in there).

2. Someone who's Boinc productivity is very inconsistent on a below-midrange PC and who runs Boinc only on weekends.
On a long weekend, the PC is left on 24hrs/day for 3 days in a row building a cache of 100 tasks that also reaches the 10 day limit (since it processes about 10 tasks/day).
Over the next 3 weekends, the computer is on sporadically and only processes 10 tasks/weekend leaving a cache of 70 tasks almost 4 weeks later.

These scenarios are extraordinary and non-recurrent, and therefore should be considered as statistical outliers.

Am I missing scenarios that could affect hundreds or even a thousand PCs year round requiring a deadline > 30 days?

correct me if I'm wrong:
Fact #1: Only the slower PCs (processing <=10 tasks/day) can accumulate 10 days worth of work.

I'm not quite sure why you believe this. Given no limits from the servers, why wouldn't a faster machine be able to accumulate 10 days worth of work? Granted, it would make the local BOINC client's built-in scheduler freak out and think it will miss deadlines and constantly go into Earliest Deadline First mode, but I see no reason why a fast machine couldn't cache 10 days worth of work.

Then again, they've changed the way the cache setting works and I may not completely understand how it actually works anymore.

Am I missing scenarios that could affect hundreds or even a thousand PCs year round requiring a deadline > 30 days?

Nope. Not that I can think of. I can only say that the decision wasn't made because a large quantity of machines would be affected. The decision was made because they want everyone to participate. It isn't about the masses crunching tons of data. It's about the individual who wants to help out in the search with whatever means they have.

Hey OzzFan,

correct me if I'm wrong:
Fact #1: Only the slower PCs (processing <=10 tasks/day) can accumulate 10 days worth of work.

I'm not quite sure why you believe this. Given no limits from the servers, why wouldn't a faster machine be able to accumulate 10 days worth of work?

Maybe I should have been more specific:
...slower PCs (processing <=10 tasks/day per device)...
&/or ...slower PCs w/o GPU (processing <=10 tasks/day)...
and with the current limit of 100 tasks/device.

As soon as a device is processing more than 10 tasks/day, the [edit]up to 100[/e] tasks in the cache will get processed faster than 10 days.

As for the philosophy that every PC is welcome, I'm fine with that.
I just don't understand the math behind a deadline past 1 month...especially if we add ghosts to the big picture (and Lionel's ~45,000 ghosts).

Thanks,
Rob :-)

Then again, they've changed the way the cache setting works and I may not completely understand how it actually works anymore.

From my research cabinet, and all very recent:

Message 1784211
Message 1785937
Message 1791681
Message 1805800

As far as the topic of WU deadlines, I believe a decrease to 30 days is not unreasonable.

You suggested reducing the deadline to deal with the problem of outstanding results, but the consequence of this is that low-powered devices won't be able to return work on time.]

The consequence of that would that the shorter the deadline, the less chance a lower powered device or a device that isn't on 24/7 will be able to complete the work on time. The Project Scientists have stated they don't want to exclude anyone from participating in the project.

I respectfully disagree.  For example, take Rob's hypothetical rig (slower PC w/o GPU (processing <=10 tasks/day)).  Even if some WUs did timeout, its cache would not run out, simply by virtue of its slowness.  Additionally, these WUs wouldn't remain in limbo for so long, and wouldn't it lighten some load on the servers from having to keep track of them for such unnecessarily extended periods of time?

As a proof of concept, Rosetta@home can be run on seriously slow machines.  Its WU deadline is 14 days, and there are no complaints on their forums from participants feeling excluded.

I wonder if it would be possible to write a script to identify "slow" rigs (ex. single-core CPUs w/o GPU) and extract their respective "Average turnaround time", and perhaps find the median or avg.

Well then gentlemen, all I can say is to take your arguments to the Project staff. I'm not totally convinced a change is necessary (even if not unreasonable), but I'm not the decision maker here. Any further arguments from me would be to presume to speak for the staff, which I don't want to do.

Thanks AMDave & OzzFan!

My motivation was to think of ways to increase the device limit by a few % without adding any strain on the database. It seems that reducing the deadline might be a good option.

I don't think anything should be changed on the servers before the SETI WoW event 2016 since:
- the event is promoted by the project staff; and
- it is likely to put additional strain on the database from having many more power rigs.

Here' my current perspective of the foreseeable future:
1. With VR +GPU rigs expected to be the >$1000 sensation for xmas 2016, there are a few months left for a proposal to be formulated, circulated for comments, and then submitted to the project staff.

2. until Petri's app-in-dev is available in Lunatics v0.4x, the demand for more than 100 tasks/device is likely to remain small until at least early fall (as can be implied in those thread links provided by AMDave)

3. it is only the few multi-CPU rigs without a GPU that can't currently take advantage of a rescheduler script to increase their stash temporarily to a "justifiable level" non-hoarder level above 100.

If anyone has concerns/Qs/constructive comments, please post.
I will likely start a new thread after WoW is over.

Thanks again to all those who have participated constructively in this thread.
Cheers,
Rob :-D

---

I wonder if it would be possible to write a script to identify "slow" rigs (ex. single-core CPUs w/o GPU) and extract their respective "Average turnaround time", and perhaps find the median or avg.

I have to think that some of this type of advance work might well be of use in convincing staff to look at the issue.
Also, reducing database size would be a good argument, given recent challenges in that area.

---

I wonder if it would be possible to write a script to identify "slow" rigs (ex. single-core CPUs w/o GPU) and extract their respective "Average turnaround time", and perhaps find the median or avg.

I have to think that some of this type of advance work might well be of use in convincing staff to look at the issue.
Also, reducing database size would be a good argument, given recent challenges in that area.

Agreed.

Decreasing the WU deadline from @54 to 30 days could hardly be characterized as drastic.  Unless your swimming in cash, don't you try to get your money's worth out of your purchases?  In business parlance, it's called ROI.  Doesn't the auto industry continuously seek to improve efficiency (mpg) and power (hp/torque)?  How about efficiency on PSUs (and here).

Raistmer, Jason, Petri, et al, do they not attempt to wring out efficiency in their endeavors? Has it not been repeatedly stated that this project's budget is, shall we say, less than ample?  In lieu of cash, would it not be prudent to seek and implement methods which would allow the project to get their money's worth from existing hardware/processes?

To satisfy curiosity, I would reiterate my query about the possibility of getting hard #s by disseminating the data via a script.

Aren't these examples of what is transpiring:

Computer Simulation Lets You Create Your Own Traffic Jam
The Phantom Traffic Jam - an explanation

Here is what it could lead to (in the extreme):

Example 1
Example 2

Forgive the appearance of proselytizing.  Maybe I should search Craigslist to exchange my soapbox for a newer GPU.