Earliest deadline is a farce

Message boards : Number crunching : Earliest deadline is a farce
Message board moderation

To post messages, you must log in.

AuthorMessage
Aurora Borealis
Volunteer tester
Avatar

Send message
Joined: 14 Jan 01
Posts: 3075
Credit: 5,631,463
RAC: 0
Canada
Message 122302 - Posted: 11 Jun 2005, 20:13:00 UTC
Last modified: 11 Jun 2005, 20:26:50 UTC

Last week I decided to do a little experiment. I changed my connect time from 2 to 4 days (I never seemed to have more than a day worth cached at any one time).

BTW this setting will eventually have to be split into a separate cache size dispite the BOSS' stubborness.

I don't have CPND connected at the moment.

Anyway, my Einstein WU immediately went into panic mode for 2 WU and then Early mode for the last one.

Meanwhile both Seti and Predictor where topping up the cash to get 4 days of work.
With all my Einstein WU gone Boinc turned it attention to Predictor whose due date where all sooner than my Seti WU.

Using earliest-deadline-first scheduling because computer is overcommitted.

Seti continues to send me new WU (about 78hr worth at the moment) as my cache of Predictor is being reduced down to 1 WU.
LHC has continuously been trying to get work thoughout this adventure into never, never land.
Seti has not been crunched since early last week although my shares are suppose to be even across the board.
The fact is, I was never in any danger of going past the due date for any WU (even Einstein) if my share choices had been followed.
I expect at this rate it will take a few months for the shares to balance out again. This is a great system in theory, but in reality it sucks bad.


Boinc V7.2.42
Win7 i5 3.33G 4GB, GTX470
ID: 122302 · Report as offensive
Astro
Volunteer tester
Avatar

Send message
Joined: 16 Apr 02
Posts: 8026
Credit: 600,015
RAC: 0
Message 122341 - Posted: 11 Jun 2005, 21:14:22 UTC

you're just now seeing this? This is what 4.35 did to me. Keep an eye on your LTD numbers, the nearly positive ones (in 4.45) should be requesting work. As one project crunches it's LTD should be going negative, while all the others should be going positive at some rate. (I'm not sure how much each will move do to resource share differences, if any). They tell me that PPAH, and Einstein (short deadline projects) will eventually stop downloading more work, and thus allow Seti, LHC (with longer deadlines) to catch up to follow your resource share.

By the way Earliest Deadline First(EDF) and Panic mode are the same thing.

does this help?

tony
ID: 122341 · Report as offensive
Profile RichaG
Volunteer tester
Avatar

Send message
Joined: 20 May 99
Posts: 1690
Credit: 19,287,294
RAC: 36
United States
Message 122542 - Posted: 12 Jun 2005, 7:22:14 UTC

You have hit the problem where if your deadline comes before your next connect time it starts the earliest first panic mode.

By making your connect time more than 50% of Einstein's deadline of 7 days you forced the scheduler into panic mode and to only work on Einstein. It then puts all of the other projects behind. They will build up long term debt (LTD).
Einstein will then no longer download until the LTD is worked off of the other projects.

When Einstein downloads again it will go back into panic mode again in 3 days and it all starts over again.

It is wise to keep the connect time less than 2 days.
Red Bull Air Racing

Gas price by zip at Seti

ID: 122542 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34258
Credit: 79,922,639
RAC: 80
Germany
Message 122545 - Posted: 12 Jun 2005, 7:39:25 UTC

Hi

And then you will go in panic mode with the berkeley servers.

greetz Mike



With each crime and every kindness we birth our future.
ID: 122545 · Report as offensive
Profile PT

Send message
Joined: 19 May 99
Posts: 231
Credit: 902,910
RAC: 0
United Kingdom
Message 122577 - Posted: 12 Jun 2005, 10:35:59 UTC - in response to Message 122545.  
Last modified: 12 Jun 2005, 10:37:15 UTC

Hi
And then you will go in panic mode with the berkeley servers.
greetz Mike


:-D ;-D

That's a good one! 2 points for you...

;-D :-D

Happy crunching
ID: 122577 · Report as offensive
Profile The Gas Giant
Volunteer tester
Avatar

Send message
Joined: 22 Nov 01
Posts: 1904
Credit: 2,646,654
RAC: 0
Australia
Message 122586 - Posted: 12 Jun 2005, 10:47:15 UTC - in response to Message 122302.  

Last week I decided to do a little experiment. I changed my connect time from 2 to 4 days (I never seemed to have more than a day worth cached at any one time).

BTW this setting will eventually have to be split into a separate cache size dispite the BOSS' stubborness.

I don't have CPND connected at the moment.

Anyway, my Einstein WU immediately went into panic mode for 2 WU and then Early mode for the last one.

Meanwhile both Seti and Predictor where topping up the cash to get 4 days of work.
With all my Einstein WU gone Boinc turned it attention to Predictor whose due date where all sooner than my Seti WU.

Using earliest-deadline-first scheduling because computer is overcommitted.

Seti continues to send me new WU (about 78hr worth at the moment) as my cache of Predictor is being reduced down to 1 WU.
LHC has continuously been trying to get work thoughout this adventure into never, never land.
Seti has not been crunched since early last week although my shares are suppose to be even across the board.
The fact is, I was never in any danger of going past the due date for any WU (even Einstein) if my share choices had been followed.
I expect at this rate it will take a few months for the shares to balance out again. This is a great system in theory, but in reality it sucks bad.


Yeah, BOINC going into PANIC mode when the amount of work in it's cache hits more than 50% of the earliest deadline is an absolute joke. For some reason Mr A believes no one should cache more than this...go figure....the schedular used to work so well and now. Doh! The devs have their heads in the sand when it comes to this issue.

Live long and crunch.

Paul
(S@H1 8888)
And proud of it!
ID: 122586 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20283
Credit: 7,508,002
RAC: 20
United Kingdom
Message 122595 - Posted: 12 Jun 2005, 11:30:54 UTC - in response to Message 122586.  
Last modified: 12 Jun 2005, 11:34:23 UTC

Last week I decided to do a little experiment. I changed my connect time from 2 to 4 days (I never seemed to have more than a day worth cached at any one time).

BTW this setting will eventually have to be split into a separate cache size dispite the BOSS' stubborness.

...I expect at this rate it will take a few months for the shares to balance out again. This is a great system in theory, but in reality it sucks bad.

Yeah, BOINC going into PANIC mode when the amount of work in it's cache hits more than 50% of the earliest deadline is an absolute joke. For some reason Mr A believes no one should cache more than this...go figure....the schedular used to work so well and now. Doh! The devs have their heads in the sand when it comes to this issue.

Live long and crunch.

This issue shows up the very different ways in which people believe Boinc should work...

From the Dev's side, a large user cache must look like a bad idea. Turnaround time is greatly increased and also the server databases have a harder time with many more 'live' WUs to keep track of. Further, no user resource need ever be wasted even with a ZERO SIZE CACHE if multiple projects are subscribed to for each user. Hence, a very small cache is preferred.

However, the die-hard s@h cruncher wants as large a cache as possible to weather out any s@h server-side hiccups. The only penalty here is the long wait for others' pending credit to be cleared for those long awaited cached WUs to be crunched.

So how to reconcile the two very different ideas without breaking something?

Perhaps splitting the 'connect time' and 'preferred cache size' would be a good idea to let people tweak how they want to. Limits would have to be included to stop 'antisocial' or 'ignorant' settings spoiling things.

Best would be to have a global 'connect time' and then 'preferred cache size' settings for each project. The BIG PENALTY here is that there would be greater complexity dumped on the servers and JM7 would blow a few neurons trying to balance everything up in the scheduler!


My preferred strategy is simply for a minimum sized cache, join two projects, and maximise the science.

Happy crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 122595 · Report as offensive
Aurora Borealis
Volunteer tester
Avatar

Send message
Joined: 14 Jan 01
Posts: 3075
Credit: 5,631,463
RAC: 0
Canada
Message 122632 - Posted: 12 Jun 2005, 13:28:58 UTC - in response to Message 122595.  
Last modified: 12 Jun 2005, 14:18:25 UTC

Last week I decided to do a little experiment. I changed my connect time from 2 to 4 days (I never seemed to have more than a day worth cached at any one time).

BTW this setting will eventually have to be split into a separate cache size dispite the BOSS' stubborness.

...I expect at this rate it will take a few months for the shares to balance out again. This is a great system in theory, but in reality it sucks bad.

Yeah, BOINC going into PANIC mode when the amount of work in it's cache hits more than 50% of the earliest deadline is an absolute joke. For some reason Mr A believes no one should cache more than this...go figure....the schedular used to work so well and now. Doh! The devs have their heads in the sand when it comes to this issue.

Live long and crunch.

This issue shows up the very different ways in which people believe Boinc should work...

From the Dev's side, a large user cache must look like a bad idea.....
Perhaps splitting the 'connect time' and 'preferred cache size' would be a good idea to let people tweak how they want to. Limits would have to be included to stop 'antisocial' or 'ignorant' settings spoiling things.

Best would be to have a global 'connect time' and then 'preferred cache size' settings for each project. The BIG PENALTY here is that there would be greater complexity dumped on the servers and JM7 would blow a few neurons trying to balance everything up in the scheduler!


The above summary pretty well covers the main point I was trying to make. There is realy no defensible argument to forcing such small caches. This will prove itself imposible as a greater number of projects come on line and the Devs already know this. (The proof is that the CC already contains a constraint on how many project one may subscribe to.)

I do think that each project should have its own cache and keep a history of WU average crunch time. The benchmark should be only the base starting point for the needed calculations as it has proven itself to be wildly inacurate.

As it now stands, any project can force its WU to be proccessed first, by creating an unnecessary short due date (e.g. Einstein).

The answer is not small cache (Modems and laptops are not going to disapear), but customer side controls. Notwithstanding the Devs basic distrust of the abilities and intellegence of the volunteer base, a preference menu will eventualy be required in the GUI.



Boinc V7.2.42
Win7 i5 3.33G 4GB, GTX470
ID: 122632 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 122686 - Posted: 12 Jun 2005, 15:51:53 UTC - in response to Message 122632.  

The above summary pretty well covers the main point I was trying to make. There is realy no defensible argument to forcing such small caches. This will prove itself imposible as a greater number of projects come on line and the Devs already know this. (The proof is that the CC already contains a constraint on how many project one may subscribe to.)

The CC does not and never has had a limit on the number of projects that you could subscribe to. It had for a while a limit on the number that could have work on your host at the same time, but that was removed.

As it now stands, any project can force its WU to be proccessed first, by creating an unnecessary short due date (e.g. Einstein).

The answer is not small cache (Modems and laptops are not going to disapear), but customer side controls. Notwithstanding the Devs basic distrust of the abilities and intellegence of the volunteer base, a preference menu will eventualy be required in the GUI.


Modem users are the sole reason for entering EDF when there is a deadline that is less than 2*the cache size. If the WU enters EDF any later than 2* the duration between connections it is going to be reported late. I expect that dialup modems will not go away this century, so any software that involves communications has to cope with them.


BOINC WIKI
ID: 122686 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 122693 - Posted: 12 Jun 2005, 16:11:05 UTC - in response to Message 122686.  



Modem users are the sole reason for entering EDF when there is a deadline that is less than 2*the cache size. If the WU enters EDF any later than 2* the duration between connections it is going to be reported late. I expect that dialup modems will not go away this century, so any software that involves communications has to cope with them.


JM7

Couldn't you use the client computer upload and download bandwidth stats that are contained within the client_state.xml file on each computer to help you in determining if it is broadband or modem? Just a thought...



Boinc....Boinc....Boinc....Boinc....
ID: 122693 · Report as offensive
Aurora Borealis
Volunteer tester
Avatar

Send message
Joined: 14 Jan 01
Posts: 3075
Credit: 5,631,463
RAC: 0
Canada
Message 122694 - Posted: 12 Jun 2005, 16:11:11 UTC - in response to Message 122686.  

The above summary pretty well covers the main point I was trying to make. There is realy no defensible argument to forcing such small caches. This will prove itself imposible as a greater number of projects come on line and the Devs already know this. (The proof is that the CC already contains a constraint on how many project one may subscribe to.)

The CC does not and never has had a limit on the number of projects that you could subscribe to. It had for a while a limit on the number that could have work on your host at the same time, but that was removed.

As it now stands, any project can force its WU to be proccessed first, by creating an unnecessary short due date (e.g. Einstein).

The answer is not small cache (Modems and laptops are not going to disapear), but customer side controls. Notwithstanding the Devs basic distrust of the abilities and intellegence of the volunteer base, a preference menu will eventualy be required in the GUI.


Modem users are the sole reason for entering EDF when there is a deadline that is less than 2*the cache size. If the WU enters EDF any later than 2* the duration between connections it is going to be reported late. I expect that dialup modems will not go away this century, so any software that involves communications has to cope with them.


And I say EDF should not overide the knowledgeble users wishes.
I note that Options or Preference, at the GUI level, still as not been addressed in any significant way. I reitterate the Devs do not believe that the users can be trusted to make reasoned choice.

BTW I now only have Seti WU on my system. All attempts to get any other projects are still being blocked. THIS IS NOT MY WISH.


Boinc V7.2.42
Win7 i5 3.33G 4GB, GTX470
ID: 122694 · Report as offensive
Heffed
Volunteer tester

Send message
Joined: 19 Mar 02
Posts: 1856
Credit: 40,736
RAC: 0
United States
Message 122739 - Posted: 12 Jun 2005, 17:40:15 UTC - in response to Message 122694.  

And I say EDF should not overide the knowledgeble users wishes.
I note that Options or Preference, at the GUI level, still as not been addressed in any significant way. I reitterate the Devs do not believe that the users can be trusted to make reasoned choice.

That's why I keep bringing up the need for an advanced settings feature, along with a dumb*** filter. Those who know how to set their preferences to keep work from multiple projects without blowing past deadlines are in good shape, and those that can't figure out how to set things up properly find themselves under the watchful eye of the dumb*** filter, and their CC in turn operates under the tight constraints of the current scheduler. Tweakers are happy, and the 'set it and forget it' crowd, or those that are simply mystified by how to configure the CC are happy, which should make dev happy...


BTW I now only have Seti WU on my system. All attempts to get any other projects are still being blocked. THIS IS NOT MY WISH.

It won't be long before tools are available to easily sneak around the schedulers back. :)

It's sad that this has to happen, but it was inevitable when the CC started taking even more control away from the user. (never a good idea, dev...) I think BOINC forgets that it's a guest on the users system, and when it starts making silly demands on the user as to how and when it's going to work, it loses the privilege of running on the system. Luckily, there are users that also see this, and are working to give the us some control back. :D
ID: 122739 · Report as offensive
Aurora Borealis
Volunteer tester
Avatar

Send message
Joined: 14 Jan 01
Posts: 3075
Credit: 5,631,463
RAC: 0
Canada
Message 122775 - Posted: 12 Jun 2005, 18:58:39 UTC - in response to Message 122739.  
Last modified: 12 Jun 2005, 19:00:01 UTC


It won't be long before tools are available to easily sneak around the schedulers back. :)

It's sad that this has to happen, but it was inevitable when the CC started taking even more control away from the user. (never a good idea, dev...) I think BOINC forgets that it's a guest on the users system, and when it starts making silly demands on the user as to how and when it's going to work, it loses the privilege of running on the system. Luckily, there are users that also see this, and are working to give the us some control back. :D


Third party addons are usefull and are sometimes neccessary to compliment a good program (such as in SETI-1). But, when they are needed to circumvent the intent of the software, then one must assume that the original concept is flawed. It is time the engineering crew have a sitdown and review the direction in which they are headed.


Boinc V7.2.42
Win7 i5 3.33G 4GB, GTX470
ID: 122775 · Report as offensive

Message boards : Number crunching : Earliest deadline is a farce


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.