client priorities

Message boards : Number crunching : client priorities
Message board moderation

To post messages, you must log in.

AuthorMessage
KWSN-GMC-Peeper of the Castle Anthrax
Avatar

Send message
Joined: 28 May 99
Posts: 274
Credit: 6,936,182
RAC: 0
United States
Message 1058004 - Posted: 20 Dec 2010, 4:07:23 UTC

I have been wondering for years now, just got the energy to see if there's answer.
Why does the BOINC client finish WU from whatever project that are due in a month and a half before it starts on ones it had that are due in less than a week?
If you don't touch it, you can't break it.
;
ID: 1058004 · Report as offensive
Profile Gundolf Jahn

Send message
Joined: 19 Sep 00
Posts: 3184
Credit: 446,358
RAC: 0
Germany
Message 1058043 - Posted: 20 Dec 2010, 9:58:51 UTC - in response to Message 1058004.  

Because BOINC processes tasks in First In First Out mode, unless that would cause a task to miss deadline. Then BOINC switches to Earliest Deadline First mode (or is supposed to do so, but there's no bug-free software anywhere:-).

Gruß,
Gundolf
ID: 1058043 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1058126 - Posted: 20 Dec 2010, 16:56:40 UTC - in response to Message 1058004.  

I have been wondering for years now, just got the energy to see if there's answer.
Why does the BOINC client finish WU from whatever project that are due in a month and a half before it starts on ones it had that are due in less than a week?


Actually, based on my "study" of BOINC and its quirks and oddnesses, I am afraid that the correct answer to your question is:

There are some things that Men were not meant to know.
ID: 1058126 · Report as offensive
Fred W
Volunteer tester

Send message
Joined: 13 Jun 99
Posts: 2524
Credit: 11,954,210
RAC: 0
United Kingdom
Message 1058182 - Posted: 20 Dec 2010, 20:39:35 UTC - in response to Message 1058126.  

There are some things that Men were not meant to know.

That's what my wife keeps telling me...
ID: 1058182 · Report as offensive
KWSN-GMC-Peeper of the Castle Anthrax
Avatar

Send message
Joined: 28 May 99
Posts: 274
Credit: 6,936,182
RAC: 0
United States
Message 1058237 - Posted: 21 Dec 2010, 0:53:38 UTC

hehe..well...thanks for the input. Silly me, I had ASS U MEd that the default would just naturally have to be by due date. You know..efficiency and logic led me astray again...sigh..

If you don't touch it, you can't break it.
;
ID: 1058237 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51469
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1058239 - Posted: 21 Dec 2010, 0:57:03 UTC - in response to Message 1058237.  

hehe..well...thanks for the input. Silly me, I had ASS U MEd that the default would just naturally have to be by due date. You know..efficiency and logic led me astray again...sigh..

You, of course, know the old addage about 'assuming'.
I have run afoul of it many times...LOL.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1058239 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1058462 - Posted: 21 Dec 2010, 14:52:49 UTC - in response to Message 1058043.  

Because BOINC processes tasks in First In First Out mode, unless that would cause a task to miss deadline. Then BOINC switches to Earliest Deadline First mode (or is supposed to do so, but there's no bug-free software anywhere:-).

Gruß,
Gundolf

As I pointed out in my new thread yesterday, my BOINC manager isn't following either of those logics. It was sitting on 1 Einstein WU for just over a day without even touching it. I increased my cache size dramatically and it went out and got 7 more Einsteins (without even asking Seti for any). Then it started working on 2 of the new ones and still ignored the one it had. So first in is not first out, and earliest deadline is not first worked on. I guess that leaves me at your parenthetical.

David
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1058462 · Report as offensive
Profile Miep
Volunteer moderator
Avatar

Send message
Joined: 23 Jul 99
Posts: 2412
Credit: 351,996
RAC: 0
Message 1058465 - Posted: 21 Dec 2010, 14:59:24 UTC

debts also influence what is selected to run, as do cache settings relativly to task deadlines. Same goes for workfetch.
But even with a few debug logs running, when you have several projects you do sometimes wonder why it decided to run what you perceive as 'not the right one at this time'.
Carola
-------
I'm multilingual - I can misunderstand people in several languages!
ID: 1058465 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14653
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1058477 - Posted: 21 Dec 2010, 15:39:35 UTC - in response to Message 1058462.  

Because BOINC processes tasks in First In First Out mode, unless that would cause a task to miss deadline. Then BOINC switches to Earliest Deadline First mode (or is supposed to do so, but there's no bug-free software anywhere:-).

Gruß,
Gundolf

As I pointed out in my new thread yesterday, my BOINC manager isn't following either of those logics. It was sitting on 1 Einstein WU for just over a day without even touching it. I increased my cache size dramatically and it went out and got 7 more Einsteins (without even asking Seti for any). Then it started working on 2 of the new ones and still ignored the one it had. So first in is not first out, and earliest deadline is not first worked on. I guess that leaves me at your parenthetical.

David

BOINC processes tasks from each separate project in FIFO order, unless interrupted by a possible missed deadline - so the first SETI task you receive will be the first SETI one crunched, and the first Einstein task you receive will be the first Einstein crunched, and so on.

But the choice as to whether to run SETI or Einstein is governed by debt, as Miep says.
ID: 1058477 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1058495 - Posted: 21 Dec 2010, 16:15:05 UTC - in response to Message 1058477.  

Because BOINC processes tasks in First In First Out mode, unless that would cause a task to miss deadline. Then BOINC switches to Earliest Deadline First mode (or is supposed to do so, but there's no bug-free software anywhere:-).

Gruß,
Gundolf

As I pointed out in my new thread yesterday, my BOINC manager isn't following either of those logics. It was sitting on 1 Einstein WU for just over a day without even touching it. I increased my cache size dramatically and it went out and got 7 more Einsteins (without even asking Seti for any). Then it started working on 2 of the new ones and still ignored the one it had. So first in is not first out, and earliest deadline is not first worked on. I guess that leaves me at your parenthetical.

David

BOINC processes tasks from each separate project in FIFO order, unless interrupted by a possible missed deadline - so the first SETI task you receive will be the first SETI one crunched, and the first Einstein task you receive will be the first Einstein crunched, and so on.

But the choice as to whether to run SETI or Einstein is governed by debt, as Miep says.

You're missing my point. BOINC ignored a day-old Einstein unit and started working on 2 new Einstein units. It did not follow FIFO or EDF, in the same project.

As for debt, isn't that relative to the resource shares? Mine are Seti 100, Einstein 20. I don't see how I can have built up as much debt to Einstein as it appears, from looking at what's in my cache right now, I have.

David
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1058495 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 1059185 - Posted: 23 Dec 2010, 20:59:58 UTC - in response to Message 1058126.  

I have been wondering for years now, just got the energy to see if there's answer.
Why does the BOINC client finish WU from whatever project that are due in a month and a half before it starts on ones it had that are due in less than a week?


Actually, based on my "study" of BOINC and its quirks and oddnesses, I am afraid that the correct answer to your question is:

There are some things that Men were not meant to know.

"Quirks and Oddnesses" come from the fact that the problem has more factors than most people have identified -- and conflicting goals.

It's easy with one project, uniform-time work units, and long deadlines.

The main conflicting goals:

1) Never return work late.

2) Allocate time fairly between projects.

Working in strict deadline order means that work with a longer deadline will always be put behind work with a short deadline. If you have a work unit that will take 20 hours, and you put it off until the deadline is 10 hours away, it will never be returned on time.

Balancing resource shares, and doing work First-in-First-out produces the fewest problems, but is not flawless. BOINC tries to project what will happen if it sticks with FIFO, and may assign "priority" to work units that would otherwise miss deadlines.

When someone says "there are things men aren't meant to know" it really means that they haven't identified all of the factors involved, completely studied the dynamics, or tracked down the white-papers on the subject.
ID: 1059185 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1059188 - Posted: 23 Dec 2010, 21:18:54 UTC - in response to Message 1059185.  
Last modified: 23 Dec 2010, 21:19:39 UTC


When someone says "there are things men aren't meant to know" it really means that they haven't identified all of the factors involved, completely studied the dynamics, or tracked down the white-papers on the subject.


Speaking as Someone, I'm sure you are right: without reading the code, ANY analysis as in this thread is not authoritative.

And, as a programmer for close to 50 years (yes, I go back to Fortran II and FMS), I can AUTHORITATIVELY state that even those who design and/or implement code frequently have no real idea of what it is doing, or how it (really) works. And not because I was a lousy programmer, either.
ID: 1059188 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 1059232 - Posted: 24 Dec 2010, 0:19:43 UTC - in response to Message 1059188.  
Last modified: 24 Dec 2010, 0:20:43 UTC


When someone says "there are things men aren't meant to know" it really means that they haven't identified all of the factors involved, completely studied the dynamics, or tracked down the white-papers on the subject.


Speaking as Someone, I'm sure you are right: without reading the code, ANY analysis as in this thread is not authoritative.

And, as a programmer for close to 50 years (yes, I go back to Fortran II and FMS), I can AUTHORITATIVELY state that even those who design and/or implement code frequently have no real idea of what it is doing, or how it (really) works. And not because I was a lousy programmer, either.

I don't go back that far, my early training was assembly language on an IBM 1620.

On cards. In 1969.

Most of the answers to questions can be found in the BOINC White Papers on the BOINC web site.

Those are pretty definitive, if sometimes slightly out of date.

You can also experiment: stop BOINC and change the DCF to something wildly out of range, like ten times what it should be, and you'll see BOINC process work "in priority mode" because it thinks it will take too long to finish.

As the work unit progresses, the work estimate shifts from an initial estimate based on what the project thought, to one based on elapsed time and percent done. At some point, the estimate says the result will clearly be on time, and BOINC will move to one that's still "in trouble" based on the belief that it'll take ten hours instead of one.

... and it will do "priority" work when it might not be late, just to be safe.

(It'd also make more sense if BOINC Manager showed download order, but I don't think you can).

I listed two factors. When I asked about how the scheduler decides what to do, there were others that I've forgotten.

It's predictable if you can track down all the factors. It is also sufficiently complex that tracking them all down is hard work. I'd love to see a "show your work" option in BOINC that made the process more visible.

The main point: the first rule is don't miss deadlines -- if deadlines are all safe, BOINC is pretty strictly FIFO and resource share. If deadlines aren't safe, it's by deadline, and then debts repair the imbalance when deadlines had to come first.
ID: 1059232 · Report as offensive
Profile skildude
Avatar

Send message
Joined: 4 Oct 00
Posts: 9541
Credit: 50,759,529
RAC: 60
Yemen
Message 1059294 - Posted: 24 Dec 2010, 3:06:16 UTC - in response to Message 1059232.  

I think some projects are just hardwired differently. I've had PG WU's fail to meet their deadline because WCG decided it wanted to process first. I've seen this activity with a few projects and realize I need to actually watch those machines a little closer when I have the greedy projects on board


In a rich man's house there is no place to spit but his face.
Diogenes Of Sinope
ID: 1059294 · Report as offensive
John M. Kendall
Avatar

Send message
Joined: 15 May 99
Posts: 30
Credit: 11,800,561
RAC: 164
United States
Message 1059313 - Posted: 24 Dec 2010, 4:44:57 UTC

Sometimes you can force a switch to tasks due sooner by suspending the active task(s). Then sometime you can't.

Of course maybe we have found the scheduling code that the aliens wrote. *:^)

"Happy Christmas to all, and to all a good-night."
ID: 1059313 · Report as offensive

Message boards : Number crunching : client priorities


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.