Task Deadlines - Near Deadline Tasks Not Running First

Message boards : Number crunching : Task Deadlines - Near Deadline Tasks Not Running First
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Chucky
Volunteer tester

Send message
Joined: 1 Feb 04
Posts: 4
Credit: 457,846
RAC: 0
Australia
Message 1759646 - Posted: 27 Jan 2016, 11:46:23 UTC

Hi Community,

A question for you. I happily use BOINC Manager running 3 projects and SETI is my original one. :)

At times I will have tasks from those sources listed in my tasks tab and have them sorted by Deadline. This makes it easier to see what deadline is approaching etc.

So why does the client number crunch some deadlines which are not next? I'm trying to work out in my head why a task with deadline of March 19th would be processing before some 30 other tasks with deadlined before it and even as close as Jan 29th and Feb 15th. It certainly doesn't make sense.

Is there any plans to make the BOINC Manager client process in order of deadline? OR can I request this feature from somewhere? I'm using 7.6.22 (x64) which I recently updated to. FYI this is not a new thing, just the first time I've made an effort to communicate about it. I'm sure such questions have been asked before.

Cheers,
Chucky!
ID: 1759646 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1759656 - Posted: 27 Jan 2016, 12:15:39 UTC - in response to Message 1759646.  

If you didn't have the tasks sorted on deadline - in fact if you had them not sorted at all - you would see that a clear policy applies:

Tasks are processed in the order they are received from the server(s).

That's known as 'FIFO', for First In, First Out. It's actually much better for processing when there are mixed deadlines in the queue. Most other projects have deadlines of one or two weeks: under your proposal, their work would always run before SETI. On the other hand, CPDN - who famously have workunits with deadlines of up to a year - would never get any work done at all, until it was too late for the research the work was supposed to support.

There is a mechanism built into BOINC to continuously monitor the workload, and the deadlines of all cached tasks. If any tasks are at risk of missing their deadlines under the normal FIFO order, they are brought forward and processed early.
ID: 1759656 · Report as offensive
Rasputin42
Volunteer tester

Send message
Joined: 25 Jul 08
Posts: 412
Credit: 5,834,661
RAC: 0
United States
Message 1759657 - Posted: 27 Jan 2016, 12:17:08 UTC - in response to Message 1759646.  
Last modified: 27 Jan 2016, 12:18:12 UTC

I have been complaining about this a long time ago.
Seti seems to be the only project that computes the tasks in order they were received.
In theory, there should be a mechanism that runs task, that are in danger of missing the first. However, i do not know, if that works.
ID: 1759657 · Report as offensive
Profile William
Volunteer tester
Avatar

Send message
Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1759664 - Posted: 27 Jan 2016, 12:32:41 UTC
Last modified: 27 Jan 2016, 12:51:52 UTC

BOINC all the time recalculates if tasks are going to meet their deadline, when running in the order received (FIFO) and according to the resource share that project has. IOW it has a list of when it's going to start a certain task and when that task is going to finish (acoording to how long _BOINC thinks_ it's going to take. may be way off)
IF a task is in danger of missing its deadline (incl. safety margin) then BOINC puts it into EDF - Earliest Deadline First, also know as 'high priority' (that's what ir used to show up as in the boinc manager).
Since 'high priority' confused the hell out of people, leading them to the wrong conclusions about what it actually meant, this label was removed from BOINC clients a while back (not the best approach IMO).
Nowadays, if you want to ascertain a task is in EDF you need to activated the 'cpu_sched_debug' log flag. WARNING advanced users only. You are prone to be more confused by the log output than informed.
IF you run it and need it explained, post a sample.
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1759664 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1759665 - Posted: 27 Jan 2016, 12:35:03 UTC - in response to Message 1759657.  

I have been complaining about this a long time ago.
Seti seems to be the only project that computes the tasks in order they were received.
In theory, there should be a mechanism that runs task, that are in danger of missing the first. However, i do not know, if that works.

Neither SETI nor any other project chooses the task processing order - it's your local BOINC (client) that does that.

To amplify my previous answer: BOINC processes the tasks received from each project server in FIFO order of tasks received from that project. The question of which project is chosen for the next task to start on a free resource takes into account project resource share and recent processing history.
ID: 1759665 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1759666 - Posted: 27 Jan 2016, 12:38:13 UTC - in response to Message 1759657.  

Seti seems to be the only project that computes the tasks in order they were received.

Seti's science applications don't do the scheduling, that's something that BOINC does. So for all projects you've added to BOINC, scheduling will happen in the order the tasks were received in.

That it's run in order that they're received in and not in order of deadline is because between all 60+ projects under BOINC the projects have various deadlines. One project may run with 24 hour deadlines, another 3 days, another 14 days, another 3 months and another maybe a year.

When you decide to run a mixture of those projects and they're done in order of deadline, the projects with the 24 hour deadline and 3 day deadline will be run forever, and by the time the 14 day deadline or 1 year deadline projects are up, it may be way too late to run those tasks before their deadline.

Now, luckily there is some debt scheduling happening as well, where BOINC will stop asking for work from a certain project if it has done a lot of work, then BOINC will try to pay back that time to the other projects allowed to ask for and getting work.

Which works fine, as long as the user doesn't start interfering and deciding that he wants BOINC to do things his way.
[rant to no one in general] Which is something I don't really get, as why is there that need to decide for BOINC how to run things? Are these people telling Windows to run only with the 3 services they like, and not with any of the others? If no, then why do they feel they need to interfere in the scheduling of BOINC? [/rant]
ID: 1759666 · Report as offensive
Profile William
Volunteer tester
Avatar

Send message
Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1759669 - Posted: 27 Jan 2016, 12:45:22 UTC - in response to Message 1759657.  

I have been complaining about this a long time ago.
Seti seems to be the only project that computes the tasks in order they were received.
In theory, there should be a mechanism that runs task, that are in danger of missing the first. However, i do not know, if that works.


SETI works with looooooooong deadlines. about 6 weeks for a mid AR task. Even my stoneage 1Ghz SSE CPU finishes a midAR tasks in about 22 hours. So, SETI is almost never in risk of missing a deadline.

And don't forget the 'minimum work' you specify works as 'maximum time between two scheduler contacts' too, so if you set a 7+7 cache and gets tasks from a project that has 7 day deadlines those tasks will instantly go into EDF. (of course the project won't be asked for tasks in a hurry again after the tasks got run first)
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1759669 · Report as offensive
Rasputin42
Volunteer tester

Send message
Joined: 25 Jul 08
Posts: 412
Credit: 5,834,661
RAC: 0
United States
Message 1759673 - Posted: 27 Jan 2016, 13:17:25 UTC

The scheduling is based on the expected time remaining,which can be vastly off, especially when a project has tasks of very different types and length.

On the other hand, CPDN - who famously have workunits with deadlines of up to a year - would never get any work done at all, until it was too late for the research the work was supposed to support.


Incorrect, see resources share.
ID: 1759673 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1759675 - Posted: 27 Jan 2016, 13:33:21 UTC - in response to Message 1759673.  

On the other hand, CPDN - who famously have workunits with deadlines of up to a year - would never get any work done at all, until it was too late for the research the work was supposed to support.

Incorrect, see resources share.

I was speaking in the context of the OP's proposal for a plain-vanilla deadline-only scheduler. You and I both know that many other factors are taken into account, but I didn't want to overwhelm somebody with a post count of 3 (the last one ten years ago) with acronyms like REC.
ID: 1759675 · Report as offensive
Rasputin42
Volunteer tester

Send message
Joined: 25 Jul 08
Posts: 412
Credit: 5,834,661
RAC: 0
United States
Message 1759676 - Posted: 27 Jan 2016, 13:35:30 UTC - in response to Message 1759675.  

OK-let's leave it at that.
ID: 1759676 · Report as offensive
Profile Chucky
Volunteer tester

Send message
Joined: 1 Feb 04
Posts: 4
Credit: 457,846
RAC: 0
Australia
Message 1759973 - Posted: 28 Jan 2016, 11:21:15 UTC

Hi All, :)

Wow that was fun refreshing tonight and seeing all the conversation. hehe. Glad I could spark some interesting answers. Thanks by the way.

Don't be fooled by 3 posts. I'm sure it's true. I'm not in science, but am in ICT. I understood about 90%+ of your comments and abbreviations. FIFO and LIFO, no worries, they are terms across many subjects (like accounting for investment transactions! or Fly In Fly Out workers for the mining industry in Australia - now there's an oddball).

BOINC is happily crunching away in the background whether I post or not. ;]

Have a great week!
Chucky!

PS. Love the rant Jord! Imagine what I've experienced after 20yrs in ICT haha. It's possible some alien lifeforms are amongst us already. *giggle*
ID: 1759973 · Report as offensive
Sleepy
Volunteer tester
Avatar

Send message
Joined: 21 May 99
Posts: 219
Credit: 98,947,784
RAC: 28,360
Italy
Message 1759980 - Posted: 28 Jan 2016, 11:59:22 UTC

Adding my 2 cents of not understanding sometimes BOINC scheduling is that if BOINC at some point gets in "Panic mode" and fears it does not make it on time, it does not begin to crunch the WUs that have the nearest deadlines. Rather, it may even put them in waiting to run and starts to crunch the longest deadlines.
As if it wanted to get a taste of those as well to make better decisions. But risking even more not to match the most urgent tasks.

I am still on 6.0.55 however and newer BOINC version may behave differently and perhaps better.

I will follow ASAP the advice published some time ago on how to install a newer BOINC version, but still preserving the old interface I like better.

I must still thank for that piece of advice and apply it.

Cheers,
Sleepy
ID: 1759980 · Report as offensive
Profile William
Volunteer tester
Avatar

Send message
Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1759993 - Posted: 28 Jan 2016, 13:43:02 UTC

@sleepy scheduling behaviour was completely redone in BOINC 7. yours still has something called 'long term debt' and yes in boinc v6, if a _project_ experienced EDF then it picks the one at the end of the list first and works it's way backwards, leading to the behaviour you are seeing. As Jord said at the time, he had yet to see a task actually no make the deadline with that system, even though it seems counterintuitive.
Anyway, workfetch, resource share and scheduling are completely different in new BOINCs. I always found it works better if you have multiple projects.
The interface got some nice features added too, you may want to try it.
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1759993 · Report as offensive

Message boards : Number crunching : Task Deadlines - Near Deadline Tasks Not Running First


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.