Processing *within* project in FIFO order rather than deadline order?

Message boards : Number crunching : Processing *within* project in FIFO order rather than deadline order?
Message board moderation

To post messages, you must log in.

AuthorMessage
nsandersen

Send message
Joined: 3 Apr 99
Posts: 3
Credit: 764,585
RAC: 796
United Kingdom
Message 1893144 - Posted: 4 Oct 2017, 12:04:30 UTC

Almost the same question as this thread:
https://setiathome.berkeley.edu/forum_thread.php?id=78932
Just adding "within each project".

I have seen a few times where Einstein@Home starts an Einstein work unit with a longer deadline and then therefore misses the deadline for an Einstein work unit with a shorter deadline, which from my point of view wasn't necessary - it would have made both deadlines if it had started with the more urgent unit.

Does it make sense to run things FIFO (again within each project) in these cases - is it a question of BOINC ensuring fairness for different researchers within each project.., ie. BOINC preventing some work units from skipping the queue?
ID: 1893144 · Report as offensive     Reply Quote
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 21099
Credit: 30,568,189
RAC: 21,888
United States
Message 1893201 - Posted: 4 Oct 2017, 17:54:48 UTC - in response to Message 1893144.  

Yes/no. Scheduling algorithms are very well understood. They do things that no sane human would consider, but they work and are very fair when tested statistically.

I'm going to guess here, as there is no schedule debug log attached for your specific case, that when the work was requested that there was more than enough crunch time available to process them in FIFO order. However something happened. Perhaps you ran other tasks on the machine that changed how much crunch was free, or perhaps the machine was off longer than average, or the estimated time was way off. At some point it became not enough crunch time to meet deadline. BOINC goes into EDF mode. In EDF mode it may not switch which workunit it is doing in favor of another because it is trying as hard as it can to get the workunit that is partially done finished ASAP. That can mean several other workunits may miss deadline. Also note EDF mode is by project. This so that BOINC obeys the preference items on how many jobs per project to run at the same time, how many cores to use, etc.

Hope this gives you an idea of what may have happened.
ID: 1893201 · Report as offensive     Reply Quote
nsandersen

Send message
Joined: 3 Apr 99
Posts: 3
Credit: 764,585
RAC: 796
United Kingdom
Message 1893335 - Posted: 5 Oct 2017, 8:09:47 UTC - in response to Message 1893201.  

That makes more sense, it seems to struggle a bit in tight cases in this EDF mode, but perfection is hard to find!

Yes, it would likely have estimated that there was enough time originally. Unfortunately my CPU time for BOINC is not guaranteed, so this sometimes gives problems for projects with just a few days as deadlines, such as Einstein@Home when sharing with a handful of other projects.

Are there any settings I can use to help it request/get slightly longer deadlines for these?

(Or shall I assume the research does not work well with longer processing times.. although I would have imagined it isn't a problem for gravity wave related work!)

Thank you.
ID: 1893335 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 8881
Credit: 115,037,197
RAC: 70,105
Australia
Message 1893336 - Posted: 5 Oct 2017, 8:19:09 UTC - in response to Message 1893335.  

Are there any settings I can use to help it request/get slightly longer deadlines for these?

Nope.
The deadlines are set by the projects.
If running multiple projects, and BOINC only has a limited amount of time each day/week to run, then running with a small to non existent cache will help reduce conflicting deadline battles. Particularly where one project has very short deadlines, and another project has very long deadlines.
Grant
Darwin NT
ID: 1893336 · Report as offensive     Reply Quote
nsandersen

Send message
Joined: 3 Apr 99
Posts: 3
Credit: 764,585
RAC: 796
United Kingdom
Message 1893582 - Posted: 6 Oct 2017, 8:07:29 UTC - in response to Message 1893336.  

OK, well internet connection is not so restricted, so I'll try that. Thank you.
ID: 1893582 · Report as offensive     Reply Quote

Message boards : Number crunching : Processing *within* project in FIFO order rather than deadline order?


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.