Choosing tasks to beat deadlines

Message boards : Number crunching : Choosing tasks to beat deadlines
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Gordon Lowe
Avatar

Send message
Joined: 5 Nov 00
Posts: 12094
Credit: 6,317,865
RAC: 0
United States
Message 1965197 - Posted: 14 Nov 2018, 22:14:20 UTC

I'm sorry if this question has been answered in the past, but I would like to be able to choose tasks to crunch giving priority to the oldest ones. Is this possible, and if so, how do you do it? This seems like something the system should do automatically, that way nothing ever gets close to it's deadline.
The mind is a weird and mysterious place
ID: 1965197 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1965198 - Posted: 14 Nov 2018, 22:23:31 UTC - in response to Message 1965197.  

BOINC always crunches FIFO - in order of receipt, first in first out. You can force it to switch to 'Earliest Deadline First' by super-loading the cache with a complete deadline's worth of tasks. Please don't even try that - you can't achieve it with SETI's long deadlines and task limits, and you are being unfair to the projects by cluttering their databases with tasks that can't be purged until you eventually get round to it.

Please, just let BOINC do it's thing. I find 0.25 days (6 hours) is fine, except on Tuesdays.
ID: 1965198 · Report as offensive
Profile Gordon Lowe
Avatar

Send message
Joined: 5 Nov 00
Posts: 12094
Credit: 6,317,865
RAC: 0
United States
Message 1965204 - Posted: 14 Nov 2018, 22:57:16 UTC - in response to Message 1965198.  

Fair advice. It just seems like it would be more efficient if it crunched in order of deadline.
The mind is a weird and mysterious place
ID: 1965204 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1965245 - Posted: 15 Nov 2018, 0:42:49 UTC

It seems like it is FIFO unless something is getting close to the deadline. I've had it stop processing a WU in the middle to work on a new WU I just got with a close deadline (2 weeks??)
ID: 1965245 · Report as offensive
Profile Gordon Lowe
Avatar

Send message
Joined: 5 Nov 00
Posts: 12094
Credit: 6,317,865
RAC: 0
United States
Message 1965247 - Posted: 15 Nov 2018, 0:48:31 UTC - in response to Message 1965245.  

It seems like it is FIFO unless something is getting close to the deadline. I've had it stop processing a WU in the middle to work on a new WU I just got with a close deadline (2 weeks??)

I haven't noticed that myself. I just get concerned when I see old tasks getting close to their deadlines, and newer ones crunching along.
The mind is a weird and mysterious place
ID: 1965247 · Report as offensive
Profile Gordon Lowe
Avatar

Send message
Joined: 5 Nov 00
Posts: 12094
Credit: 6,317,865
RAC: 0
United States
Message 1965252 - Posted: 15 Nov 2018, 1:15:26 UTC - in response to Message 1965247.  

This is the sort of thing that concerns me -





I have a task from October sitting there that hasn't even started yet, with a December deadline, but tasks sent just today, with January deadlines, are being crunched.
The mind is a weird and mysterious place
ID: 1965252 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1965263 - Posted: 15 Nov 2018, 1:43:49 UTC

I have a task due Dec 4 that I just got today. If I haven't gotten to it in Fifo order by 2 weeks (guess) out (Nov 20) then it will swap to it. It usually takes me 5-7 days to work through my store of WUs, so no worry from me.
ID: 1965263 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1965297 - Posted: 15 Nov 2018, 5:26:59 UTC - in response to Message 1965252.  



I have a task from October sitting there that hasn't even started yet, with a December deadline, but tasks sent just today, with January deadlines, are being crunched.


ok. I just read this again, and it bothers me. Fifo is the general rule, so it should be working on the WUs you got in October.

It looks like the 2 you have left from October were assigned to your GPU and you can only do one at a time on your GPU. Note the info in the application name column.
The rest of the WUs you have are for your CPU and it looks like you can do 2 CPU processes at the same time.
ID: 1965297 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 9954
Credit: 103,452,613
RAC: 328
United Kingdom
Message 1965308 - Posted: 15 Nov 2018, 7:05:11 UTC

I have a task from October sitting there that hasn't even started yet, with a December deadline, but tasks sent just today, with January deadlines, are being crunched.


Except the October ones are GPU tasks and seem to be taking 11 hours.

Whereas the November tasks are CPU which seem to be taking around 3 to 4 hours.

So knowing nothing about Apple Macs it still seem that perhaps there is a problem with the GPU, it is old but 11 hours still seems a bit much.
ID: 1965308 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1965315 - Posted: 15 Nov 2018, 7:50:37 UTC - in response to Message 1965204.  

Fair advice. It just seems like it would be more efficient if it crunched in order of deadline.

Nope.
It's the date that the WU is sent out that is important, not it's return by date; it's most efficient is to crunch them in the order they arrive.
The Seti database that has to keep track of all the WUs and all the Computers and all the users & all the results would actually be worse off if WUs were processed in order of their due date.
Grant
Darwin NT
ID: 1965315 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22199
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1965320 - Posted: 15 Nov 2018, 8:37:53 UTC

Process by deadline would mean that "long deadline" tasks would sit around until they bubbled to the top, and if one was running a project that always gave out tasks with very short deadlines they would always run, and projects with very long deadlines would bog down for ages.
FIFO makes life "easier" for BOINC to sort out which task to run next - just take the one off the top of the pile. Periodically there is an internal check to see if any task is in danger of not being processed by its deadline, in which case it gets promoted up the list and runs at high priority. I think some projects milk this by sending out tasks with very short deadlines and very long "expected processing requirements" so they promoted to high priority.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1965320 · Report as offensive
Profile Gordon Lowe
Avatar

Send message
Joined: 5 Nov 00
Posts: 12094
Credit: 6,317,865
RAC: 0
United States
Message 1965352 - Posted: 15 Nov 2018, 15:17:05 UTC - in response to Message 1965308.  

Well, my Dec. deadline straggler is now crunching, but Bernie, you make a great point. Is there a way to test a GPU?






And Rob, what you said also makes sense. I guess the system is working and I'm just inflicting my human worries on things. ;~)
The mind is a weird and mysterious place
ID: 1965352 · Report as offensive
Profile Gordon Lowe
Avatar

Send message
Joined: 5 Nov 00
Posts: 12094
Credit: 6,317,865
RAC: 0
United States
Message 1965401 - Posted: 15 Nov 2018, 19:41:22 UTC

Ok, everything sent before today has been processed, and I have a new batch in queue, all from November 15th, and as you can see, once again, the two December deadline tasks are sitting idle while the later deadline tasks crunch merrily away. It just bothers me, no big deal. ;~)




The mind is a weird and mysterious place
ID: 1965401 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1965402 - Posted: 15 Nov 2018, 19:47:12 UTC - in response to Message 1965401.  

if you look at the application column you can see which tasks are assigned to gpu vs cpu. you can crunch WUs faster on your cpu, and you have 2 cpus so tasks in the cpu queue go quicker than the ones in your gpu column which take longer and can only process one at a time.
ID: 1965402 · Report as offensive
Profile Gordon Lowe
Avatar

Send message
Joined: 5 Nov 00
Posts: 12094
Credit: 6,317,865
RAC: 0
United States
Message 1965404 - Posted: 15 Nov 2018, 19:50:21 UTC - in response to Message 1965402.  

Yeah, well, I just need to stop looking. ;~)
The mind is a weird and mysterious place
ID: 1965404 · Report as offensive

Message boards : Number crunching : Choosing tasks to beat deadlines


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.