Best way to control flow of new tasks?

Questions and Answers : Windows : Best way to control flow of new tasks?
Message board moderation

To post messages, you must log in.

AuthorMessage
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2023450 - Posted: 17 Dec 2019, 5:13:49 UTC

Win10 1909, up-to-date, last checked 20 min ago; Intel i5-8250u
BOINC 7.14.2 (x64); no later version available
participate in: SETI@Home, Einstein, and Rosetta. So far.

BOINC runs only when I'm at the computer or downloading.

My computer usage can be variable. Some weeks, I might use the computer twice as much as others.

All too frequently, my computer fails to complete tasks before their deadline.

This seems a condition for getting a reasonable minimum of tasks.
    My computer will start work on tasks very soon after downloading them, complete them nearly as quickly as it can, and seldom fail to complete a task within the deadline. All would seem to be of advantage to the projects.


    My computer will seldom run out of work because if one project stops sending tasks, there are likely two other projects available to provide a task if needed. I consider this an advantage.



I set constraints to:

    Store at least 0.01 days of work. (near 15 min)


    Store up to an additional 0.04 days of work. (near 60 min)


    Almost all other constraints unlimited (at most 100% of CPUs, etc.)



Expected BOINC response:

    Once downloading a task (most any task these days! 😊), BOINC will recognize it has more than the allotted 75 min of work and decide not to download further tasks.


    When a task is within 15 min of completing, it downloads from the priority project according to published BOINC criteria.


    When a task completes, it starts the waiting task.



An expectable BOINC response: Same as above, but on a project basis. Each project always has one task on my computer but has two if the first one is within 15 min of completing.

The actual BOINC response: Far more tasks downloaded.

For example, now:

    One project has 7 tasks here, with elapsed + remaining times spread between around 5 hours for the most recently started tasks and roughly 24 hours for the first-started tasks. Three of these tasks are past their deadline.


    One project as one task here. It estimates to use 10 hours of computer time. It went overdue today.


    One project with four tasks here, all estimating an hour or two of computer time. The two longest-running tasks estimate 50% done with about two weeks to its deadline. The next task due has used 28 seconds of time. The last task, with three more weeks to its deadline, is about 20% complete.



My question: Have I options to bring task flow closer to my goal?


ID: 2023450 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13206
Credit: 208,696,464
RAC: 304
Australia
Message 2023452 - Posted: 17 Dec 2019, 5:47:45 UTC - in response to Message 2023450.  
Last modified: 17 Dec 2019, 5:55:12 UTC

    Store at least 0.01 days of work. (near 15 min)

    Store up to an additional 0.04 days of work. (near 60 min)

My question: Have I options to bring task flow closer to my goal?

Swap around your cache settings, so it becomes
Store at least 0.04 days of work.
Store up to an additional 0.01 days of work.

Once you make the changes & save them, click on Update on the Manager for it to get those new settings.
It will still take some time as the Manager determines how long it takes to complete a WU, how many hours a day the system is running, and how much time when the system is running is available for BOINC to use, but swapping those settings values around should help things to sort themselves out.
Grant
Darwin NT
ID: 2023452 · Report as offensive
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2023528 - Posted: 18 Dec 2019, 2:56:47 UTC - in response to Message 2023452.  

😏 Oh! Another of those, "Why didn't I think of that?" moments. Or another of those, "Fifty-fifty chance, and what'd I get?!" moments.

🙏 Thanks for the quick reply. I'll monitor for several weeks to see it adjust the estimates. Much appreciated.

. . . Garry

PS. Stay warm! 🤣 In other words: Enjoy the indoors! 😊
ID: 2023528 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13206
Credit: 208,696,464
RAC: 304
Australia
Message 2023548 - Posted: 18 Dec 2019, 7:09:02 UTC - in response to Message 2023528.  

PS. Stay warm! 🤣 In other words: Enjoy the indoors! 😊
Here, this time of the year, it's cooler indoors thanks to the aircon (27°c). Outside it' been around 36°c most days with the humidity between 50%-75%, and only getting down to 27°c or so over night.
Grant
Darwin NT
ID: 2023548 · Report as offensive
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2025525 - Posted: 30 Dec 2019, 3:31:12 UTC - in response to Message 2023548.  
Last modified: 30 Dec 2019, 3:32:33 UTC

As to SETI:

Swap around your cache settings, so it becomes
Store at least 0.04 days of work.
Store up to an additional 0.01 days of work.


Did that. Also drained down the tasks on this computer. Then allowed more tasks to come in. Watched for a while.

My perception: Probably improved, as you say. However, it seems to want to have eight active tasks at a time (the number of threads on the computer, perhaps not a coincidence) and perhaps five or six more waiting.

Accordingly, there remains quite a high probability of missing deadlines. Sigh.

I haven't seen the computer hit 100% CPU with either one or two tasks running, but it always has 100% with three tasks running.

If I could keep only one or two tasks in inventory per project, I could keep all projects busy and balanced while greatly reducing the chance of being late.

Any other way to accomplish the goal?

Thanks in advance.

As to geography: Dooh! 🤣 I was horribly geographically mistaken. This time of year, most folks in the Northern Hemisphere envy your weather. Your flag should have prevented the error, but I placed you well north of me. My mistake entirely.

Enjoy your weather! I went out tonight with high winds and blowing snow on the roads. I doubt you envy that!
ID: 2025525 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13206
Credit: 208,696,464
RAC: 304
Australia
Message 2025545 - Posted: 30 Dec 2019, 8:44:33 UTC - in response to Message 2025525.  

Any other way to accomplish the goal?
Nothing comes to mind, other than waiting a while longer.
Have all the other projects number of WUs also declined in accordance with the changed cache settings? The less time a system is running, and BOINC is able to process work when the system is on, then the longer it will take for changes to become effective.
And there have been some Seti server issues recently, which could cause some odd behaviour as it tries to balance out resource share with other projects.
Grant
Darwin NT
ID: 2025545 · Report as offensive
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2025657 - Posted: 31 Dec 2019, 2:22:28 UTC - in response to Message 2025545.  

Have all the other projects number of WUs also declined in accordance with the changed cache settings?


I don't know that I'm giving the system a chance to fully adapt.

I just adjusted the computing options to "at least 15 min; at most 15 min more". So maybe it has to start over.

I'm still vulnerable to late tasks. For instance, perhaps three to six days ago, I received a task from Rosetta that has accumulated 6 hours and needs 18 hours more in the next 24 hours. It'll be late.

Rosetta has only one more task here. It needs 7 hours in the next 3 days. It'll probably make it.

Einstein has two tasks here, both due in 14 days, requiring 20 hours between them. Should make it.

SETI has six tasks here, four due in 3 1/2 weeks and two due in 5 weeks, requiring less than 14 hours between them. Easy.

During the time I downloaded these tasks, I probably aborted six or eight tasks on arrival. Some of those probably wouldn't have come in had all tasks here been available to run at download time.

A download process I'll try:
1) Stop network activity.
2) Resume all tasks in inventory.
3) Put all projects in "allow new tasks".
4) Resume network activity. Allow downloading to complete.
5) If too many tasks come in, put all projects in "no new tasks". Abort appropriate new tasks. Suspend last-due tasks to give the first-due tasks a better chance. Then manage resuming additional tasks so that the first-due tasks complete on time. Perhaps maintain balance by typically running two tasks for each project at a time. When low on tasks, return to step 1.

Maybe after a time, I'll see more improvement.
ID: 2025657 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13206
Credit: 208,696,464
RAC: 304
Australia
Message 2025679 - Posted: 31 Dec 2019, 5:56:40 UTC - in response to Message 2025657.  
Last modified: 31 Dec 2019, 5:59:01 UTC

I don't know that I'm giving the system a chance to fully adapt.
That is definitely the case,
During the time I downloaded these tasks, I probably aborted six or eight tasks on arrival.
What you are doing is stopping the manager from determining what your system can & can't do. Hence it keeps doing the wrong thing.


Don't abort tasks, don't suspend tasks to let others process, don't select & deselect No New Tasks. Everything you're doing there changes things, and so the Manager has to re-work everything each time you do those things. Set it back to
Store at least 0.04 days of work.
Store up to an additional 0.01 days of work.

This will provide a very low upper limit on the amount of work it will get. Don't abort tasks. Don't pause some to let others process. Don't set & then un-set No New Tasks.
Just let it do it's thing. If WUs miss the deadline that's good, it allows the manger to determine just how long it takes for it to process work once it downloads it.
It wouldn't surprise me if it takes close to a month for things to fully settle down, but if you go back to the 0.04 days of work/ additional 0.01 days of work cache, and just let it be I would expect it should come pretty close to sorting things out within 2-3 weeks- as long as you don't abort tasks, suspend & re-enable tasks, disallow & re-allow new work.
Just let it do it's thing.

Micro-managing work processing almost always results in anything but the desired result.
Grant
Darwin NT
ID: 2025679 · Report as offensive
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2025849 - Posted: 1 Jan 2020, 3:42:12 UTC - in response to Message 2025679.  

🤣 (blush) I'm the guy being tactfully told, "You're saying you don't have time to help drain the swamp because you're fighting alligators?" (Pick the scary water critter where you are!) 🙂

Well done, sir!

.04 and .01. No task suspended. All projects allowed new tasks. I'll hold myself back until at least Jan 21.

Happy New Year!

(And I'll note that since I'm a bit over two hours away, the vast majority of the world has already celebrated. And Australia is among the earliest!)
ID: 2025849 · Report as offensive
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2028184 - Posted: 17 Jan 2020, 21:02:26 UTC - in response to Message 2025849.  

I'll hold myself back until at least Jan 21.
It's a little earlier than I said, but I'll report. My system has settled down nicely, despite an unusual period of three or four days with the computer off. (And maybe because of it!)

Steady-state seems to be keeping one task in progress per thread, plus up to two waiting (and often none or one). Task lengths vary but seem appropriate to due dates.

At this minute, three tasks have the earliest of the due dates in five days. Two other tasks are due a week later. One other task is due one day later yet. All times remaining are reasonable for the time I expect the computer to be on.

From the BOINC perspective, I can accept this steady-state as "getting a reasonable minimum of tasks" (my goal).

FYI, the scheduler probably settled within roughly 36 hours of operation, roughly 1.5 times the longest tasks I'm getting. If that's typical, it seems likely it'll seldom cause a late task, unless daily usage varies a great deal.

Perhaps my usage was varying more than I was aware. Or perhaps that period of off time was helpful in getting on track.
ID: 2028184 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13206
Credit: 208,696,464
RAC: 304
Australia
Message 2028194 - Posted: 17 Jan 2020, 21:18:43 UTC

Glad it's behaving as desired.
Grant
Darwin NT
ID: 2028194 · Report as offensive
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2028670 - Posted: 20 Jan 2020, 20:26:52 UTC - in response to Message 2028194.  
Last modified: 20 Jan 2020, 20:27:52 UTC

Well, I was too hasty in becoming satisfied with the scheduler.

In the last day or so, it accepted three tasks from Rosetta. Each needs 24 hours of processing. All are due tomorrow.

That seems improvable!

. . . Garry
ID: 2028670 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 20186
Credit: 416,307,556
RAC: 380
United Kingdom
Message 2028677 - Posted: 20 Jan 2020, 22:27:01 UTC

Have you done any Rosetta work previously?
If not it is the usual poor initial estimate....
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 2028677 · Report as offensive
Darrell Wilcox Project Donor
Volunteer tester

Send message
Joined: 11 Nov 99
Posts: 303
Credit: 180,954,940
RAC: 118
Vietnam
Message 2028682 - Posted: 20 Jan 2020, 23:09:31 UTC - in response to Message 2028670.  

@Garry
I suggest you visit your Rosetta preferences and set the desired processing time to one (1) hour. If this is met,
increase it by an hour or two after a week or so. Rosetta is able to complete short WUs.

Einstein, however, is terrible at giving a reasonable amount of work.
ID: 2028682 · Report as offensive
Garry

Send message
Joined: 7 Jul 02
Posts: 40
Credit: 535,102
RAC: 1
United States
Message 2028818 - Posted: 23 Jan 2020, 19:41:18 UTC - in response to Message 2028682.  

@RobSmith
I've been doing Rosetta for quite some time, since Jan 1 without changing any computing preferences.



@DarrelWilcox
Ah! Would but that all projects had such controls.

Thanks tons. This'll go a long way to solving my problems.



Have a great day!

. . . Garry
ID: 2028818 · Report as offensive

Questions and Answers : Windows : Best way to control flow of new tasks?


 
©2021 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.