BOINC WU priority for projects

Message boards : Number crunching : BOINC WU priority for projects
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
S@NL - EJG
Volunteer tester

Send message
Joined: 21 Apr 00
Posts: 64
Credit: 25,162,101
RAC: 0
Netherlands
Message 77570 - Posted: 8 Feb 2005, 17:46:08 UTC - in response to Message 77069.  

> But what has happened is that I've already run out of Predictor units, BOINC
> is now switching between CPDN and SETI, leaving the Einstein WU's untouched
> (not 1 even part crunched).

I had something like this a couple of days ago, on a PC running Seti/Predictor/Einstein. The Einstein WU just didn't start. I suspended all projects except Einstein (using Boinc 4.65 at that moment), forcing Boinc to start the Einstein WU. It did start, then I unsuspended the other projects and everything was back normal after that.
ID: 77570 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 154
Credit: 10,720,947
RAC: 822
United Kingdom
Message 77564 - Posted: 8 Feb 2005, 17:34:30 UTC - in response to Message 77491.  
Last modified: 8 Feb 2005, 17:35:02 UTC

> Each project runs on each own individual queue, there 1st wu downloaded is 1st
> crunched.
> Switching between projects happens based on resource-share and the dept
> between them, the dept is checked each time a wu have finished or based on how
> often should switch between projects, default every 60 minutes.


Hi Ingleside,

See my reply to Matt - but maybe if a long deadline project like CPDN gets the focus of BOINC, then possibly it doesn't want to relinquish control to other projects - I got the impression that in my case BOINC had "ignored" the 5 WU's that needed processing - this went on for about 3 days before I realised that these WU's would not be finished in time, but there was no way I could get BOINC to do anything about them !!

>
> Neither of these cares about deadlines, meaning a project with multiple
> deadlines will crunch the 7-day-wu before the later downloaded wu with only
> 1-day-deadline, and the client can happily switch to a CPDN-wu even was only 5
> minutes left to crunch an Einstein@home-wu that passed it's deadline 1 hour
> ago...


I'm sure that this DOES happen - that earlier downloaded WU's get priority over later WU's with shorter deadlines.


>
> There's plans to let the client temporarily boost individual projects
> resource-share so wu will not pass deadline, and not ask a low
> resource-share-project so often for work, but till this is implemented the
> changes in v4.6x expected released in a fortnight with manual pausing of
> projects/wu and removing individual wu should hopefully minimize most problems
> with the current implementation.
>

That would be welcome - also anything that allows the user a greater degree of control over what is being crunched aould help - maybe there's a need for an "advanced" setting in BOINC to allow for this type of situation.

regards,

Timbo
regards,
Tim
Founder, UK BOINC Team
Join us @ UK BOINC Team: http://www.ukboincteam.org.uk/newforum
ID: 77564 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 154
Credit: 10,720,947
RAC: 822
United Kingdom
Message 77561 - Posted: 8 Feb 2005, 17:23:01 UTC - in response to Message 77413.  

> Here's my understanding, keeping in mind I am only a novice computer user:
>
> Boinc allocates resources depending on the ratio per project you specify. As
> it switches to each project it does the work unit that ends the soonest *for
> each individual project* and only when it's that project's turn to crunch.
>
> So, even if your Predictor work units end a week before your Seti ones, the
> Predictor ones will only get crunched when Boinc decides it's time to allocate
> time to Predictor based on the ratio you give.
>

Hi Matt,

Thanks for the reply.

Sadly my experience over the last few days does not quite bear out your view. Granted, it does seem to usually "switch" between projects (every 60 mins) in more or less the way it should, with one notable exception:

Running CPDN and Einstein (only), caused CPDN to continue crunching without pausing and left the Einstein WU's untouched. No matter of rebooting, or changing resource shares (to 98% Einstein vs. 2% CPDN) made any difference - leaving the 5 Einstein WU's just sitting there and not being crunched. Then the deadline passed.

regards,

Timbo
regards,
Tim
Founder, UK BOINC Team
Join us @ UK BOINC Team: http://www.ukboincteam.org.uk/newforum
ID: 77561 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 77510 - Posted: 8 Feb 2005, 12:20:58 UTC - in response to Message 77498.  

> Indeed centralising the ability to make any changes the preferences of
> the projects to which a user is attached to be a welcome improvement to
> BOINC's manageabilty.

I pointed out in a series of e-mails to the BOINC Developers list that with the clean separation of the BOINC GUI from the BOINC Daemon, coupled with the ability to suspend on project and work unit level, and the fact that the BOINC Manager can now contact other computers in the "local cluster" (farm) that one of the comupters can become a "local master" and regulate the entire farm node to also ensure better resource management and project allocation.

ID: 77510 · Report as offensive
Profile Daykay
Avatar

Send message
Joined: 18 Dec 00
Posts: 647
Credit: 739,559
RAC: 0
Australia
Message 77498 - Posted: 8 Feb 2005, 11:29:42 UTC - in response to Message 77437.  

> I have a related question: Why isn't resource share either on the boinc
> server(site) or controlled locally? (then the work request could automatically
> make the proper adjustments so that 3 days work is 3 days worth of work)
>
> Controlling all the projects at once would be nice. Instead of having to go
> to each project.
>

Indeed centralising the ability to make any changes the preferences of the projects to which a user is attached to be a welcome improvement to BOINC's manageabilty.
Kolch - Crunching for the BOINC@Australia team since July 2004.
Search for your own intelligence...
ID: 77498 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 29
Norway
Message 77491 - Posted: 8 Feb 2005, 9:22:37 UTC - in response to Message 77411.  

Each project runs on each own individual queue, there 1st wu downloaded is 1st crunched.
Switching between projects happens based on resource-share and the dept between them, the dept is checked each time a wu have finished or based on how often should switch between projects, default every 60 minutes.

Neither of these cares about deadlines, meaning a project with multiple deadlines will crunch the 7-day-wu before the later downloaded wu with only 1-day-deadline, and the client can happily switch to a CPDN-wu even was only 5 minutes left to crunch an Einstein@home-wu that passed it's deadline 1 hour ago...


There's plans to let the client temporarily boost individual projects resource-share so wu will not pass deadline, and not ask a low resource-share-project so often for work, but till this is implemented the changes in v4.6x expected released in a fortnight with manual pausing of projects/wu and removing individual wu should hopefully minimize most problems with the current implementation.
ID: 77491 · Report as offensive
Pascal, K G
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 2343
Credit: 150,491
RAC: 0
United States
Message 77481 - Posted: 8 Feb 2005, 7:50:01 UTC

Nope nothing in BOINC will do it, but you can help stop this problem by cutting your connect to time back to like 1 or 2 days......THey are working on the problem as we speak.....
Semper Eadem
So long Paul, it has been a hell of a ride.

Park your ego's, fire up the computers, Science YES, Credits No.
ID: 77481 · Report as offensive
Profile ghstwolf
Volunteer tester
Avatar

Send message
Joined: 14 Oct 04
Posts: 322
Credit: 55,806
RAC: 0
United States
Message 77437 - Posted: 8 Feb 2005, 4:23:31 UTC

I have a related question: Why isn't resource share either on the boinc server(site) or controlled locally? (then the work request could automatically make the proper adjustments so that 3 days work is 3 days worth of work)

Controlling all the projects at once would be nice. Instead of having to go to each project.


Still looking for something profound or inspirational to place here.
ID: 77437 · Report as offensive
Profile MattDavis
Volunteer tester
Avatar

Send message
Joined: 11 Nov 99
Posts: 919
Credit: 934,161
RAC: 0
United States
Message 77413 - Posted: 7 Feb 2005, 18:08:14 UTC

Here's my understanding, keeping in mind I am only a novice computer user:

Boinc allocates resources depending on the ratio per project you specify. As it switches to each project it does the work unit that ends the soonest *for each individual project* and only when it's that project's turn to crunch.

So, even if your Predictor work units end a week before your Seti ones, the Predictor ones will only get crunched when Boinc decides it's time to allocate time to Predictor based on the ratio you give.
-----
ID: 77413 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 154
Credit: 10,720,947
RAC: 822
United Kingdom
Message 77411 - Posted: 7 Feb 2005, 18:00:05 UTC - in response to Message 77369.  

> v4.1x takes resource_share and active_frac into account then asking for work,
>
> so the expected run-time for each project will be 3-6 days, but the cpu-time
> for each project will be their resource-share so total cached work for all
> projects will also be 3-6 days, not 24.

Hi Ingleside,

Question: Considering BOINC v4.x, how does BOINC resolve which WU's to process first and then which WU to process next.

Is it based on a history thing or what?

Reason I ask is as per my original question at the beginning of this thread - how come short "completion deadline" WU's do not get priority over much longer "completion deadline" WU's??

regards,

Timbo

regards,
Tim
Founder, UK BOINC Team
Join us @ UK BOINC Team: http://www.ukboincteam.org.uk/newforum
ID: 77411 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 29
Norway
Message 77369 - Posted: 7 Feb 2005, 14:05:08 UTC - in response to Message 77287.  

> First, with 4.19, I believe that the number in connect to the server every x
> days is the low mark.

Yes, v4.1x still asks for 2x the cache-setting.

> So you would be getting 3 - 6 days of work per project
> times 4 projects is 24 days worth of work. This is not a really good idea.
>

v4.1x takes resource_share and active_frac into account then asking for work,
so the expected run-time for each project will be 3-6 days, but the cpu-time for each project will be their resource-share so total cached work for all projects will also be 3-6 days, not 24.
Of course, CPDN-wu will still take longer to crunch. ;)


v4.5+ on the other hand doesn't act the same way, but a change to the scheduling-server earlier today should hopefully fix this.
ID: 77369 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 77287 - Posted: 7 Feb 2005, 3:46:14 UTC

First, with 4.19, I believe that the number in connect to the server every x days is the low mark. So you would be getting 3 - 6 days of work per project times 4 projects is 24 days worth of work. This is not a really good idea.

Second, 4.6x is doing a bit better about host venues (work/school/home).


BOINC WIKI
ID: 77287 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 154
Credit: 10,720,947
RAC: 822
United Kingdom
Message 77185 - Posted: 6 Feb 2005, 19:39:02 UTC - in response to Message 77149.  

>
> If you don’t care about the credits being on one account, create new accounts
> with a different email address and move the computers on dialup to that.

I'm already considering doing this, although I'd prefer to have the credits go to just the one account (keeps my team score up, even if the credits I would get from 2 different accounts would be the same...!).

regards,

Tim
regards,
Tim
Founder, UK BOINC Team
Join us @ UK BOINC Team: http://www.ukboincteam.org.uk/newforum
ID: 77185 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 154
Credit: 10,720,947
RAC: 822
United Kingdom
Message 77183 - Posted: 6 Feb 2005, 19:36:23 UTC - in response to Message 77141.  
Last modified: 6 Feb 2005, 19:36:52 UTC

> Why would the machines on dialup need a larger cache?

Because they are slower PC's and I only get to see them about once a week (they are some distance from me).


> Now, what I'd do is get one of the small DSL "routers" that has a modem-backup
> feature. If you never connect the broadband connection, then it'll be
> dialup-only, and your machines can easily share the dialup connection.


I'll look into that, but as the PC's are remote to where the ADSL facility is, then I'm not sure what you propose would be any better than what I currently use (a single dial-up PC, a small hub and FreeProxy software to link the PC's together when they want to up/download).

regards,

Timbo
regards,
Tim
Founder, UK BOINC Team
Join us @ UK BOINC Team: http://www.ukboincteam.org.uk/newforum
ID: 77183 · Report as offensive
Ziran
Volunteer tester
Avatar

Send message
Joined: 19 Mar 03
Posts: 32
Credit: 721
RAC: 0
Sweden
Message 77149 - Posted: 6 Feb 2005, 17:48:38 UTC
Last modified: 6 Feb 2005, 17:49:52 UTC

>or wait until BOINC get's it's act together and the home/work/school
>preferences start working correctly - are they still u/s?? - what I need to do
>is to set a different connection criteria for teh PC's on dial-up compared to
>the one on ADSL).

If you don’t care about the credits being on one account, create new accounts with a different email address and move the computers on dialup to that.
ID: 77149 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 77143 - Posted: 6 Feb 2005, 17:27:49 UTC - in response to Message 77141.  

I wish I had known about this type of router back when I was trying to share my dial-up connection. I didn't realize that there were any routers out there with dial-up back-up.

Jim

> Now, what I'd do is get one of the small DSL "routers" that has a modem-backup
> feature. If you never connect the broadband connection, then it'll be
> dialup-only, and your machines can easily share the dialup connection.
>
ID: 77143 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 77141 - Posted: 6 Feb 2005, 17:25:54 UTC - in response to Message 77107.  


> Thanks for that - trouble is this: 1 PC (the one in question) has the ADSL
> connection.
>
> All the others (which are a bit slower) are on dial-up. And so these need a
> larger cache to be stored locally (they upload/download about once a week -
> take about 2 hours to send the completed work and to download new WU's).

Why would the machines on dialup need a larger cache?

Sure a bigger cache means fewer calls, but a bigger cache also means more data when it does call. Add up the calls for a couple of weeks, and the off-hook time is the same -- more short calls or fewer, longer calls.

Now, what I'd do is get one of the small DSL "routers" that has a modem-backup feature. If you never connect the broadband connection, then it'll be dialup-only, and your machines can easily share the dialup connection.
ID: 77141 · Report as offensive
Profile Jim Baize
Volunteer tester

Send message
Joined: 6 May 00
Posts: 758
Credit: 149,536
RAC: 0
United States
Message 77129 - Posted: 6 Feb 2005, 17:03:47 UTC - in response to Message 77107.  

> Thanks for that - trouble is this: 1 PC (the one in question) has the ADSL
> connection.
>
> All the others (which are a bit slower) are on dial-up. And so these need a
> larger cache to be stored locally (they upload/download about once a week -
> take about 2 hours to send the completed work and to download new WU's).
>
Why not network all the computers together so that they share the ADSL connection?

Also, I have 3 or 4 projects running on my machines, but I also have my number set to connect to the sever every 0.5 days. To my knowlege I have not had a WU go over deadline.

Jim
ID: 77129 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 154
Credit: 10,720,947
RAC: 822
United Kingdom
Message 77107 - Posted: 6 Feb 2005, 15:45:42 UTC - in response to Message 77085.  
Last modified: 6 Feb 2005, 15:49:47 UTC

> This is what he meant by ”the number you set to connect to the server”
> This number determine how many WU’s that are cached on your computer. If you
> set it to 3, you will get 3 days of work stored on your computer. Now, BOINC
> doesn’t take in consideration your resource share for each project, so you
> will get 3 days of work stored fore each project. With 4 active projects you
> will get 12(in reality 6-12) days of work stored on your computer.

Thanks for that - trouble is this: 1 PC (the one in question) has the ADSL connection.

All the others (which are a bit slower) are on dial-up. And so these need a larger cache to be stored locally (they upload/download about once a week - take about 2 hours to send the completed work and to download new WU's).

So, the end result is a fast PC working on 4 current projects, but which BOINC is not allowing "soon to pass" deadline WU's to be crunched first.

And a small farm of PC's crunching for 2 projects each, which need to be fed just once a week (in a separate location - otherwise they'd all be on ADSL !!).

Guess I need to look again at how many projects each PC can best serve....

(or wait until BOINC get's it's act together and the home/work/school preferences start working correctly - are they still u/s?? - what I need to do is to set a different connection criteria for teh PC's on dial-up compared to the one on ADSL).

regards,

Timbo
regards,
Tim
Founder, UK BOINC Team
Join us @ UK BOINC Team: http://www.ukboincteam.org.uk/newforum
ID: 77107 · Report as offensive
Ziran
Volunteer tester
Avatar

Send message
Joined: 19 Mar 03
Posts: 32
Credit: 721
RAC: 0
Sweden
Message 77085 - Posted: 6 Feb 2005, 14:57:08 UTC - in response to Message 77069.  

At the moment BOINC doesn’t take deadlines in consideration then deciding witch WU to crunch. I think manual control of witch WU to crunch is somewhat implemented in BOINC 4.5x, witch is in Beta at the moment.


> 2) the number you set to connect to the server
>
> I don't call any number, as I have an 1Mb ADSL (always on) net connection.
>
> 3) This might help - my BOINC network connection is set to connect every 1-3
> days (but usually returns completed WU's straight away).

This is what he meant by ”the number you set to connect to the server”
This number determine how many WU’s that are cached on your computer. If you set it to 3, you will get 3 days of work stored on your computer. Now, BOINC doesn’t take in consideration your resource share for each project, so you will get 3 days of work stored fore each project. With 4 active projects you will get 12(in reality 6-12) days of work stored on your computer.

With 4 active projects (5 then LHC comes on line) how interested are you in having WU’s stored on your computer? If there is 4 active projects, the probability that all of them being down at the same time is very small, so all the cache is good for in your case is if your connection to the internet goes down for a longer period of time(but not to long to miss deadlines). Lowering the number will also shorten the turnaround time for the WU.

I would recommend that you set the “connect to server every” to something between 0.1 and 1.

Is there any downside to this that i have missed other then putting more stress on the scheduler?

ID: 77085 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : BOINC WU priority for projects


 
©2020 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.