Reasource Shareing Between Multiple Projects

Message boards : Number crunching : Reasource Shareing Between Multiple Projects
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187118 - Posted: 21 Jan 2012, 12:30:41 UTC

Does anyone have any links to information about how to set resource at project@preferences and what variables in BOINC that controls it and how they work?

I now have set up a small study of the APR and how it is affected when running multiple projects with different resource set.

Since adding Collatz as project number 6 made SETI totally stop from running.
I know want to figure out how to set the variables as discribed in project@preferences.

resource 100 on 2 tasks gives 50/50 to the projects it is said in the prefs page..

But I don't think it is that simple. It must be more variables to make use of.

links to documents that discribe how this work would be appriciated.



TRuEQ & TuVaLu
ID: 1187118 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34841
Credit: 261,360,520
RAC: 489
Australia
Message 1187123 - Posted: 21 Jan 2012, 12:50:01 UTC - in response to Message 1187118.  

Personally I set all my backup projects to 0% resources and leave SETI at 100% as it's my main/preferred project.

Also other projects with shorter reporting deadlines will tend to overwhelm SETI with its longer ones so I also have my backup projects set to NNT until I need to call on them and then I select whether I need CPU or GPU from them so I only get enough to keep them happy for a day or so.

Cheers.
ID: 1187123 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187172 - Posted: 21 Jan 2012, 16:25:40 UTC - in response to Message 1187123.  
Last modified: 21 Jan 2012, 16:26:43 UTC

Personally I set all my backup projects to 0% resources and leave SETI at 100% as it's my main/preferred project.

Also other projects with shorter reporting deadlines will tend to overwhelm SETI with its longer ones so I also have my backup projects set to NNT until I need to call on them and then I select whether I need CPU or GPU from them so I only get enough to keep them happy for a day or so.

Cheers.



Yep, that is also how i've done it.
Suspending projects and NNT them...

I don't really want to call it a backup project anymore....
Now I want to balance the projects it if possible.
To something like:

Running 1GPU with 6projects with different credits(I know.....credits).
There is 24 hours in 1 day

Then I would like to have for example SETI use 12hours and Milkyway 6hours and give the other 4 projects an equal share of what is left of that 24hour period.

Atleast I am trying here to figure out a way to come close to something like that.
TRuEQ & TuVaLu
ID: 1187172 · Report as offensive
Profile Link
Avatar

Send message
Joined: 18 Sep 03
Posts: 834
Credit: 1,807,369
RAC: 0
Germany
Message 1187213 - Posted: 21 Jan 2012, 18:59:23 UTC - in response to Message 1187172.  

Then I would like to have for example SETI use 12hours and Milkyway 6hours and give the other 4 projects an equal share of what is left of that 24hour period.

That would be:
SETI: 72
MW: 36
and 4x: 9
ID: 1187213 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1187219 - Posted: 21 Jan 2012, 19:18:41 UTC - in response to Message 1187172.  

Personally I set all my backup projects to 0% resources and leave SETI at 100% as it's my main/preferred project.

Also other projects with shorter reporting deadlines will tend to overwhelm SETI with its longer ones so I also have my backup projects set to NNT until I need to call on them and then I select whether I need CPU or GPU from them so I only get enough to keep them happy for a day or so.

Cheers.



Yep, that is also how i've done it.
Suspending projects and NNT them...

I don't really want to call it a backup project anymore....
Now I want to balance the projects it if possible.
To something like:

Running 1GPU with 6projects with different credits(I know.....credits).
There is 24 hours in 1 day

Then I would like to have for example SETI use 12hours and Milkyway 6hours and give the other 4 projects an equal share of what is left of that 24hour period.

Atleast I am trying here to figure out a way to come close to something like that.

Unfortunately BOINC doesn't function that way. It operates a debt system. So when one project is running it build up a debt for the other(s). If you had 4 projects with equal share. Then BOINC is not going run each project 6 hours a day. It may instead run 1 or 2 at a time and then switch to one, or more, of the others.

BOINC user Manual
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1187219 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187226 - Posted: 21 Jan 2012, 19:35:20 UTC
Last modified: 21 Jan 2012, 19:35:50 UTC

Thank you Hal9000

Unfortunately BOINC doesn't function that way. It operates a debt system. So when one project is running it build up a debt for the other(s). If you had 4 projects with equal share. Then BOINC is not going run each project 6 hours a day. It may instead run 1 or 2 at a time and then switch to one, or more, of the others.


That's what I see. 1 or 2 projects switching and then again 1 or 2 projects(different) and then back to the first 2 again.

Then I can't really do what I tried to do.

Thank you for verifying what I've seen.
TRuEQ & TuVaLu
ID: 1187226 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1187228 - Posted: 21 Jan 2012, 19:41:50 UTC - in response to Message 1187226.  

Thank you Hal9000

Unfortunately BOINC doesn't function that way. It operates a debt system. So when one project is running it build up a debt for the other(s). If you had 4 projects with equal share. Then BOINC is not going run each project 6 hours a day. It may instead run 1 or 2 at a time and then switch to one, or more, of the others.


That's what I see. 1 or 2 projects switching and then again 1 or 2 projects(different) and then back to the first 2 again.

Then I can't really do what I tried to do.

Thank you for verifying what I've seen.

It is a function of design. Which on a faster machine seems like it should be able to work in the mode you are wanting.

Since the make BOINC to work with as many computers and projects as possible the current mode is better for that scenario.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1187228 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187231 - Posted: 21 Jan 2012, 19:49:25 UTC - in response to Message 1187228.  

Thank you Hal9000

Unfortunately BOINC doesn't function that way. It operates a debt system. So when one project is running it build up a debt for the other(s). If you had 4 projects with equal share. Then BOINC is not going run each project 6 hours a day. It may instead run 1 or 2 at a time and then switch to one, or more, of the others.


That's what I see. 1 or 2 projects switching and then again 1 or 2 projects(different) and then back to the first 2 again.

Then I can't really do what I tried to do.

Thank you for verifying what I've seen.

It is a function of design. Which on a faster machine seems like it should be able to work in the mode you are wanting.

Since the make BOINC to work with as many computers and projects as possible the current mode is better for that scenario.



Where is the "debt" stored for the projects?
Is there anyway I can see the number and how it is changed for the differnt projects I am running?

TRuEQ & TuVaLu
ID: 1187231 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1187267 - Posted: 21 Jan 2012, 22:06:48 UTC - in response to Message 1187231.  

Thank you Hal9000

Unfortunately BOINC doesn't function that way. It operates a debt system. So when one project is running it build up a debt for the other(s). If you had 4 projects with equal share. Then BOINC is not going run each project 6 hours a day. It may instead run 1 or 2 at a time and then switch to one, or more, of the others.


That's what I see. 1 or 2 projects switching and then again 1 or 2 projects(different) and then back to the first 2 again.

Then I can't really do what I tried to do.

Thank you for verifying what I've seen.

It is a function of design. Which on a faster machine seems like it should be able to work in the mode you are wanting.

Since the make BOINC to work with as many computers and projects as possible the current mode is better for that scenario.



Where is the "debt" stored for the projects?
Is there anyway I can see the number and how it is changed for the different projects I am running?

Like most other information the vales are stored in your client_state.xml.

IIRC to see the values in the GUI you select a project, from the projects tab, and select Properties. Then under Scheduling you will see
CPU scheduling priority: 0.0 [This is the CPU STD(Short Term Debt)]
CPU work fetch priority: 0.0 [This is the CPU LTD(Long Term Debt)]
GPU scheduling priority: 0.0 [This is the GPU STD(Short Term Debt)]
GPU work fetch priority: 0.0 [This is the GPU LTD(Long Term Debt)]

You can compare them to the values in your client_state.xml for verification. If the numbers are 0 then you have no debt for that project/device. Otherwise the numbers will be negative or positive.

Your debt, work queue size, and connection setting all play a role in how BOINC decides to download and process tasks.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1187267 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187290 - Posted: 21 Jan 2012, 22:49:47 UTC

IIRC to see the values in the GUI you select a project, from the projects tab, and select Properties. Then under Scheduling you will see
CPU scheduling priority: 0.0 [This is the CPU STD(Short Term Debt)]
CPU work fetch priority: 0.0 [This is the CPU LTD(Long Term Debt)]
GPU scheduling priority: 0.0 [This is the GPU STD(Short Term Debt)]
GPU work fetch priority: 0.0 [This is the GPU LTD(Long Term Debt)]


There the numbers where. Again thank you.

It looks a bit different in 7.0.11

I have under "scheduling" something called scheduling priority that is a different value in each project.

It looks to me that it is the project with the lowest number -0.57 that is runing.

Then it gets more clear to me which project that will be about next to run.
It is SETI that has been waiting to run for about 2 days now.

Thank you very much.
TRuEQ & TuVaLu
ID: 1187290 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1187297 - Posted: 21 Jan 2012, 23:02:34 UTC - in response to Message 1187290.  
Last modified: 21 Jan 2012, 23:03:29 UTC

It looks a bit different in 7.0.11

I have under "scheduling" something called scheduling priority that is a different value in each project.

It looks to me that it is the project with the lowest number -0.57 that is runing.

Then it gets more clear to me which project that will be about next to run.
It is SETI that has been waiting to run for about 2 days now.

That's because Boinc 7.0.x uses a completely new method of calculating debt, and has a brand new scheduler too, there's not even a wiki entry on how it works yet,

Claggy
ID: 1187297 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187378 - Posted: 22 Jan 2012, 8:21:59 UTC - in response to Message 1187297.  

It looks a bit different in 7.0.11

I have under "scheduling" something called scheduling priority that is a different value in each project.

It looks to me that it is the project with the lowest number -0.57 that is runing.

Then it gets more clear to me which project that will be about next to run.
It is SETI that has been waiting to run for about 2 days now.

That's because Boinc 7.0.x uses a completely new method of calculating debt, and has a brand new scheduler too, there's not even a wiki entry on how it works yet,

Claggy


Ok.

No SETI task yet ran. 3days of runtime.

It feel like there is a "wanted to be run" factor from 1 or 2 projects when I run 6 at a time on the scheduler...

They have a deadline March.... Maybe that's why.
If that is part of the calculation then it doesn't seem to matter what resource share I set. :(
TRuEQ & TuVaLu
ID: 1187378 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187385 - Posted: 22 Jan 2012, 9:22:32 UTC
Last modified: 22 Jan 2012, 9:24:12 UTC

I think it would be a neat feature where the BM user could use BM-managing and a sort of relative "Time-sharing(24h-interval)" option in BM to schedule task run times.

But that would probebly lead to some of the tasks getting aborted due to deadline.
Since some projects sends 100 tasks at a time and other between 1-60 tasks.
But I don't think that would be a problem for the user since BM will do it's work based on the "time-sharing(24h-interval)".

And it will let the user control how BM would run the scheduling.
I dunno if it would be possible.....As always from me, "it's just a thought".
TRuEQ & TuVaLu
ID: 1187385 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187392 - Posted: 22 Jan 2012, 10:21:59 UTC

And it looks like I got fooled....
In BM 7.0.11 it seems to be the highest scheduling number that will run next.
I think I said the opposite before.....

I have 4 projects tha is building "debt" higher then SETI for 3 days now....
They are at aprox -0.4 each and seti is at -0.56 and albert that hasn't been running for 3 days is at -2.22

I wonder how long it will take for the counters to reach albert here...
At current rate I would guess 2-4 weeks.


TRuEQ & TuVaLu
ID: 1187392 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187476 - Posted: 22 Jan 2012, 18:47:30 UTC

Project preferences in BM 7.x.x observations here.


Still 4 projects with aprox -0.4
Albert is -1.87 an improvement
And SETI is -0.46 also an improvement. No tasks ran yet.

Maybe the projects catch up with SETI tomorrow sometime.
TRuEQ & TuVaLu
ID: 1187476 · Report as offensive
Wembley
Volunteer tester
Avatar

Send message
Joined: 16 Sep 09
Posts: 429
Credit: 1,844,293
RAC: 0
United States
Message 1187506 - Posted: 22 Jan 2012, 20:55:59 UTC

The more projects you run, the smaller your cache should be. It allows BOINC to react better/faster to debt fluctuations.

ID: 1187506 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187525 - Posted: 22 Jan 2012, 22:11:31 UTC - in response to Message 1187506.  

The more projects you run, the smaller your cache should be. It allows BOINC to react better/faster to debt fluctuations.


I have a cache of 2days.
TRuEQ & TuVaLu
ID: 1187525 · Report as offensive
LadyL
Volunteer tester
Avatar

Send message
Joined: 14 Sep 11
Posts: 1679
Credit: 5,230,097
RAC: 0
Message 1187639 - Posted: 23 Jan 2012, 10:38:39 UTC

I my experience, Boinc 7 is much better at honouring resource share than boinc 6 was.

easiest way for a quick overview of debt is to run one cycle of work_fetch_debug logging.

Workfetch will be initiated when the calculated remaining work drops below the 'connect every' setting and will then ask for as much work as specified in the 'aditional work' setting. It will ask from the project for which the debt entry is the smallest (calculated as work done recently compared to resource share). If no work is available from that project it will ask from the next larger and so on until it gets work.
With SETI this has the usual problem of it being luck whether you get work on a request, so as with boinc 6 you end up getting more work from the other projects. SETI eventually rises in priority until you get lucky.

With BOINC 7 cache needs to be set quite differently from the approach used with previous versions to get the same effect.
Because it will wait until the 'connect every' minimum level is reached before it asks for work (instead of the frequent top ups with previous) if you want to be sure to have work for 3 days available at all times on the machine you need to set 'connect every' to 3. If you want it to ask often for work (to get that elusive SETI task) you should set 'additional' to a small value.
ID: 1187639 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187689 - Posted: 23 Jan 2012, 17:11:36 UTC - in response to Message 1187639.  

I my experience, Boinc 7 is much better at honouring resource share than boinc 6 was.

easiest way for a quick overview of debt is to run one cycle of work_fetch_debug logging.

Workfetch will be initiated when the calculated remaining work drops below the 'connect every' setting and will then ask for as much work as specified in the 'aditional work' setting. It will ask from the project for which the debt entry is the smallest (calculated as work done recently compared to resource share). If no work is available from that project it will ask from the next larger and so on until it gets work.
With SETI this has the usual problem of it being luck whether you get work on a request, so as with boinc 6 you end up getting more work from the other projects. SETI eventually rises in priority until you get lucky.

With BOINC 7 cache needs to be set quite differently from the approach used with previous versions to get the same effect.
Because it will wait until the 'connect every' minimum level is reached before it asks for work (instead of the frequent top ups with previous) if you want to be sure to have work for 3 days available at all times on the machine you need to set 'connect every' to 3. If you want it to ask often for work (to get that elusive SETI task) you should set 'additional' to a small value.


my scheduling priority number for projects that runs seems to be between -0.4 -- -0.5 SETI went below -0.46 today and did 6 tasks and now it is running from time to time. Now all I wait for is Albert that has -2.1 to start to run with the other projects that runs when they are higher then -0.46(aprox).

I am happy to see my SETI caught up in the debt calculation.

Is there any fast way to reset the number -2.1 to get my albert to join the other projects instead of the long time waiting?

And yes I think this way feels ok for running alot of projects. A bit slow with my SETI catching up and now Albert too.
Collatz, Milkyway, Primegrid that I run rotates nicely and i think SETI will now as well.
TRuEQ & TuVaLu
ID: 1187689 · Report as offensive
Profile TRuEQ & TuVaLu
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 505
Credit: 69,523,653
RAC: 10
Sweden
Message 1187690 - Posted: 23 Jan 2012, 17:16:48 UTC - in response to Message 1187639.  


Because it will wait until the 'connect every' minimum level is reached before it asks for work (instead of the frequent top ups with previous) if you want to be sure to have work for 3 days available at all times on the machine you need to set 'connect every' to 3. If you want it to ask often for work (to get that elusive SETI task) you should set 'additional' to a small value.


I have it set to 0 (always connected).


easiest way for a quick overview of debt is to run one cycle of work_fetch_debug logging.


How do I run 1 cycle of work_fetch_debug logging?
TRuEQ & TuVaLu
ID: 1187690 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Reasource Shareing Between Multiple Projects


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.