Resource Share

Message boards : Number crunching : Resource Share
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Ace Casino
Avatar

Send message
Joined: 5 Feb 03
Posts: 285
Credit: 29,750,804
RAC: 15
United States
Message 1429687 - Posted: 17 Oct 2013, 14:07:30 UTC

Does Resource Share work between SETI and Einstein?

If no, that’s all I needed to know.

If yes, I’ll give more detail.

Thanks
ID: 1429687 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1429699 - Posted: 17 Oct 2013, 14:35:17 UTC - in response to Message 1429687.  

Yes, that's what it's there for. Between SETI and Einstein, or between any number of projects.
ID: 1429699 · Report as offensive
Profile skildude
Avatar

Send message
Joined: 4 Oct 00
Posts: 9541
Credit: 50,759,529
RAC: 60
Yemen
Message 1429702 - Posted: 17 Oct 2013, 14:45:59 UTC

several things happen that cause work imbalances.

1) a project sets very short TAT for work thus making their work high priority

2) You have a work imbalance and BOINC is attempting to set your workload up to match a long term view of work being done. If one project appears to be low BOINC will do more of it until it see what it thinks is a balance.

3) some projects tend to over send work. IIRC Einstein and a couple of the Protein project has done this to me. I've ran the Einstein Project and had so much work that I couldn'd complete all of tha work and the work I already had on board. Einstein didn't respect the 3-4 extra work and instead filled my cache with 10-20 days worth of work.


In a rich man's house there is no place to spit but his face.
Diogenes Of Sinope
ID: 1429702 · Report as offensive
Profile Ace Casino
Avatar

Send message
Joined: 5 Feb 03
Posts: 285
Credit: 29,750,804
RAC: 15
United States
Message 1429706 - Posted: 17 Oct 2013, 14:58:52 UTC

Richard,
I’ve been trying for almost 5 months now to change the resource share of SETI and Einstein. I have several computers and I keep increasing the resource share of SETI and decreasing the share on Einstein, it’s not working.

Resetting projects doesn’t seem to work. I lowered cache size on projects. Just to give you an idea where I am now on resource share: On my i7 computer I have SETI 10,000 v Einstein 30 share. It’s been like this for a few weeks and no change. There is no significant ‘scheduling priority’ deficit for either project.

It's not like I just changed the settings and I'm impatient. 5 months now (might be longer) with zero change.

Thanks
ID: 1429706 · Report as offensive
Profile William
Volunteer tester
Avatar

Send message
Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1429709 - Posted: 17 Oct 2013, 15:18:07 UTC

What do you want to achieve and why do you say it's not working?
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1429709 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1429731 - Posted: 17 Oct 2013, 16:12:49 UTC - in response to Message 1429706.  

Richard,
I’ve been trying for almost 5 months now to change the resource share of SETI and Einstein. I have several computers and I keep increasing the resource share of SETI and decreasing the share on Einstein, it’s not working.

Resetting projects doesn’t seem to work. I lowered cache size on projects. Just to give you an idea where I am now on resource share: On my i7 computer I have SETI 10,000 v Einstein 30 share. It’s been like this for a few weeks and no change. There is no significant ‘scheduling priority’ deficit for either project.

It's not like I just changed the settings and I'm impatient. 5 months now (might be longer) with zero change.

Thanks

Stop being so Damn Greedy, set a Small cache size of only a day or two.

At present Seti has a 100 CPU and 100 GPU limit of tasks in progress, that is only a day or so of work for a fast host, all your hosts are sat on their 200 task limit,
Boinc will fill up with work with Seti which will be enough to keep your hosts in work for quite a few days, then it'll fill up from work from other projects,
say you have your cache set to 10 days, you might get enough work from Seti for only 3 days, Boinc will then go and get 7 days work from Einstein to fill your cache,
Resource share will never work in those circumstances.

Claggy
ID: 1429731 · Report as offensive
Profile Ace Casino
Avatar

Send message
Joined: 5 Feb 03
Posts: 285
Credit: 29,750,804
RAC: 15
United States
Message 1429744 - Posted: 17 Oct 2013, 16:30:13 UTC - in response to Message 1429731.  
Last modified: 17 Oct 2013, 16:33:18 UTC

Here's the definition of Resource Share:

Resource share
Determines the proportion of your computer's resources allocated to this project. Example: if you participate in two BOINC projects with resource shares of 100 and 200, the first will get 1/3 of your resources and the second will get 2/3.

It does not talk about Greed.

I pretty much knew the reason it wasn't working is the WU limits of SETI, and the 45 day vs. 10 deadlines of Einstein.

Resource share has always been a weak link with BOINC.

I have 1 computer that does Einstein and Rosetta, resource share works ok.

So, does it work or not with Einstein and SETI?
ID: 1429744 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1429753 - Posted: 17 Oct 2013, 16:55:29 UTC - in response to Message 1429744.  
Last modified: 17 Oct 2013, 16:58:53 UTC

Which part of, Seti has of a limit of 100 CPU and 100 GPU tasks in progress didn't you understand?

With your resource share of 10,000 to 30, Seti needs 333 tasks per one Einstein task, that can never happen at present,

Setting a cache size higher than what is needed to get those 100 CPU and 100 GPU tasks will mean you fill up with work from your other projects. and because of this, Resource Share doesn't matter, Full Stop.
(But eventually Boinc will say Einstein isn't the highest priority and refuse to ask for work)

Claggy
ID: 1429753 · Report as offensive
Profile Ace Casino
Avatar

Send message
Joined: 5 Feb 03
Posts: 285
Credit: 29,750,804
RAC: 15
United States
Message 1429799 - Posted: 17 Oct 2013, 18:22:30 UTC

I'm within the standards of BOINC, it's Resource Share that is flawed, just as I suspected.

ID: 1429799 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1429814 - Posted: 17 Oct 2013, 19:29:16 UTC

Resourse Share as defined doesn't work in BOINC.Period.
There are some SPECIAL configurations (just as Claggy proposed) where Resourse Share could work almost as defined. But in general - it doesn't work.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1429814 · Report as offensive
Profile Fred E.
Volunteer tester

Send message
Joined: 22 Jul 99
Posts: 768
Credit: 24,140,697
RAC: 0
United States
Message 1429819 - Posted: 17 Oct 2013, 19:37:39 UTC

I'm within the standards of BOINC, it's Resource Share that is flawed, just as I suspected.

I've also had problems with BOINC v7 resource share. Here's 3 thoughts that have helped me:

1) You can add this option to the options section of cc_config.xml:

<rec_half_life_days>N</rec_half_life_days>
A project's scheduling priority is determined by its estimated credit in the last X days. Default is 10; set it larger if you run long high-priority tasks. New in 6.13.1

I find that with a value for N of one day. BOINC 7 still runs wild for a day and then starts to settle down. Better than waiting 10 days after changing resource share settings. I don't know much about this option, just that it helps me.

2) I run Einstein on No New Tasks so I can use a higher cache setting for SETI. About every other day I reduce the cache setting and allow work fetch for Einstein or other projects. Then back to no new tasks and a cache setting suitable for SETI. But you have more computers so this may be unattractive to you. The alternative is to run at a low cache setting.

3). I use a max concurrent setting in app_config.xml to limit the number of concurrent tasks for an application. Works fairly well at Einstein, but the design uses the app's short name. That doesn't work so well when gpu and cpu apps have the same short name, but I do use it for Astropulse due to heat problems. If you're not familiar with app_config.xml, it's a plain text file like cc_config.xml and it goes in the project's data directory. This will work only on your hosts with 7.0.64, it wasn't in earlier versions. There's a new menu option to read in on the fly so you don't have to restart BOINC.

If interested, many of us can provide more how-to, examples, etc.
Another Fred
Support SETI@home when you search the Web with GoodSearch or shop online with GoodShop.
ID: 1429819 · Report as offensive
Profile William
Volunteer tester
Avatar

Send message
Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1430164 - Posted: 18 Oct 2013, 10:56:30 UTC

With boinc 7 if you set your cache small, resource share works out well enough on the multi project systems I know.

Ok you do have a point. David knows nothing about stabilising systems - so workfetch and with that resource share reacts in a fairly chaotic way to any perturbation. A perturbation in this context is any 'can't get enough work from the project I'd like work from now'.

Some projects always have plenty of work on tap. Rosetta and Einstein are examples.
Some projects only have work once in a while like LHC or SIMAP.
Some projects go through dry times or have other limits in place - GPUGrid, SETI.

Whenever you add a project to the mix that cannot supply as much as is demanded when it is demanded will upset the balance [perturbation].

With LHC, when a new batch of work is released, BOINC invariably plays catchup.

With SETI when you have cache settings higher than the current limit allows, you will make lots of 'limit reached - no tasks sent' requests and boinc will fetch work from the next project. It will also go into backoff. It will always try to fetch from the highest project but if thre's nothing or not enough to be had you will get more from the other projects than you would like to see.

Drop to a sensible small cache (with a fast machine that might be VERY small) and you should see that crunching times equalise over time. And unless you make the REC halflife adjustment Fred E. mentions it will take quite some time before RECs and priorities equalise.

I can look at work_fetch_debug output if desired.

Else, go on squabbling, I don't care any more. I'm already trying to get credit sorted.

NB boinc is far from bug free. But you can make life for yourself even more difficult if you try to fight the system.
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1430164 · Report as offensive

Message boards : Number crunching : Resource Share


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.