I am actually going to do all the Einstein tasks on my computer before I allow Einstein to download more work. I'm not working with a large cache only 0.2 + 1.5 days, but LTD is -414484.481867.
All other methods I have tried either leave the LTD static or if I let it run automatically on any sort of "sensible" cache settings, it just keeps going up and up.

I'm sure this has been hashed several times here but I can't get my arms around it.

I have 7 computers each with increasing long term debts building. The back-up project doesn't get much work, but it does get a bit and keeps pushing my LTD scores up. Most of the computers have a debt in the millions. I have a resource share set at 0.1% on the backup project, connect continuously to the net, and have a 1 d cache as default.

Boinc seems incapable of digging out of this hole. I have disabled new work on a couple of the slowest computers, but that didn't matter.

The back-up project will occasionally sync with its home-base, but in almost all cases it is denied new work due to the existing local seti queue. So I would guess I'm sending about 1 result back per month per cpu to the back-up project ( the one with an increasing LTD ).

Any ideas? Does boinc balance LTD only if the imbalance is small? It seems that something is broke.

Oh yes, I am running boinc 5.10.45 on some and 6.? on a couple. So I don't think that is the 'problem' here.

The behavior of 5.x was to not allow work fetch from a project if it had deadline trouble. This restriction is gone in 6.x.

Try upgrading to 6.2.19 (or later) and the problem should slowly correct itself.

---

BOINC WIKI

@'WaveMaker'
When you are first starting out with unbalanced resource shares the client will download up to a full queue of work from the backup project. The client may also get a full queue of work any time it has to contact the backup project. Once that work is processed you usually will not see that project come up again for a long time unless the main project(s) have problems sending work. So as others have mentioned shorter queues generally work better.

---

BOINC WIKI

BOINCing since 2002/12/8

I have pretty much zeroed my cache on the quad and on the duo. The quad (of course) had trouble getting connected to seti last night and so downloaded more einstein wu's and started them. Not what I want, of course. The quad is running boinc 5. Since then it has downloaded a couple seti wu's and started them. So right now I have 4 seti wu's and 4 einstein wu's, all running or waiting to run (having started over the night at some point). This is with a 0/0 setting, and the LTD has gotten even bigger. I think I will not wait too much longer to see what happens, before changing my ci/co settings.

However, my duo is running 6.2.18. It also cleared its queue. but hasn't downloaded more einsteins. Because the duo it is 50% slower per wu than the quad, I don't have a lot of download requests to examine yet. So the 0/0 setting with 6.2 remains viable for now. I'll watch the LTD to see if it starts to improve before changing anything.

The other computers are slower and I won't bother concluding anything about them yet at all.

Hi, glad somebody started about this, as I never really understood(also never really looked into it ;)).
I have 5 hosts, 3 QUADs and 2 C2D. My back-Up projects are EINSTEIN and LHC, I don't wanted to be difficult as well, so I divided it 1(Einstein/LHC on 3(SETI).
So, on my quads, 3 cores always run SETI and 1 core EINSTEIN(on 2QUADs)and 1core LHC (VISTA).
Only the C2D has only SETI and theP4Dual(Smithfield)EINSTEIN.
When I look at the real recource share, it's completely different!
LHC has no work, all the time, EINSTEIN does.
When I look @ my credits, it's a different story, SETI has 10x the credit off EINSTEIN and LHC, about 1/100, compaired to SETI.
This is strange.
Where do you see your Long Term Debt? Is it listed along with your credits?

---

I have pretty much zeroed my cache on the quad and on the duo. The quad (of course) had trouble getting connected to seti last night and so downloaded more einstein wu's and started them. Not what I want, of course. The quad is running boinc 5. Since then it has downloaded a couple seti wu's and started them. So right now I have 4 seti wu's and 4 einstein wu's, all running or waiting to run (having started over the night at some point). This is with a 0/0 setting, and the LTD has gotten even bigger. I think I will not wait too much longer to see what happens, before changing my ci/co settings.

However, my duo is running 6.2.18. It also cleared its queue. but hasn't downloaded more einsteins. Because the duo it is 50% slower per wu than the quad, I don't have a lot of download requests to examine yet. So the 0/0 setting with 6.2 remains viable for now. I'll watch the LTD to see if it starts to improve before changing anything.

The other computers are slower and I won't bother concluding anything about them yet at all.

Well, you really hit the crux of the biscuit with your first observation here.

When I was testing with SAH as the primary (RS biased to SAH 99.96%) and a three day cache the best actual time split my T2400 could pull was around 90% for SAH, and resulted from the weekly outage. This was in spite of the fact there was more than enough work for SAH to cover the outage and the host can contact the project any time it wants to.

The slower hosts did progressively worse in terms of actual time split. This is due mostly to the fact the low share forced them to run the 'backup' tasks in EDF once they got into deadline trouble. For them, SAH was getting typically 75% of the uptime or less, and thus the constant increase in magnitude for LTD was inevitable.

So what it really boils down to is with the way the current CC fetch policies work when you are running high bias shares, the actual split you get is almost solely dependent on project availability for the primary project. Running the CI/CO down helps some for the fast hosts, because this tends to 'tighten up' the amount of free slack available for the backups. However, you just can't get around the fact if the backup is empty and the primary doesn't respond to the fetch for any reason, you will draw at least one from a backup.

This means that even fast hosts may have some trouble maintaining RS in the short term, especially if you don't gate off Tuesdays in order to avoid a predictable backup fetch. They should be able to get fairly close to your set share in the long term however.

For slower hosts though, I'm pretty well convinced they will not be able to hold the share, short or long term, as it stands without manual intervention on a regular basis.

One other thing which should be kept in mind here, the other main reason these policy changes were made was to better accommodate the folks who really do only connect to the network periodically, and I'm talking in terms of days between opportunities here. If you really do connect very infrequently, then the backup when only absolutely necessary policy means you stand a good chance of running out of work completely before the next connection.

However, I think it would be better to have two completely separate policy sets for intermittent and always available connections. The real problem is that the needs and characteristics of the two are different, and not completely compatible with each other.

Alinator

Where do you see your Long Term Debt? Is it listed along with your credits?

there is a client_state.xml file in your boinc directory. make a copy and open the copy. in there you will see the ltd and std and a whole lot of other gibberish grouped by project. that number is changing in real time, so make the copy so that you don't interfere with some other process.

boinclogx will show you as well, if you install it, using 'show overall view'

Hi, glad somebody started about this, as I never really understood(also never really looked into it ;)).
I have 5 hosts, 3 QUADs and 2 C2D. My back-Up projects are EINSTEIN and LHC, I don't wanted to be difficult as well, so I divided it 1(Einstein/LHC on 3(SETI).
So, on my quads, 3 cores always run SETI and 1 core EINSTEIN(on 2QUADs)and 1core LHC (VISTA).
Only the C2D has only SETI and theP4Dual(Smithfield)EINSTEIN.
When I look at the real recource share, it's completely different!
LHC has no work, all the time, EINSTEIN does.
When I look @ my credits, it's a different story, SETI has 10x the credit off EINSTEIN and LHC, about 1/100, compaired to SETI.
This is strange.
Where do you see your Long Term Debt? Is it listed along with your credits?

There is no read out or summary report of STD or LTD provided in any standard distribution of BOINC from Berkeley.

So you have to either look at it in the client_state file, or use a third party tool to see it 'real time'. BOINCView probably gives the best overall picture of what's going on if you have more than one host to monitor.

One other thing to keep in mind when looking at these issues is that you cannot use credit directly as an indicator of holding your RS, since our old friend CPP rears its ugly head again.

The total CPU time used per project is the only accurate gauge for this purpose.

Alinator

I couldn't wait and have to go out, so I set my quad to ci/co = 0.0833/0.0833 (2h/2h). I figure 2h is the time to complete 2 normal wu's on a single core. I picked 2+2h because a 'typical' Tuesday outage is 4h. Perhaps I'm being too agressive. (Perhaps these ci/co settings should be in wu's?)

After making the change, boinc immediately downloaded 18 new wu's for me. But I still have at least 8h to see the 4 einstein's that started to get purged (actually I suspect they will be there until about 11/6 which is their deadline.)

I have pretty much zeroed my cache on the quad and on the duo. The quad (of course) had trouble getting connected to seti last night and so downloaded more einstein wu's and started them. Not what I want, of course.

I'm confused.

If SETI was unreachable and BOINC needed work, it should have downloaded Einstein, shouldn't it?

... or is there something here that I'm missing -- like maybe a large cache setting?

There are two goals here:

1) Don't run out of work.

2) Follow resource share.

Those aren't always compatible. #1 takes priority, with the assumption that BOINC can get back to #2 at some later date.

---

I couldn't wait and have to go out, so I set my quad to ci/co = 0.0833/0.0833 (2h/2h). I figure 2h is the time to complete 2 normal wu's on a single core. I picked 2+2h because a 'typical' Tuesday outage is 4h. Perhaps I'm being too agressive. (Perhaps these ci/co settings should be in wu's?)

I think I see part of the problem.

What BOINC is trying to do is honor your resource share averaged over the long term.

What should happen on Tuesdays:

1) Run out of SETI

2) Try to get more

3) SETI down, so get Einstein and crunch

4) Accumulate debt while crunching extra Einstein

5) After the outage is over, finish up Einstein and crunch SETI to pay back the debt.

In other words, you should see several hours of Einstein-only, followed by several hours of SETI-only.

I'm saying this because you're setting CI and CO based on the typical Tuesday maintenance outage, and I don't think that's a consideration.

Also, a BOINC client that has accumulated excessive "Einstein" debt should only download Einstein when it can't get SETI. I'll defer to JM7 on exactly when that should happen -- I always thought letting the cache go dry before doing something to increase debt was the right thing to do.

---

I think I see part of the problem.

What BOINC is trying to do is honor your resource share averaged over the long term.

What should happen on Tuesdays:

1) Run out of SETI

2) Try to get more

3) SETI down, so get Einstein and crunch

4) Accumulate debt while crunching extra Einstein

5) After the outage is over, finish up Einstein and crunch SETI to pay back the debt.

In other words, you should see several hours of Einstein-only, followed by several hours of SETI-only.

I'm saying this because you're setting CI and CO based on the typical Tuesday maintenance outage, and I don't think that's a consideration.

Also, a BOINC client that has accumulated excessive "Einstein" debt should only download Einstein when it can't get SETI. I'll defer to JM7 on exactly when that should happen -- I always thought letting the cache go dry before doing something to increase debt was the right thing to do.

Except, that's not what happens.

First, it's not a matter of running out of SAH, it's a matter of merely having 'room' for another SAH task when the project is not available.

Second, the CC won't go to crunching EAH directly and then get back to SAH immediately, since EAH is not a tight deadline project for most late model hosts. Therefore, the EAH task ends up sitting in the cache mostly until it starts approaching the EDF gate, since its time slice allocation is based on RS as well.

Furthermore, LTD gets added to the projects multiplied by the RS factor. IOW, if you are running a 10 to 1 split, then every hour run for the backup counts as plus 10 for the primary in the LTD column. Needless to say debt builds quickly when running the backup, and that's the main reason you can see the almost monotonic debt increase with the high bias scenario.

Alinator

I think I see part of the problem.

What BOINC is trying to do is honor your resource share averaged over the long term.

What should happen on Tuesdays:

1) Run out of SETI

2) Try to get more

3) SETI down, so get Einstein and crunch

4) Accumulate debt while crunching extra Einstein

5) After the outage is over, finish up Einstein and crunch SETI to pay back the debt.

In other words, you should see several hours of Einstein-only, followed by several hours of SETI-only.

I'm saying this because you're setting CI and CO based on the typical Tuesday maintenance outage, and I don't think that's a consideration.

Also, a BOINC client that has accumulated excessive "Einstein" debt should only download Einstein when it can't get SETI. I'll defer to JM7 on exactly when that should happen -- I always thought letting the cache go dry before doing something to increase debt was the right thing to do.

Except, that's not what happens.

First, it's not a matter of running out of SAH, it's a matter of merely having 'room' for another SAH task when the project is not available.

Second, the CC won't go to crunching EAH directly and then get back to SAH immediately, since EAH is not a tight deadline project for most late model hosts. Therefore, the EAH task ends up sitting in the cache mostly until it starts approaching the EDF gate, since its time slice allocation is based on RS as well.

Furthermore, LTD gets added to the projects multiplied by the RS factor. IOW, if you are running a 10 to 1 split, then every hour run for the backup counts as plus 10 for the primary in the LTD column. Needless to say debt builds quickly when running the backup, and that's the main reason you can see the almost monotonic debt increase with the high bias scenario.

Alinator

To which I have two comments:

1) JM7 says that these issues are fixed in 6.2.19, and we should be careful that we're talking about the right version.

2) I run three projects with a 1000:1 split across them, and I don't see it.

I have absolutely no doubt that SAH and EAH won't divide equally once they're queued. As I understand it, short-term scheduling deals mainly with getting work that is already downloaded out of your queue and not missing deadlines. It is controlled by short term debt, and the simulator. Resource share is a factor as long as there is no deadline pressure.

Long Term Debt has more to do with adding work to the queue, and again, it isn't the only factor that decides how much, and what, BOINC will add to the queue.

I think we also have a micro-view of what is intentionally a macro problem. If long-term debts are huge, BOINC should not have a mix of work because it's trying to correct an imbalance. If schedules are tight, we should see debts increase since they're the mechanism that corrects for this later.

---

It sounds like Alinator understands this the best. I think we agree that boinc will simply not correct itself for a high rs imbalance and when seti is the primary project (with weekly planned outages).

It occurs to me that the essence of the issue I'm experiencing (on 5.x and 6.x clients alike) has to do with the requirement that the cache is kept full. That is, in a sense, silly. What should be required is that the cpu's are kept busy. So that means that the cache fill can droop for extended periods of time without penalty. For example, why not wait until the cache drops to 75% or 50% full before downloading 'emergency' wu's to fill it up. The number of wu's to download could easily be made via some sort of dynamic decision algorithm, similar to the scheme used to download or upload results when there are network connection issues.

What I don't like is to force a lot of fudging of parameters and manual interventions. A smarter front end would/should minimize this need and keep the number crunchers (this message board) happy as well.

It sounds like Alinator understands this the best. I think we agree that boinc will simply not correct itself for a high rs imbalance and when seti is the primary project (with weekly planned outages).

If you and Alinator are going to agree, then I guess I'm wasting my time trying to present a different viewpoint.

My resource share imbalance is 1000:1, which seems fairly high. I don't carry a very small cache, or a "maximum" cache either.

I don't see the issue, and Alinator does, so I suspect it has something to do with something different between what Alinator does, and what I do.

I also think it'd take weeks for a change to really show, and if I'm not mistaken, you've been changing things much more quickly.

... and if JM7 says "fixed in 6.2.19" then it seems like that'd be worth trying.

But, maybe that's just me.

---

When I said try upgradint to 6.x, I did not realize that the problem was occuring with small caches.

Yes, the actual resource shares in this case are going to be driven by the availability of S@H if S@H is available less than the requested resource fraction.

Because many computers are running offline much of the time, BOINC attempts to keep the queue full to the specified time at all times that it has an internet connection. The way to avoid the problem is to:

1) Set your Connect every X to about 8 hours.
2) Set extra work to a small value.
3) Set a connection schedule so that Tuesdays during the normally scheduled outage BOINC is not allowed to connect. The reason for the 8 hours is that you probalby only have an idea of when exactly the outage is going to start. If S@H is available when you connect back in, little or no Einstein work will be downloaded.

---

BOINC WIKI

I have tried connect every 12 hours and do have comms inhibited Tues 17:00 UTC till Wed 05:00 UTC.

But I also want total of 2 day cache to cater for the occasional Seti w/end outages.

I built up massive LTD's for Einstein and Seti Beta during the week Seti introduced AP work and w/end Einstein introduced S5R4. During these outages and the next 4 weeks, I did about 6 months SetiB work, as per resource share, by being force into EDF mode. I have suspended SetiB since that work was completed.
But running, without intervention by me, the computer has never run out of Einstein work due to small seti hiccups and debts, short and long, have stayed at exactly the same values since then.

I think, one can probably discuss this for ages, only real importance is, not running out off work and meeting deadlines.
I have a 3 days cache and a 3 to 1 resource share, in favor off SETI and never ran out off work, so far.
And, also important, always meet the deadlines, (without going into High Priority Mode), so I think, this is an ideal resource-set, for me atleast.
Don't know what difference BOINC there is between version 6.2.xx and 5.10.45.
(And BOINC 6.1.0.)
I like the output off the 3:1, setting, even it's not really 3:1, but (10:1)EINSTEIN and (200:1) for LHC.

---

Just wanted to make a note here of my little experiment:

I ran my quad to zero wu's by turning off new tasks from einstein or seti. Then I enabled seti, and then einstein. Seti downloaded about 18 shorties and about 9 standard wu's (expected run times of 17m and 60m). (About 4.25h/core on my quad.) The shorties have a short deadline (7d) and the standard wu's have a long deadline (23d). However, boinc then downloaded 1 einstein, despite the LTD being at 4.8Million against einstein, with a deadline (17d) in-between the bimodal seti wu's. My cache settings are ci/co=0.0833/0.08, corresponding to two hours each (2/24 = 0.0833; co was rounded down on the form for some reason). I'm running boinc 5.x on this client. There were no reported problems with communications with either project.

Now I'll watch to see if the LTD goes down using these small cache settings, realizing that Tuesdays may systematically disrupt the boinc algorithm. The small cache only allows for 2h of 'full' cache settings. The einstein wu's are expected to run for about 10h on this computer (a quad).

I expected to see just seti's being downloaded because of the LTD and because there were no problems getting wu's from seti. I don't understand why an einstein request was made at all.

Just wanted to make a note here of my little experiment:

I ran my quad to zero wu's by turning off new tasks from einstein or seti. Then I enabled seti, and then einstein. Seti downloaded about 18 shorties and about 9 standard wu's (expected run times of 17m and 60m). (About 4.25h/core on my quad.) The shorties have a short deadline (7d) and the standard wu's have a long deadline (23d). However, boinc then downloaded 1 einstein, despite the LTD being at 4.8Million against einstein, with a deadline (17d) in-between the bimodal seti wu's. My cache settings are ci/co=0.0833/0.08, corresponding to two hours each (2/24 = 0.0833; co was rounded down on the form for some reason). I'm running boinc 5.x on this client. There were no reported problems with communications with either project.

Now I'll watch to see if the LTD goes down using these small cache settings, realizing that Tuesdays may systematically disrupt the boinc algorithm. The small cache only allows for 2h of 'full' cache settings. The einstein wu's are expected to run for about 10h on this computer (a quad).

I expected to see just seti's being downloaded because of the LTD and because there were no problems getting wu's from seti. I don't understand why an einstein request was made at all.

SETI only gave you a bit over 3 hours / CPU, and therefore your cache was not completely full - thus the request from Einstein.

---

BOINC WIKI