Well, it's rather refreshing to have a break in the shorty storm. Most of my rigs have their cache limits hit, the only fly in the ointment.
But at least the tasks that are cached have some run time for the GPUs, not 80-90% 2 minute drills.

Now, about them there limits..........

I dont know, but I have a very bad feeling that limits will not be raised soon, if ever. Servers are still maxed out with limits in place. Bearing in mind the disconnect between Seti staff who are still looking for "spare" computing power, and top crunchers with farms.
Please prove me wrong, but on the technical board, as far as I know, there has been no positive reply to a request to raise the limits.

Regards,

A

---

omg i have work

I just had a long weekend away and on return I only had 1 rig to give a nudge to its uploads but nothing ran out of work so I have no reason at all to panic. :D

Cheers.

---

Its hotter than a monky's bum in here

Well it's been 10-15C cooler here than usual for this time of year, so no worry here with heat yet, but with the help of a proxy my Q6600 is now hitting the limit again after not connecting for 3.5 days so it seems that I'm getting 5-6 days worth of work out of my 10 day cache setting on that rig (joining my other 2 rigs bouncing off the limits). :)

Cheers.

---

Its hotter than a monky's bum in here

Isn't that supposed to be, "It's hot enough to boil a monkey's bum in here, Your Majesty..."


"...and she smiled quietly to herself."

EDIT - Yes, it is.

Went to my office Sunday night about 9:00pm. Some kind soul had turned the furnace completely off. No heat and no air circulation. I walked in the front door, it was freezing. I climbed the 39 Steps to my office and was cold. Grabbed the doorknob to my office, inserted the key and noticed that my hand was warm on the imitation brass.

I threw open the door and was met by a blast of hot air. There were fans blowing like crazy in four computers. I'll bet it was 100+F in there.

I quickly crossed the room hardly noticing the body of the secretary laying lifeless on the floor (I checked for a pulse...I didn't have one, so she must have been dead), and flung open the windows on two corners of my corner office. I wish I had raised them, but I flung them so I had to go down to the street and pick them up and replace the glass - again.

After nearly two hours of replacing window panes and running the central heating's fan (but no heat) with two windows broken, it got cool enough to breath in my office, so I went to check the body I had seen on my floor... It was gone. The body had simply vanished when the temperature dropped. I put both my brain cells to work and was able to deduce that I must have been fooled by a full body mirage. (like, in a desert, where it's like, warm and stuff)

Of course! A mirage! I don't even HAVE a secretary!

But if I did have, and she'd been locked in that office, then, look-out because she'd have been passed-out or dead or something because it was HOT in there. I know hot when I feel it and it was hot. I felt it. (and so, incidentally was the non-existent secretary, although she would have been hotter had she not been all icky and dead; hot I mean, the dead secretary was "hot", not felt).

If I could just pipe that into the return air ducts at my office, the rest of the building wouldn't need to be heated... unless I flung a lot more windows.

(the computer exhaust, folks, I'm saying pipe the heat into the return air ducts not pipe the hot, dead, non-existent secretary -- couldn't feel her even if you stuff her in a return air duct; and how much fun would it be to feel a dead secretary in an air duct anyway?)

EDIT EDIT EDIT - Why do I have to explain my sentences to you people so much?

Well, thanks to the weather here being unseasonably warm for this time of year and the crunchers all having Seti to do, the kitties are all toasty warm here too. Haven't had to fire up the furnace yet this year.

Nothing is melting down here though.

---

"Time is simply the mechanism that keeps everything from happening all at once."

While again switching over to AP only, I get so depressed that I can hardly work. I only have 85 AP's in progress, and if I don't get more in the next week or so, my computers will run dry.

The pain, the pain.

A lot of APs were splitted today. Why didn't you got some of them?

Merry Christmas, and happy new WUs folks :-)

Yeah, the same to you. As a present for these holidays, I give to you AP WUs which I could get for myself, but I leave them to you :)

---

Well, it's rather refreshing to have a break in the shorty storm. Most of my rigs have their cache limits hit, the only fly in the ointment.
But at least the tasks that are cached have some run time for the GPUs, not 80-90% 2 minute drills.

Now, about them there limits..........

I dont know, but I have a very bad feeling that limits will not be raised soon, if ever. Servers are still maxed out with limits in place. Bearing in mind the disconnect between Seti staff who are still looking for "spare" computing power, and top crunchers with farms.
Please prove me wrong, but on the technical board, as far as I know, there has been no positive reply to a request to raise the limits.

Regards,

A

I think now, with the major shorty storm passed for the moment, would be a good time.

---

"Time is simply the mechanism that keeps everything from happening all at once."

I think now, with the major shorty storm passed for the moment, would be a good time.

Yes, it would be a good time, but I really believe nobody at Seti has the slightest intention of raising the limits.
Just my opinion.

Regards,

A

Edit: As in my post above this one, could somebody prove this rather miserable theory wrong?
Anything from the technical board regarding replies to requests from crunchers to raise the limits (there are a lot,) by Seti staff?

---

The limits are in place to protect against massive overfetching when GPU work is again properly estimated server side, IOW when changeset [trac]changeset:24217[/trac] is revised or removed. Getting the AP validator revised to exclude early exit times from averaging is needed first to ensure the APR for Astropulse applications is reasonable, unless someone decides the ~1200 new hosts per day don't deserve protection (new is either really new or simply a new hostID).

The project staff could decide to throw caution to the winds, but I don't expect that. Similarly, I don't expect BOINC to have a fair limiting mechanism anytime soon.

                                                                   Joe

The limits are in place to protect against massive overfetching when GPU work is again properly estimated server side, IOW when changeset [trac]changeset:24217[/trac] is revised or removed. Getting the AP validator revised to exclude early exit times from averaging is needed first to ensure the APR for Astropulse applications is reasonable, unless someone decides the ~1200 new hosts per day don't deserve protection (new is either really new or simply a new hostID).

The project staff could decide to throw caution to the winds, but I don't expect that. Similarly, I don't expect BOINC to have a fair limiting mechanism anytime soon.

                                                                   Joe

Thank you Joe! It is clear that continued patience is what is called for.

Steve

---

Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website

Only in the past several hours does it look like everyone has finally reached their limits (see Cricket graphs). My 2 big rigs only today started getting work to keep both the CPU's and GPU's busy, and only reached their limits in the last hour or so.

Now would be the time to raise the limits in small increments (with an expected rush at the beginning of each), waiting for everyone to reach the new limit before raising again. Eventually we may get back to as it was before.

---

I will play Devil's Advocate here.

Why raise the limits and stress an already overstressed system even further ?

The current limits equate to about a 1 1/2 to 2 day cache on my fastest rigs, more than enough to get them through the weekly planned outage. By keeping the current limits it helps avoid any unplanned outages. As long as the limit can be maintained I see no problem.

As long as the rigs don't run dry we should all be happy.

(BTW I normally run a 4 day cache)

T.A.

I will play Devil's Advocate here.

Why raise the limits and stress an already overstressed system even further ?

The current limits equate to about a 1 1/2 to 2 day cache on my fastest rigs, more than enough to get them through the weekly planned outage. By keeping the current limits it helps avoid any unplanned outages. As long as the limit can be maintained I see no problem.

As long as the rigs don't run dry we should all be happy.

(BTW I normally run a 4 day cache)

T.A.

In it's most basic form it's about what caches (in general) are 'really for', as opposed to just having 'lots of tasks on the hosts'.

In computer science, as with transport logistics & warehousing, operating with no (or insufficient) local caches results in supply interruptions whenever there is a stall at the slowest link in the chain. Having sufficient caches places initial startup load demand, but smooths out & allows for service interruptions without bringing the whole chain to a stall if some link in the chain drops out temporarily or periodically. It acheives that through techniques called 'latency hiding' which if perfectly balanced for transport might be something like 'Just in time delivery', which theorises a perfect sized cache at each stage to keep steady flow at the consuming end with minimal storage or interruptions.

In those respects, obviously there are finite limits on the total work that can be supplied, though reasonable local caches allows for periodic interruption or maintenance to occur to improve overall efficiency of the system effectively smoothing the peak demands. Too big caches obviously also has its own problems with the finite workunit storage setup being centralised, though by rights total in/out turnover should be no different but more constant & spread out if balanced caches are in play. Less peak server stress & end system downtime without much cost.

Unfortunately the bigger part of that cost to build 'our' caches would likely be that workunit storage, so a very real physical limit at this time. Fingers crossed the efforts of the GPU Users group will be a big help along those lines.

Jason

---

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.

In it's most basic form it's about what caches (in general) are 'really for', as opposed to just having 'lots of tasks on the hosts'.

In computer science, as with transport logistics & warehousing, operating with no (or insufficient) local caches results in supply interruptions whenever there is a stall at the slowest link in the chain. Having sufficient caches places initial startup load demand, but smooths out & allows for service interruptions without bringing the whole chain to a stall if some link in the chain drops out temporarily or periodically. It acheives that through techniques called 'latency hiding' which if perfectly balanced for transport might be something like 'Just in time delivery', which theorises a perfect sized cache at each stage to keep steady flow at the consuming end with minimal storage or interruptions.

In those respects, obviously there are finite limits on the total work that can be supplied, though reasonable local caches allows for periodic interruption or maintenance to occur to improve overall efficiency of the system effectively smoothing the peak demands. Too big caches obviously also has its own problems with the finite workunit storage setup being centralised, though by rights total in/out turnover should be no different but more constant & spread out if balanced caches are in play. Less peak server stress & end system downtime without much cost.

Unfortunately the bigger part of that cost to build 'our' caches would likely be that workunit storage, so a very real physical limit at this time.

Jason

Agreed Jason
But I think you overlook the fact that after an outage there will always be a rush as those that are totally out of work seek to restart and those who still have work but whose caches have run down seek to build back up to the preset level.

On a restart cache size makes no difference to the demand for work/server load. For what you say to be effective the servers would have to look at who still has plenty of work, who is running low and who is empty and prioritise the allocation of work units accordingly. This does not happen and we get the usual "log jam" which can take a week to clear. If the system could prioritise work unit allocation effectively so that
1) Everybody has SOME work
2) Everybody has (eg) 2 days work
3) Top up caches to the set limit

As happens in the real world examples you quote above, where after an "outage" in a "just in time" supply situation the supplier gives priority to those with the least amount of product on hand and then catches up with the rest over the next few days, the recovery from a server outage would be a lot smoother.

T.A.

Agreed Jason
But I think you overlook the fact that after an outage there will always be a rush as those that are totally out of work seek to restart and those who still have work but whose caches have run down seek to build back up to the preset level.

On a restart cache size makes no difference to the demand for work/server load. For what you say to be effective the servers would have to look at who still has plenty of work, who is running low and who is empty and prioritise the allocation of work units accordingly. This does not happen and we get the usual "log jam" which can take a week to clear. If the system could prioritise work unit allocation effectively so that
1) Everybody has SOME work
2) Everybody has (eg) 2 days work
3) Top up caches to the set limit

As happens in the real world examples you quote above, where after an "outage" in a "just in time" supply situation the supplier gives priority to those with the least amount of product on hand and then catches up with the rest over the next few days, the recovery from a server outage would be a lot smoother.

T.A.

That's right, we're talking about a theoretical optimum concept versus realities. One thing that would happen in such an iodealised mechanism when things come online is perhaps a staged onramp and priorotising the biggest consumers first as described. That's still a logistics perspective, and somewhat at odds with the way project backoffs and manual retry buttons work for us. In that 'ideal' system you would never have run completely dry in the first place, therefore the peak is briefer and controllable.

We know the time estimates are at best marginal, so perhaps a better quantity for onramp rationing is the current # of tasks, but not allowing that to ramp up will pretty much guarantee sustained peak, or in CPU cache analog terms 'thrashing', or in manufacturer terms perhaps plant shutdowns & layoffs due to supply shortages.

Even if we take # of tasks as currently implemented & ramp that up gradually until the demand levels out (hopefully at around 70% of max bandwidth), there's still the server workunit storage issues not solved as a hard limit. For what it's worth under smooth running I also find my systems run continuously with 3 days cache, tolerant of normal short maintenance outages AND backoffs. What they can;t seem to tolerate is the continuos heavy contention on the comms line, during which most requests are at best met with 'Project has no tasks available'. Might as well have saved the bandwidth & not asked for work.

Jason

---

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.

Yeah, I'll be happy with the dang limits until the next time the servers trip up for more than a day or so, or there is trouble coming back up from an outage.

I am NOT happy having to choose Einstein or any other Boinc project when Seti is my passion and having some steps taken on the remedial action necessary to get the cache limits lifted has not been very forthcoming. The foot dragging on this is getting a bit irksome.

Up until the Boinc botch, my caches have allowed me to weather many a Seti storm of one kind or another.

You can't blame me for desiring a return of my ability to cache enough work to cover an extended outage.

---

"Time is simply the mechanism that keeps everything from happening all at once."

The limits are ok when everything is running as it should be, but unfortunately whenever we seem to have a hic-cup the server seems to stoke the fire with masses of AP's and shorties.

If there could be a little bit of pre-sorting of the tapes so that we don't get flooded at peak demand times.

Or is it because the servers get too many shorties in one go it causes the hic-cup in the first place?

---

Kevin

Each tape is split twice, once for the MB, and a second time for the AP.
The MB come in two runtime "sizes" normal, and shorties; there is no difference in the splitting process between the two, it is determined by the way the telescope was moving, or not, during the data collection. S@H has no control over this movement as it is piggybacking on other people's data collection.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?