As Martin has already mentioned, BOINC already includes "exponential backoff" to spread connection-attempts over longer time-interval.

Doesn't quite seem to work like that. ...since then it has been held for a random time between 0 and 4 hours. One 'back-off' on day five was for less than two minutes.

That's exactly how it works. There are two ideas at play:
1: To randomly spread the access attempts;
2: To slowly decrease the frequency of those access attempts.

... I still think that the fact that the back-off is per result, rather than per project, is a design weakness in BOINC which is contributing to 'too many connections'...

It's not that simple. You can get individual results that always fail to upload and those should not completely block any one project.

I agree, the present Boinc logic for handling failed communications can be improved. We can wait to see "what" and "how" for the present problems and then see the fix in a later Boinc release.

Its all part of the experiment!

Happy crunchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

I still think that the fact that the back-off is per result, rather than per project, is a design weakness in BOINC which is contributing to 'too many connections'.

The only bad design here, is that the user does not have the option making this a "per result", "per project" or "per client" back off option. And for the "per project" option having a choice of setting oldest only or try all of them if the first one uploads.

I see no problem with the server handling "too many connection attempts" with a quick response to go away, as long a the client does so quickly and does not set there for several minutes giving false hope to the user by saying "Uploading".

Remember, the client is running on the users machine. It is the users machine, and the ways in which the machine is used should be controlled by multiple choices made by the user. Not the programmer who's primary interest is in solving workload problems of the project.

Very good suggestion Grant. Done.

Thanks.

Now all we need is people to actually read it...
*fingers crossed*

I read it and it seems to me that we should donate actual money. If I could afford it at this second I would do it now. Howver I cannot. I am in it for the science and as soon as I have the means to donate by god I will. I can promise that.

I hope everyone will just be patient and look at all the other projects around. Still be faithful to SETI and check her out often. I know I am not going anywhere.

---

"By faith we understand that the universe was formed at God's command, so that what is seen was not made out of what was visible". Hebrews 11.3

[quote]I keep seeing in these forum threads various valid complaints. Here are some reasons off the top of my head that SETI@home Classic is shutting down, and other facts of life that may very well address your particular concern:
[quote]

Just so there isn't any miscommunication/misrepresentation I thought I'd take the time to at least post my reply to this thread that was pasted on the "Classic" boards.

And now let's see.... We're so busy that we couldn't even come to the Classic
boards (where the majority of original volunteers are) and tell us!!!!

I've been here for almost 6 1/2 years. I've been through all the problems and
stuff we've had to deal with and always stuck it out because that's the kind of
guy I am. I was dedicated to helping with what to me was an important area of
science and that alone was my main reason for suffering through it all with all
the rest of us old timers.

We all knew that the project was only a 3 or 4 year deal and that maybe
something was going to replace it, but pardon us for our ignorance in assuming
that when that day was to come along that it was going to be a smooth
transition and all our efforts were going to be appreciated. Instead we are
treated like a POS, an afterthought and told what we were/are doing here at the
end is a waste of time. Maybe I'm old fashioned or something but I was kind of
expecting a lot of joy and celebrations both at Berkley and amoung the
volunteers for a job well done and a lot of pats on the back, maybe even a
special certificate we could download. Instead we're being dismissed because ...

I'll hang around at BOINK until they eventually update our stats from Classic
and print out my certificate and then "that's all folks". I won't bother doing
any work for BOINK since apparently that'll also be a waste of time.

I've been willing to give the guys at seti@home the benefit of doubt and at
least give them the same chances that we ALL gave them during the troubled
times we've had over the past 6 1/2 years. However, all doubts have now been
removed and its time to move on. What I once looked on with "PRIDE" I now feel
abused.

To the many people/friends I've made over the years I say THANK YOU for all the
help, companionship. You all have helped me learn so much more about so many
things I never would have undertaken on my own that I don't how to repay any of
you except with THE BIGGEST THANK YOU I can muster up from my heart.

Arion

PS.... Matt my response is not directed specifically to you as you have been our fearless leader through so many troubled times. I know things have been difficult here at the end for everyone, however maybe you could at least pass some of these thoughts along to those in a position to maybe attempt to understand why so many people got involved in the first place and maybe make a few changes that could bring the real heart of the project back to so many of us that feel that it was abondoned for whatever reasons.

Arion

---

Hi Matt,

while I do have to accept some of your comments as facts, I can't agree with some others.

5. The current crises are just par for the course in the history of our SETI project and public resource computing. Things break, deep breaths are taken, and they get fixed eventually. This isn't good for making our participants happy of course, but we do our best with what we got and so far our user base has stuck with us through the painful periods (thank you!!!).

From my point of view, this was to happen. The Boinc infrastructure never was that stable then the old one (even though the old one may have been a mess)

8. Though I don't have any accurate numbers to back this up, I personally feel that so far SETI@home/BOINC has had more uptime on average than Classic. We had science database crashes every week at times back in the day, several whole weeks when we were down for database recovery or because somebody stole a cable, network bandwidth issues that brought us down for months. And these were just the public-facing downtime events (among but a few).

though I do not have any accurate numbers either, I do disgagree. I never had a problem with these hickups since installing setiproxy for the classic seti. I just had to set the cache large enough. I even could control this centrally in my infrastructure. This has changed with boinc. I still have the ability to do caching, but for some reason, this never worked that well as with setiproxy and I had to check each system, whether it had a problem not just the central proxy.
So even the BOINC infrastructure may have had a better uptime, it did not work during that time (just like now, in my opinion we have a downtime of boinc, even though this is not reflected in the stats).

In addition, some systems will no longer be able to connect to boinc, as they can't reach the http proxy directly, but that's a different story.

Now I'm off to bed. Going to LA tomorrow. Be back Sunday night.

- Matt

So do I, I'm not quite sure, whether I will still support SETI in the future as I did in the past, because (as also mentioned by Arion) 'the handling of the human factor was suboptimal'. This happend with the RC5DES project and made me move to seti, mybe I have to look for another one again.
This is not against you Matt, you are just the one who made some statements I use to repsond to. In fact it's mostly based on the impression I got about the treatment I got from SETI.

I wouldn't mind of only minor issues would be present, in every change, there is a limitation. But I never got the feeling, that the boinc stuff had the same quality as seti classic and the only reason why I started looking at boinc was to get knowledge about the limitations (which is also why I initially immediatly stopped using boinc after the first try).

With frustrated regards,

Patrick

---

Kind regards,

teddi

... I wouldn't mind of only minor issues would be present, in every change, there is a limitation. But I never got the feeling, that the boinc stuff had the same quality as seti classic and the only reason why I started looking at boinc was to get knowledge about the limitations (which is also why I initially immediatly stopped using boinc after the first try).

With frustrated regards,

Patrick

Looks like you're looking at this from a very blinkered and very personal point of view. With such a negative and highly expectant initial viewpoint, you cannot help but be bitterly disappointed.

Note that Boinc and Boinc-s@h are an experiment and that we are volunteers and guinea-pigs in that experiment. No one has done this type of project before. Also, the "game rules" have changed since s@h-classic. The caching "restrictions" are all part of making for a fairer and more fun and more useful game.

Note especially that this project is not run for you.

If you are enthused that the science is worthwhile and that the game rules are acceptable, then good and please join the game.

Otherwise, you're welcome to watch or to play in another game.

Meanwhile, considering the "real-world" rules of the game that Berkeley must play, I think they do a very good job and play a real hot game. The best game on this planet.

Have fun,
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

Does this statement Matt made on 12/10/05 really ring true? I think not!

I suggest you do a bit more reading, then you might realise that not all of the problems with Classic were visible to the public. And even then, as someone that's been crunching Seti since 1999 i can remeber [i/]many[/i] minor outages with Classic, as well as a couple of big ones.

Having been here in BOINC for about 18 months now- this is the second longest outage since i've been here (and may yet be the longest), but other than one dodgy version of BOINC that brought systems to a grinding halt i've not had any problems with the software & most other outages have been less than a few days. And given that i have a cache that's good for a few days they haven't affected me at all.
Un even this one has yet to have any effect as i've still got about 2 dys worth of work. If i run out of work, so what? When it becomes available again, i'll start crunching again.

The most disappointing/annoying thing about this present problem is that it's occured as people are transitioning from one system to the other.





Oh, Arion & teddi-
This would have been a more appropriate thread for your posts.

---

Grant
Darwin NT

A good thing about the shutdown of Classic will be that there won't be anymore excuses of how Classic was using hardware and resources for other projects. If the problem was with the backend part of Classic, why wasn't there an effort to fix that problem? Most new programs all the way around do not build on previous successes and failures. It is apparent that the learning curve at least for the new program is STILL VERY steep, considering all of the problems I've seen over the past few months.

Since what I am doing at this time is redundant on Classic and is not supporting science, all I crunch until 15 December is for personal satisfaction to show my dedication and commitment to Classic. After the deadline when results will not be accepted, I'll shut down my small farm and go on to other things. At least I will not be causing problems for all of you running BOINK.

I'm personally proud that my small contribution to SAH has helped bring distributed computing to fruition. Now I'm not only crunching basically useless SETI data (the sensitivity of the main instrument is a joke..we need a big array on the far side of the moon) but I'm also helping design the new LHC and crunching Gravity Antenna data.
Other important projects are receiving the resources they need. It's time to retire classic SAH like a worn out car which is fine. The new one has a CD player and a DVD in the back for the kids. :)
Problems with the system will always exist with ever underfunded basic research. Set up another project and crunch needed data for somebody else while they fix it.

---

If you don't touch it, you can't break it.
;

I'm also proud of what my computers donated to the program too. I also learned a lot from the other crunchers at classic the last 2yrs. And I'm no computer whizz. Almost from the beginning, people were talking about the new seti coming. When I hit 10,000 w/u in May I started moving computers here. Except for one major outage, it runs okay...yeah, not 100%, some minor problems, the credit thing, cobblestones is it(???), hey it's new. Changes will occur, especially this new program, but, so what. I'm looking at the other programs, maybe this gravity thing...or???? It will work out. Dan Myers

---

..... If the problem was with the backend part of Classic, why wasn't there an effort to fix that problem? Most new programs all the way around do not build on previous successes and failures. .....

The fix is BOINC

---

Oh, Arion & teddi-
This would have been a more appropriate thread for your posts.

[quote]

As a matter of fact it isn't the place where I should have posted. I replied to Matt's original post which IS appropriate.

As for whinning and bitching and having hissy fits there are times and places and that forum is probably a good place to put them. As for me, I've rarely over the past 6 1/2 years complained about much at all. I switched over my slowest machine to boinc last summer and started working with it so when the time came I'd be up to speed and know what it was all about. I pulled that machine off boinc a few months ago to help me finish my last few personal goals with Classic and had intented on bringing it back over when Classic shut down. Matt has done wonders over the years with seti@home. To have him spell out things the way he did in his post it only seemed fitting that I have an opportunity to express my opinion in relation to his post. I DON'T consider that whinning, bitching or throwing a hissy fit.

---

I'm a long-time SETI user, but a short-time BOINCer.

And I'm new to these fora.

But I'd like to respectfully suggest that Matt's answer be re-posted at the top of the board with a big "BOINC SPEAKS" or somesuch.

I haven't been here long enough to know who KNOWS and who THINKS they know, but it sounds like Matt knows his stuff.

We are all frustrated by the lack of concrete information, especially given that SETI Classic has been ripped out of our "cold dead hands" and subsituted with something somewhat flawed at the moment.

Firm information from people who know would be very welcomed.

---

"We're all mad here. I'm mad. You're mad."
"How do you know I'm mad?" said Alice.
"You must be," said the Cat, "or you wouldn't have come here."

Thanks for the info Matt.

One thing I think that needs to be addressed is the "exponential back off" thing.
Once the number of files waiting to be uploaded/downloaded (by a host) reaches a critical limit, there always will be a file trying to up/down load.

My 2 pcs have reached this point.
The max back off seems to be about 4 hours, and one file will try for 3+ minutes before it backs off.
It's clear (to me, at least) that once you have a certain number of files trying to upload, that while one file tries & fails, another will come to the end of its back-off time, and be ready.

I do 21 a day across 2 pc's.
That's one upload every 1.1 hours.
Now that I have 80+ waiting to upload, there is always at least one file trying to upload. I have gone from hitting the server less than once an hour, to hitting it constantly.
The size of the user-base, coupled with the speed of modern machines suggest to me that the back-off time needs to be increased by an order of magnitude, and to take into account the WU's deadline.
None of my work will go overdue (I know, the deadlines have been scrapped for the mo, but Boinc does not know this) till the 17th. Boinc could take this into account, and back off the upload of these results for 3-5 days not hours.

In the same vein, some (of my) results have 12 days to go. But Boinc hammers these at the server, regardless of the fact I have 12 days left to upload them.
Back these off 10 days, and then in 10 days time the problems should be fixed, and they will go through without problems.

my worth.

{edit:typos}

---

Foamy is "Lord and Master".
(Oh, + some Classic WUs too.)

Looks like you're looking at this from a very blinkered and very personal point of view. With such a negative and highly expectant initial viewpoint, you cannot help but be bitterly disappointed.

I'm not quite sure, where you got this impression

Note that Boinc and Boinc-s@h are an experiment and that we are volunteers and guinea-pigs in that experiment. No one has done this type of project before. Also, the "game rules" have changed since s@h-classic. The caching "restrictions" are all part of making for a fairer and more fun and more useful game.

right and it was an experiment I was interested in and intersted in supporting it.

Note especially that this project is not run for you.

I knew that, so what's your point here ?

If you are enthused that the science is worthwhile and that the game rules are acceptable, then good and please join the game.

depends on what you treat as game rules. Especially community projects live and die with the support they get from the community. I think, even in science, this is something which should be considered.

Have fun,
Martin

will try.

Regards

Patrick

---

Kind regards,

teddi

... thank You Matt to give us all this internal informations!

For me Seti Classic is a kind of "private tradition".
But times change and if it's time to let it go, i'll do.

I allso like BOINC, it's going to become a new "private tradition".
In BOINC is more inside, there are more possibilities, allso for me!

But Matt, many people are very closed (in feelings) to SETI@home! Not only since today!!!
So i mean it is very sad to read things like Your's at this Point - the first time!
It would be a great thing if we, the user could experience internas about SETI@home and BOINC some more often!
Not all, not every day, no "date-terminus"!!! But sometimes!
It's to different worlds and they belongs together!
We the user lives mostly with rumors, or the User have lots of work
(by klicking links, surfing many Websides, study and translate other medias or maybe to write many emails!) to do to get some intern facts. Mostly our "practic facts" in the dayly life are things like "crunching" server or other problems.
It's okay to have open FAQs or a really living Forum and the technical news, but for all it's not enough! I miss the personal, the private part/kind/step, the "big"- or better the intern connections!

Do You know what i mean?

However, the SETI@home and BOINC - Team is doing a very great job!
Go ahead!

Thank You!

Michael Kopsieker (Miko)

---

http://altes-beckhaus.com - my home & job
http://crunching-family.at - my team

None of my work will go overdue (I know, the deadlines have been scrapped for the mo, but Boinc does not know this) till the 17th. Boinc could take this into account, and back off the upload of these results for 3-5 days not hours.

In the same vein, some (of my) results have 12 days to go. But Boinc hammers these at the server, regardless of the fact I have 12 days left to upload them.
Back these off 10 days, and then in 10 days time the problems should be fixed, and they will go through without problems.

I agree it would be good, but imagine the wailing and nashing of teeth if that were to be implemented. And the people still repeatedly clicking on update till their fingers go numb.

---

Grant
Darwin NT

Thanks for the info Matt.

One thing I think that needs to be addressed is the "exponential back off" thing.
Once the number of files waiting to be uploaded/downloaded (by a host) reaches a critical limit, there always will be a file trying to up/down load.

My 2 pcs have reached this point.
The max back off seems to be about 4 hours, and one file will try for 3+ minutes before it backs off.
It's clear (to me, at least) that once you have a certain number of files trying to upload, that while one file tries & fails, another will come to the end of its back-off time, and be ready.

I do 21 a day across 2 pc's.
That's one upload every 1.1 hours.
Now that I have 80+ waiting to upload, there is always at least one file trying to upload. I have gone from hitting the server less than once an hour, to hitting it constantly.
The size of the user-base, coupled with the speed of modern machines suggest to me that the back-off time needs to be increased by an order of magnitude, and to take into account the WU's deadline.
None of my work will go overdue (I know, the deadlines have been scrapped for the mo, but Boinc does not know this) till the 17th. Boinc could take this into account, and back off the upload of these results for 3-5 days not hours.

In the same vein, some (of my) results have 12 days to go. But Boinc hammers these at the server, regardless of the fact I have 12 days left to upload them.
Back these off 10 days, and then in 10 days time the problems should be fixed, and they will go through without problems.

{edit:typos}

Hear! Hear! I like your idea of fixing the "back off" issue.
Unfortunately, I believe this issue is a BOINC client problem, not at SAH problem. Should re-post these ideas over on the BOINC forums.

--- to all ---
Herein lies the beauty of the BOINC client (others have touched on this as well). Got a problem with project A.....crunch numbers for project B and C. Contribute to science on more than one front. Once you start racking up points for those other projects, you will have those same good feelings of contributing to something just as you have had with SAH all these years.

---

thanks for the information Matt...scientific research and discovery takes much patience...particularly when operating on a shoe-string budget...i think reading your explanation will give many people something to think about...and a reason to be patient while these difficulties are resolved.

---

PROUD TO BE TFFE!