In most English speaking countries a "result" is indeed what comes after something. At Berkeley a result in a duplicate of a work unit, and it is called a result before it is sent, while it is out in the field, and after it comes home.

And you know this is not true based on your reading of the white papers that explain the BOINC design?

The goal is to make computing affordable.

You don't make it affordable by spending lots of money. You make it affordable by minimizing costs.

OzzFan is making this point because he's actually read the white papers. He's bothered to figure it out.

If you disagree, you're either too lazy to research, or you'd rather make up the rules as you go.

The goal is not to make the SETI servers more reliable, but to understand that people's expectations are far off base. SETI never promised to have work available all the time.

I don't believe what I'm reading here...

Again, due to a lack of reading or comprehension.

More reliable servers (and a steady flow of work) are desirable, but they're things we'd like, not necessarily based on reality.

SETI@Home (and Serendip) get a "free ride" because there is no money for telescope time -- they don't control the telescope.

When the telescope is used to transmit (planetary radar) it's off. When they're doing maintenance it's off. When they get data, they get whatever they get.

Data is sent on hard disks via FedEx because that's the cheapest way to buy bandwidth.

The project has exceedingly finite resources.

The BOINC client has a cache to keep work when things are broken or work is unavailable.

Uploads and downloads are scheduled to try to minimize the peak server loads and deal with congestion and outages.

Reporting is done in batches to limit transactions to what will fit with the relatively tiny servers.

... and the overall BOINC design accepts that this won't always be enough to keep everyone busy.

Why? Because it's cheaper than building high-reliability servers -- by 10x or more.

Redundancy is wickedly expensive.

But, most people would rather make up the goals instead of read what BOINC or SETI@Home actually says.

I agree it would be interesting to see the data further back. Does anyone have it? There are graphs of both users and hosts on Scarecrow's site, my comments apply pretty much to both.

Maybe, maybe not. Need more data. Lots of us here in the 10 year club...

According to BOINC Stats and Scarecrows statistics over half of all the active BOINC users are active SETI users. The other 40% or so are spread over more than 50 other projects. SETI's position is safe for a while.

I think S@H has proven that very old second hand servers, running freeware and volunteer written code, have limits.

How will any more users add to that knowledge?

How can a very limited staff of scientists do science if they have to spend time coddling annoyed users?

To finally manage to process all incoming data in the realtime (or close to that)?

First, I need to see the scientific value in that. Second, you don't need a team of Berkeley scientists to accomplish that. Lots of e-commerce sites have shown us that all it takes is time and money.

Time and money are very finite resources at S@H.

Every minute and every penny spent of keeping users contented takes away from real science.

A balance must be struck.

Users (and their machines) ARE the true power of S@H - It was many times underlined by S@H staff and it's well self explanatory. The more users the more power - the more power the bigger things you can achieve.

In the long run, I agree. But right now users and power don't appear to be the limit.

How will adding more users today help us when Arecibo is shut down,

or when the last of the old servers finally dies?

Need to spend some time on those topics too, and right now.

All it would produce is more people griping about lack of work, lack of respect, lack of customer service, etc. I hope they have found more important things to spend their time on.

Like what?

Volunteers are lined up and waiting right now, that issue can wait.

I expect S@H would love to have Edison's funding, corrected for inflation. Who said they're giving up on the science?

And feeling the pressure (apart from stressing staff a bit) influence changes and improvement.

My experience in working in and managing scientific and technical teams is that pressure to improve the science does help. Pressure to keep vaguely defined "outsiders" happy makes people update their resumes.

Certainly not public relations, unless it generates new funding.

In my moments of quiet despair about this project (and I do have them) I wonder if part of the experiment is to push the users, and see at what point we all give up. As I said before, we are not there yet. The 10 year club survivors all appear devoted and passionate after all these years.

[quote]If this bother you, goodbye. Thanks for the CPU cycles. There is somebody standing in line to take your place.

So the fact that you spend more money on computers makes you a better person than me, and gives you a bigger say in how S@H is run? Seriously?

Maybe we have an issue in language usage here. It sounds to me like you are griping.

The fine print was just an attempt to lighten up a conversation that I feel has become much more serious than it deserves to be lately. I know humour doesn't always translate well.

Eric posted at least twice this last outage. That is a 100% improvement over the last outage. Things are looking up.

Now if the Campus will fulfill the rumor of a gigabit connection between seti and the world, We could at least get rid of one problem and I'd think It would make up for what the campus did recently to our connection. :D

I'm always amazed/amused at all the people who keep predicting that SETI users will leave in large flocks because of the "bad service". Take a look at Scarecrow's graph of users. In particular, look at the graph at the very bottom of this page.

The number of active hosts is pretty flat, while the total number of hosts climbs slowly over the last 365 days (which includes some major "bad service" events). This tells me that there is some turn over in users who sign up, give it a try, and then quit for whatever reasons.

Since S@H has a hard time generating enough work for this existing user base, why would they bother with any extra effort to retain more users or generate new users?

All it would produce is more people griping about lack of work, lack of respect, lack of customer service, etc. I hope they have found more important things to spend their time on.

If this bother you, goodbye. Thanks for the CPU cycles. There is somebody standing in line to take your place.

Fine Print: this polite kiss-off reflects the views of this user only, and is not the offical position of this station, this network, or its sponsers.

I'm not saying it isn't bad right now. I'm just saying it has been worse.

I've been told that there was a time during Classic where the servers were down nearly a month.

The important thing to take from all this is that the project never guaranteed 100% uptime or that workunits would be available at all times.

If you're referring to Classic, then you're sorely mistaken. It was not on very rare occasions, but just as problematic as it is now. Some times were worse then others back then, just as they are now. People forget.

No, because they didn't share as much with us. We did just as much work with less information. Computer were so slow back then that the project could be down an entire week and most people would still be working on the same workunit. This is one of the reasons why people don't remember very well.

Yes, they are much smaller. None of them have over 297,000 active participants with such a rabid following, building crunching farms with multiple nVidia cards just to increase their numbers (not saying it doesn't happen - just not on the scale it happens here on SETI).

I would even bet that most of those projects, being newer and all, have started with newer hardware - possibly even hardware that isn't donated or in beta. They have had the opportunity to spend their NSF money on decent stuff. SETI, on the other hand, uses donated/beta hardware all the time.

I'm aware that S@H is a HUGE project, but technology have found solutions to solve such issues. I think that Matt et al. and the BOINC 2-rack farm is not able to support the demand of such number of crunchers/mangle sci data. I'm feeling like there is a need for additional layer in the S@H processing (i.e. sites with WUs) which would distribute work for lets say geographical region and users should be allowed to choose couple of them (to have redundancy).

The problem with this idea is that all the workunits must come from a single source (the tapes/hard drives at SETI). Likewise, they must return to the same source. This suggestion just moves the problem around but does nothing to actually solve the bandwidth issue.

Sounds like a lot more technology going toward the solution, which means even more points of failure in the system.

Plus, in order to "synchronize" with Berkeley, they'd be using up some of the bandwidth, which simply moves the problem around but does nothing to solve the issue.

If we made the system that complex, how would that be big science on a small budget?

The problem is that you're trying to find ways to make the server have more uptime. This is contrary to the entire thesis of BOINC. The idea is that you do not need a server with 100% or even 80% uptime.

The goal is not to make the SETI servers more reliable, but to understand that people's expectations are far off base. SETI never promised to have work available all the time.

The sooner that is understood, the sooner people wouldn't get so upset or angry when something happens.

Patience is a virtue.

Stable? Something tells me you haven't been paying attention.

There has been worse outages than this one. The project has never been stable in the sense that most people think.

Please do not put words in my mouth -- because that is not my position at all.

I want to see it work, perfectly, every day,[..]

but there are two very common fallacies that color your thinking:

1) The servers are down right now, and this time-critical work has to get back right now!

2) The project requires smooth flow at all times.

On the first point, we're working on signals that are decades old -- assuming that the source is relatively nearby. It likely isn't.

On the second point, the amazing thing about BOINC that is not true of most other internet based activities is that the project itself can tolerate some bad times.

The problem, which you have illustrated quite well, is that the BOINC servers aren't unhappy. The BOINC clients are retrying on their own, and will get work through when they can.

... and many of the users who watch closely are worried and upset.

I'm not just sitting back and saying "this is how it is" -- when I'm not correcting people when they mis-state my position, I'm reading BOINC source code and thinking about ways to stop the BOINC clients from hammering the BOINC servers.

I think I can see ways to make things better. They're worth trying.

In the meantime, I think we all get to choose: we can be anxious or worried or angry, or we can see if there is some way to make BOINC work better, or we can see if there isn't some way to help get the gigabit upgrade.

Remember that the SETI@Home operations staff is basically three people.

Given the choice of frequent updates, or Matt giving the tasks his undivided attention, I vote for undivided attention.

I had the same comment as Sten-Arne.

It is working as you'd expect it to work when you have 180,000 BOINC clients trying to connect to one or two upload servers.

It's easy to say "more servers" but SETI has a finite amount of money and equipment, a finite amount of room in the server closet, a finite number of electrical outlets, and a finite amount of A/C.

As usual with everything in life, I do not expect it to work at all. In that way if it indeed doesn't work, I do not get disappointed, but if it works I do get very happy :-)

Matt, do whatever you're doing, but:

1) drop a line just to keep people in the loop
2) disable deadline checking or people might not get credit for all the hard work their computers already did

Cheers

Profi

p.s.
Certainly situation is opposite to thread title "Working as expected" - especially when looking from a cruncher's point of view..

That depends on how you expect it to work (or not work) :-)

Guess what? A *second* drive on thumper failed this morning, around the same time the other drive failed yesterday. This system is on service, so we should get some replacements soon. But there's no obvious signs of why these two failed so close in succession. They were both on the same drive controller, but there's a 15% chance of that happening at random. The temperatures all look sane.

In better news, we got to the bottom of the weird splitter sequence number problems I spotted yesterday. Now that we understand what happened and why this really isn't a problem at all. Basically, data that was meant to be tacked on the tail of one raw data file ended up at the start of the next file instead. No biggie.

As far as those overflow workunits taking forever... Jeff and Eric wrote some code (and checked it twice) to scour the database for such workunits and "cancel" them. Immediately we saw our pipelines flood with requests for new work.. so expect some delays for a while. We hope to eventually give credit to those who got stuck with these troubled workunits.

- Matt