Continued server problems.

We're continuing to have issues due to a database problem early last week and a botched attempt to fix it.

The problem is that the result and host tables in the database have grown large enough, and hosts have gotten fast enough that the lookup of result in process for a host and the enumeration of new results to send don't finish before the web connection times out either on the server or the client side. This resulted in hosts being assigned large number or results to compute without the transaction that tells them about these results being completed. The host. think it received no results would then contact the server for more results, which it would again not receive.

This isn't a hardware problem. The database currently fits in memory and the processors are fast. We've just crossed a threshold where each host computes fast enough that host queues and the result table have become large enough to cause this problem. To solve it, we've put per host limits on results in process back in place. But hosts that are having this problem will probably continue to have it until the average number of results per host has fallen to a workable level. That could take weeks.

For a more permanent fix, we plan do more work in each result by quadrupling the size of the workunits. But that fix will probably take months to implement and test.
____________

Thanks for the news, we will struggle through.

AS this problem only seems to occur when AP tasks are available, might it be possible to balance the rates of splitting. At the moment the AP tasks are being split much faster than the normal MB tasks.

Eric, thanks for the news!


By the way ..
In the *panic thread* in the NC subforum, some members report that they have *better* server contact via PROXY usage.
Maybe a S@h router need a reboot?


* Best regards! :-) * Sutaru Tsureku, team seti.international founder. * Optimize your PC for higher RAC. * SETI@home needs your help. *
____________


>Das Deutsche Cafe. The German Cafe.<

Does that explain why the Schedulers are currently (2115, 9/18/12 Berkeley time) down?
____________
.

Confirms a theory I had as to why proxy servers work - they send the data stream slower as they are sending to multiple other systems as well.
Should be able to configure my router QOS to send, say, 30K / sec max for seti program. For now at least though, finally found a good proxy.
Good luck with the fix - you are burdened by your success!
____________

Thank you Eric.

Can we all now just wait and not start filling these boards with "When is this going to be fixed threads"

If it takes months it takes months I for one will be happy to wait.
____________

Eric, thank you very much for that comprehensive report explaining what is going on. That is exactly the sort of information that everybody needed to be told, the scope of the problem, and what is being done about it.

I agree with Bernie, let's not fill up the boards with "when" posts, but also I would suggest that we don't all add "suggestions" on how to fix it. Eric and the team at the Lab have years of experience with this project, and will know what has to be done and how to do it. They already know it isn't a hardware problem, and the fix will be down to manhours, which with Matt's current absence is going to take time.

Most Boinc projects were scoped out in the days of CPU computing, and the advent of GPU power has brought about this crossing of the threshold that Eric mentions. In past times the man with the quad was seen as god, these days crunchers with 4 or 6 GPU cards are not that unusual.

Again, many thanks Eric, and I am happy to wait a couple of months for it all to settle down, as I know it will given time. There is always Seti Beta of course.

Can we have a new switch in `SETI@home preferences` for - VLAR on GPU.
In the `Run only the selected applications` section,
that would overide the default seting and so reduce the calls to the servers,
for those like me that can crunch them on ATI (or any other GPU that can handle them)

So we are going to be having this same problem for months eh!!!.

Wow, at least we have an explanation but I am not sure I like the answers.
Couple months of this an I will be ready for the hospital... LOONEY BIN

Michael Miles

So we are going to be having this same problem for months eh!!!.

Wow, at least we have an explanation but I am not sure I like the answers.
Couple months of this an I will be ready for the hospital... LOONEY BIN

Michael Miles