Panic Mode On (78) Server Problems?

Fred,
It appears that the 64 unit reporting limit you were told has already been changed by the staff. If you look at my post http://setiathome.berkeley.edu/forum_thread.php?id=69890&postid=1302985 and Paul's post http://setiathome.berkeley.edu/forum_thread.php?id=69890&postid=1303071 it would appear that they have now set the limit to 100 CPU, 100 GPU and 100 reports, or, probably, 100 on everything.

Sadly, this is not going to fix what is obviously a different problem, as the system has proven for a long time that it can handle far larger numbers of uploads, downloads, and caches per host. As most of us know, this whole problem has been recently introduced and will require a specific fix. Limiting our hosts only compounds the problem by forcing our hosts to hit the servers far more often.

I wish Matt were back, as this is his expertise (I know this to be true from my extensive, personally conducted tour of SETI and visit with him not too long ago). He would be able to track down the problem faster and patch things up quicker. Furthermore, there is also just too much work for the remaining staff to tackle problems quickly, and we are beginning to see the results of this.

Oh, for a government that would realize the importance of and would fund true scientific research, or, should I say, true scientific research that does not lead to more powerful weapons and surveillance technology!

But I digress... If you only knew what I knew....... (that almost sounds like lyrics to a song, lol!)

I believe others have been reporting the scheduler problem -- here's what I see.

No problems with uploads.

No problems with reporting *IF* I have set no new work.

Big problem with reporting and getting work if I have not set no new work. The scheduler goes into suspended animation -- it takes 50 to 10 minutes for the scheduler to time out and release that action. The report back is 'Timeout was reached, servers may be down. They are not, but the scheduler is in trouble as it has been for the past week.

My approach at this point, pending some confirmation back at the shop that this problem -- which others have reported from what I've read -- and that a fix is in view is simply to have all my SETI systems configured for no new work and let them complete and clear out and have other projects pick up the slack.

I am certain that once folks acknowledge the problem and work on it, we'll get some information regarding anticipated resolution.

I would note I seen this problem during the past week or so on multiple systems running multiple different versions of the BOINC software.

It appears I don't have a cc_config.xml on my system already. I believe it goes in programdata\boinc (I'm running Win 7 64-bit) but it didn't seem to help (yes I shut the BOINC Manager down and restarted it). I'll paste the cc_config.xml below to get an opinion if I built it right.

<cc_config>
<options>
<max_tasks_reported>100</max_tasks_reported>
</options>
</cc_config>

That looks okay - and yes, it belongs on the top level data dirctory, not one of the project directories or the program directory. Scheduler was modified a few weeks ago to accept a max of 64, so there's little point in using higher values unless you run other projects and need it there.

Project is running inconsistently. Looking at my log for last 8 hours, I see failure to connect, timeouts, no tasks available, over the limit messages, and I got 18 cpu tasks overnight. I couldn't get any for 24 hours before that. I'm still below limit for cpu work and down to 99 cpu tasks for 6 cores. Could be a problem here, so I'm looking for corroboration that Scheduler is refusing you when you're certain that you are below the limit (50/cpu core and 400/gpu). Hard to tell with the variety of responses and connection issues.

I don't want to have it extend the time limit -- I have other projects going and when the scheduler does this, it preempts other projects reporting for the duration. I'm hoping that the root cause is acknowledged, identified and dealt with -- until then, I'm completing SETI work on hand and getting and processing work from other projects.

The root cause of the problem is, obviously, the server taking too long to work out which tasks to send and assemble them into a suitable reply to your request. But there's no simple switch on the server to say 'extend the time limit': we'll just have to wait for them to uncover the root cause, and fix that instead.

It would seem that these server troubles started right when the New York disaster took place.

I am not sure if this is a direct problem to what is going on with the scheduler but I hope it gets fixed very soon.
The limits imposed this time around make it very hard as just reporting finished tasks is a problem as well as getting enough work to feed 6 cpu cores and one GTX 460 for one day when I have am supposed to have enough work to keep all happy for 3 days according to my prefs settings.

I just had a thought, but I don't know how feasible it is or whether it might help anything.

Would it help if they turn off downloads for five minutes every hour or half hour to allow uploads, reports, and miscellaneous traffic to get through unimpeded? They'd probably have to even stop ghost resends for this to work, but if it works it should dramatically reduce the number of ghosts after a few days.

edit: Even better would be to turn off uploads except during that five minute period, but to have any real effect on the network it would probably have to be a mod to the client software so it wouldn't try to send when it shouldn't.
____________
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

11/7/2012 2:32:31 PM
SETI@home
Not requesting tasks: some download is stalled


OK, I can kinda understand setting limits. That way more people can get work.
But the above I just don't get. Now you need to babysit your systems to make
sure all your downloads run as they should, or you can not have any work? I
am really starting to understand why people are becoming upset. This was
supposed to be a set and forget system wasn't it?

I guess I never got the old messages. It just upset me last night before I
went to work I had about 4 hours worth of cache built up. Thought that was
good. It would get more work as it ran. I guess as soon as I left, one
stalled. So when I checked after I got home, nothing. Aborted that download
and started over. Now I got it again. Frustrating!!It just seems like such a
waste to leave my systems on and not have any work because I can not babysit
them 24/7

I guess I never got the old messages...

I guess I never got the old messages...

You couldn't get the old messages, because there haven't been any (as Richard just stated ;-).

Gruß,
Gundolf

Just let me know if that project will ever working again... ATM I'am sick of it
____________

:)...This reminds me of a song from HeeHaw.

Gloom despair and agony on me
Deep dark depression excessive misery.