Panic Mode On (59) Server problems?

Message boards : Number crunching : Panic Mode On (59) Server problems?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 12 · Next

AuthorMessage
Profile soft^spirit
Avatar

Send message
Joined: 18 May 99
Posts: 6497
Credit: 34,134,168
RAC: 0
United States
Message 1162729 - Posted: 15 Oct 2011, 19:55:52 UTC - in response to Message 1162710.  

Time to raise the kibble limits!

Meow meow meow!


+1 More popcorn please!!!


Janice
ID: 1162729 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1162964 - Posted: 16 Oct 2011, 18:38:15 UTC - in response to Message 1162729.  


Looks like trouble ahead.
The MB Assimilator queue has started climbing again, and the MB & AP Ready to Send buffers are shrinking- and the splitters haven't cranked it up to bring them back to their usual levels.
Couple more hours & we'll be out of work.
Grant
Darwin NT
ID: 1162964 · Report as offensive
Profile S@NL Etienne Dokkum
Volunteer tester
Avatar

Send message
Joined: 11 Jun 99
Posts: 212
Credit: 43,822,095
RAC: 0
Netherlands
Message 1162966 - Posted: 16 Oct 2011, 18:55:56 UTC - in response to Message 1162682.  


That is a tough nut to crack I believe. We need a volunteer with a serious button finger, pushing every damned AP WU through the validator manually, while at the same time being able to logical thinking (is this one some WU that should be validated against the wingman, or sent out again?)

Great job that one, isn't it? LOL

Edit: The person volunteering should get 10% of all credit he pushed through.
:-)


Volunteering then ??? If they fed-ex me the server overnight I'm in too ;-)
ID: 1162966 · Report as offensive
Mikey
Avatar

Send message
Joined: 15 May 99
Posts: 15
Credit: 4,504,202
RAC: 0
Canada
Message 1162970 - Posted: 16 Oct 2011, 19:03:39 UTC

I've been unable to upload for that last 24 hours or so. Keep getting the

16/10/2011 3:02:17 PM Internet access OK - project servers may be temporarily down.

message. Anybody else?
ID: 1162970 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1162971 - Posted: 16 Oct 2011, 19:06:03 UTC - in response to Message 1162970.  

I've been unable to upload for that last 24 hours or so. Keep getting the

16/10/2011 3:02:17 PM Internet access OK - project servers may be temporarily down.

message. Anybody else?

Checkout the "HE connection problems thread".
Grant
Darwin NT
ID: 1162971 · Report as offensive
Profile S@NL Etienne Dokkum
Volunteer tester
Avatar

Send message
Joined: 11 Jun 99
Posts: 212
Credit: 43,822,095
RAC: 0
Netherlands
Message 1162973 - Posted: 16 Oct 2011, 19:15:41 UTC - in response to Message 1162970.  

I've been unable to upload for that last 24 hours or so. Keep getting the

16/10/2011 3:02:17 PM Internet access OK - project servers may be temporarily down.

message. Anybody else?


just stopped using a proxy ever since this problem first occured. Had few problems ever since uploading... Just an idea ???

DL limits on WU's , AP validation etc. completely different thing ofcourse, but everybody experiencing these minor glitches...
ID: 1162973 · Report as offensive
Dave Stegner
Volunteer tester
Avatar

Send message
Joined: 20 Oct 04
Posts: 540
Credit: 65,583,328
RAC: 27
United States
Message 1163039 - Posted: 16 Oct 2011, 22:29:14 UTC

AP ready to send just slid to 0 and it looks like MB will be there in a few hours.

Dave

ID: 1163039 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1163041 - Posted: 16 Oct 2011, 22:33:19 UTC - in response to Message 1163039.  

AP ready to send just slid to 0 and it looks like MB will be there in a few hours.

Well, we made it through MOST of the weekend...
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1163041 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1163085 - Posted: 17 Oct 2011, 2:44:03 UTC

Hold onto your caches, kitties.
Ready to send on both AP and MB now down to zero, so work requests will be more miss than hit for awhile.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1163085 · Report as offensive
Profile soft^spirit
Avatar

Send message
Joined: 18 May 99
Posts: 6497
Credit: 34,134,168
RAC: 0
United States
Message 1163099 - Posted: 17 Oct 2011, 3:21:27 UTC

looks like plenty ready to crunch.. do you think they might be running it lean in order to bump up the limits?


Janice
ID: 1163099 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1163191 - Posted: 17 Oct 2011, 13:24:16 UTC - in response to Message 1163099.  

looks like plenty ready to crunch.. do you think they might be running it lean in order to bump up the limits?


No new limits yet, but I am pleasantly surprised how well the caches maintained overnight since WU production slowed.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1163191 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1163258 - Posted: 17 Oct 2011, 17:45:41 UTC

MB splitters kicked back on and RRTS is sitting around it's cut-off point of ~250k. AP is still zero and all of those splitters are not running. Maybe whatever is causing the validator to fall on its face is affecting the rest of the AP operation finally. I would have to imagine the WU storage is getting pretty full holding all of those "waiting for validation" tasks.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1163258 · Report as offensive
MikeN

Send message
Joined: 24 Jan 11
Posts: 319
Credit: 64,719,409
RAC: 85
United Kingdom
Message 1163319 - Posted: 17 Oct 2011, 21:23:58 UTC

looks like the SETI people may be trying to fix the APvalidate problem. The AP splitters have been 'not running' for a few hours now, presumably to stop more AP units going out until the problem is fixed. In the last few minutes I have seen APvalidate3 go from 'not running' to 'running' and currently all 3 APvalidators are 'disabled'. With a bit of luck, by the time I get up in the morning we may have a temporary or even permanent fix and a massive amount of credit being allocated for the backlog.
ID: 1163319 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 1163322 - Posted: 17 Oct 2011, 22:13:09 UTC - in response to Message 1163319.  
Last modified: 17 Oct 2011, 22:24:19 UTC

Looking @ the SERVER Status Page, I'm seeing

ap_validate1 synergy Disabled
ap_validate2 synergy Disabled
ap_validate3 synergy Disabled
also,
ap_splitter1 synergy Not Running
ap_splitter2 lando Not Running
ap_splitter3 lando Not Running
ap_splitter4 vader Not Running
ap_splitter5 vader Not Running
ap_splitter6 vader Not Running

As of 17 Oct 2011 | 21:50:07 UTC (; 23:50:07 local).

Hope they can fix this, well tomorrow is Maintenance Day, but can not complain
as I've been served well, lately, don't even have Pending Work.....(A few, I think).
I do miss AstroPulse, also it's another (BROADBAND) and (<1 micro-second) pulse find, search!
And the same 1.4xx.xxx.xxx Hz or 1.4GHz.
(It probably also could find rather slow spinning neutron stars)?

ID: 1163322 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8964
Credit: 12,678,685
RAC: 0
United States
Message 1163323 - Posted: 17 Oct 2011, 22:20:36 UTC

Maybe I've just been lucky but I've had work steadily for a cpl weeks now :)


ID: 1163323 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 1163324 - Posted: 17 Oct 2011, 22:28:15 UTC - in response to Message 1163323.  
Last modified: 17 Oct 2011, 22:38:03 UTC

Hi Blurf, long time no see.
No, work enough, never been out of, MB, only AstroPulse has some kind of problem,
which surely gets fixed :)
And those connection issues, sometimes I see an occasional 17-10-2011 6:19:14 | SETI@home | Temporarily failed upload of 19se11ab.29212.476.6.10.103_1_0: can't resolve hostname
17-10-2011 9:10:01 | SETI@home | Started upload of 24se11ac.28427.4566.14.10.136_0_0
17-10-2011 9:10:07 | SETI@home | Finished upload of 24se11ac.28427.4566.14.10.136_0_0
17-10-2011 9:37:35 | SETI@home | Sending scheduler request: To fetch work.
17-10-2011 9:37:35 | SETI@home | Requesting new tasks for CPU
17-10-2011 9:37:38 | SETI@home | Scheduler request completed: got 1 new tasks
17-10-2011 9:37:40 | SETI@home | Started download of 09se11ac.20899.11519.14.10.245
17-10-2011 9:37:48 | SETI@home | Finished download of 09se11ac.20899.11519.14.10.245

But it's a matter of seconds sometimes minutes, or 8 hours, one more request starts
the 'time-cycle', changes back to 4 seconds. All 5 rigs have BOINC 6.12.34. This is
my LT, only use the (T2400) CPU. (Dual-Core).
ID: 1163324 · Report as offensive
Profile SciManStev Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Jun 99
Posts: 6658
Credit: 121,090,076
RAC: 0
United States
Message 1163335 - Posted: 17 Oct 2011, 23:53:07 UTC

AP units are now clearing at a very fast rate. What ever was stuck, must be free now.

Steve
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website
ID: 1163335 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1163345 - Posted: 18 Oct 2011, 0:57:52 UTC - in response to Message 1163335.  
Last modified: 18 Oct 2011, 1:20:29 UTC

AP units are now clearing at a very fast rate. What ever was stuck, must be free now.

Steve

Hot damn. I'm going to be busy for an hour or so adding all of these granted credits to my spreadsheet. Also, look at that RAC!

Edit: And it only took 14 minutes. Guess I'm getting faster at this. Also, the validation storm gave me 34,612 credits.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1163345 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1163350 - Posted: 18 Oct 2011, 1:44:05 UTC

My RAC just jumped up 4000 from when I saw it earlier today.

Most of that was from validating AP work.

ID: 1163350 · Report as offensive
MikeN

Send message
Joined: 24 Jan 11
Posts: 319
Credit: 64,719,409
RAC: 85
United Kingdom
Message 1163403 - Posted: 18 Oct 2011, 7:14:56 UTC - in response to Message 1163319.  
Last modified: 18 Oct 2011, 7:22:12 UTC

All my dreams came true last night! Got up this morning and 40 APs had validated overnight causing my RAC to jump back to something more reasonable:))

Thanks you for spending most of Monday fixing this problem guys. How about a brief note to let us know what the problem and solution was. Nothing detiled, just a 5 minute message when you get in on Tuesday would be great.

Interestingly, looking down the list of APs still pending, all but two are waiting for wingmen to complete. The two that are not are both cases where two people had already been allocate credit and my result came in third (but before my deadline)

http://setiathome.berkeley.edu/workunit.php?wuid=794483682
http://setiathome.berkeley.edu/workunit.php?wuid=794481401

looks like they may have found a way of validating the simple cases, but have left more complicated issues like this.
ID: 1163403 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 12 · Next

Message boards : Number crunching : Panic Mode On (59) Server problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.