Panic Mode On (95) Server Problems?

Message boards : Number crunching : Panic Mode On (95) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 14 · 15 · 16 · 17 · 18 · 19 · 20 . . . 22 · Next

AuthorMessage
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1646206 - Posted: 25 Feb 2015, 4:09:54 UTC

Oscar shows as down.
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1646206 · Report as offensive
Dr Who Fan
Volunteer tester
Avatar

Send message
Joined: 8 Jan 01
Posts: 3207
Credit: 715,342
RAC: 4
United States
Message 1646221 - Posted: 25 Feb 2015, 5:34:28 UTC - in response to Message 1646186.  

Carolyn,forums & associated database services keep timing out or unavailable ... just came back up after being out for 20-30 minutes.
Too much mysql i/o load from the "fixed" AstroPulse database merger?
ID: 1646221 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 1646249 - Posted: 25 Feb 2015, 6:31:46 UTC - in response to Message 1646181.  
Last modified: 25 Feb 2015, 6:42:51 UTC

Is anybody managing to make contact with the scheduler?

Rarely.

If the Server status page is halfway accurate, there's a ton of MB work waiting to be downloaded. However with 1 download server still off line, and 90% or more of Scheduler requests failing (They're all there- Couldn't connect to server, HTTP internal server error, Failure when receiving data from the peer), it looks like i'll be out of work in a few more hours.


EDIT- and when I do finally connect, the response- Project has no tasks available.
Given the length of the outage, even if the Scheduler were working properly that would be the most likely response for several hours as everyone hits the feeder at the same time.
Grant
Darwin NT
ID: 1646249 · Report as offensive
Kevin Olley

Send message
Joined: 3 Aug 99
Posts: 906
Credit: 261,085,289
RAC: 572
United Kingdom
Message 1646263 - Posted: 25 Feb 2015, 6:50:25 UTC - in response to Message 1646249.  

Is anybody managing to make contact with the scheduler?

Rarely.



Things are starting to improve, have been able to make contact with server a couple of times and reported finnished tasks, now getting no tasks available.
Kevin


ID: 1646263 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 5375
Credit: 30,870,693
RAC: 1
United States
Message 1646266 - Posted: 25 Feb 2015, 6:57:40 UTC - in response to Message 1646263.  

Is anybody managing to make contact with the scheduler?

Rarely.



Things are starting to improve, have been able to make contact with server a couple of times and reported finnished tasks, now getting no tasks available.

Ditto here, was just able to upload 58 reports by manual update, but no tasks available. It's dead for the time being. Hopefully heralding the return of AP.

:D .g

"Sour Grapes make a bitter Whine." <(0)>
ID: 1646266 · Report as offensive
BONNSaR

Send message
Joined: 9 Nov 04
Posts: 38
Credit: 21,538,589
RAC: 9
Australia
Message 1646270 - Posted: 25 Feb 2015, 7:05:50 UTC - in response to Message 1646266.  

Just did a Reset Project then manual Update and received 39 MB tasks
ID: 1646270 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 1646287 - Posted: 25 Feb 2015, 8:01:06 UTC - in response to Message 1646270.  

Well, something's still borked.
The Scheduler is now responding to requests, but it's almost always "Project has no tasks available". Network traffic is way down, so it's not due to the limitations of the feeder- something else is stopping the work from flowing.
Grant
Darwin NT
ID: 1646287 · Report as offensive
Profile S@NL Etienne Dokkum
Volunteer tester
Avatar

Send message
Joined: 11 Jun 99
Posts: 212
Credit: 43,822,095
RAC: 0
Netherlands
Message 1646290 - Posted: 25 Feb 2015, 8:03:14 UTC

It took abusive retry button behavior, but now the servers were friendly and my cache is back up to 200 tasks again. No point of holding out for AP's just yet I think...
ID: 1646290 · Report as offensive
Phil Burden

Send message
Joined: 26 Oct 00
Posts: 264
Credit: 22,303,899
RAC: 0
United Kingdom
Message 1646297 - Posted: 25 Feb 2015, 8:25:16 UTC - in response to Message 1646287.  

Well, something's still borked.
The Scheduler is now responding to requests, but it's almost always "Project has no tasks available". Network traffic is way down, so it's not due to the limitations of the feeder- something else is stopping the work from flowing.


Not only that, the Ready To Send buffer is showing 900K+ units waiting to be dispatched, a figure I've never seen before, it's usually around the 2-300K mark.

P.
ID: 1646297 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13727
Credit: 208,696,464
RAC: 304
Australia
Message 1646328 - Posted: 25 Feb 2015, 10:02:35 UTC - in response to Message 1646297.  

And now we're back to Scheduler errors again.
Grant
Darwin NT
ID: 1646328 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1646340 - Posted: 25 Feb 2015, 10:29:37 UTC - in response to Message 1646328.  

And now we're back to Scheduler errors again.

Intermittently. What did I call it a few days ago - stumble, limp, stumble, stumble, limp?

The crickets are chirping (fitfully), and RTS has gone down to below 700K - so somebody's getting some work, just not me at the moment. Are there a lot of VLARs around?
ID: 1646340 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1646442 - Posted: 25 Feb 2015, 14:27:35 UTC

I'm not seeing a whole lot of errors on my end. Tuesday has a higher error count, but that just has to do with the everything being down for maintenance.
Personally I'm nor worried about the "HTTP gateway timeout" that I get on this machine at work. As those occur sometimes due to the nature of the MS TMG server being used. In that it sometimes decides I have to many connections open and blocks all traffic intermittently. I would have thought setting the limit to 10,000 would be enough...

As of 2015-02-25 14:20 UTC
Showing result for date: 2015-02-25
Scheduler Request Count: 26
Scheduler Success Count: 24,    92 % of requests
Scheduler Failure Count: 2,     7 % of requests

Failure Details:
__________________________________________________________
Description                     Count   Total % Failure %
"Timeout was reached":          0       0 %     0 %
"Couldn't connect to server":   0       0 %     0 %
"Couldn't resolve host name":   0       0 %     0 %
"Receiving data from the peer": 0       0 %     0 %
"HTTP gateway timeout":         2       7 %     100 %
"HTTP internal server error":   0       0 %     0 %
"HTTP service unavailable":     0       0 %     0 %

Showing result for date: 2015-02-24
Scheduler Request Count: 73
Scheduler Success Count: 54,    73 % of requests
Scheduler Failure Count: 19,    26 % of requests

Failure Details:
__________________________________________________________
Description                     Count   Total % Failure %
"Timeout was reached":          0       0 %     0 %
"Couldn't connect to server":   0       0 %     0 %
"Couldn't resolve host name":   0       0 %     0 %
"Receiving data from the peer": 0       0 %     0 %
"HTTP gateway timeout":         6       8 %     31 %
"HTTP internal server error":   10      13 %    52 %
"HTTP service unavailable":     3       4 %     15 %

Showing result for date: 2015-02-23
Scheduler Request Count: 70
Scheduler Success Count: 69,    98 % of requests
Scheduler Failure Count: 1,     1 % of requests

Failure Details:
__________________________________________________________
Description                     Count   Total % Failure %
"Timeout was reached":          0       0 %     0 %
"Couldn't connect to server":   0       0 %     0 %
"Couldn't resolve host name":   0       0 %     0 %
"Receiving data from the peer": 0       0 %     0 %
"HTTP gateway timeout":         1       1 %     100 %
"HTTP internal server error":   0       0 %     0 %
"HTTP service unavailable":     0       0 %     0 %

Showing result for date: 2015-02-22
Scheduler Request Count: 73
Scheduler Success Count: 71,    97 % of requests
Scheduler Failure Count: 2,     2 % of requests

Failure Details:
__________________________________________________________
Description                     Count   Total % Failure %
"Timeout was reached":          0       0 %     0 %
"Couldn't connect to server":   0       0 %     0 %
"Couldn't resolve host name":   0       0 %     0 %
"Receiving data from the peer": 0       0 %     0 %
"HTTP gateway timeout":         1       1 %     50 %
"HTTP internal server error":   0       0 %     0 %
"HTTP service unavailable":     1       1 %     50 %

Showing result for date: 2015-02-21
Scheduler Request Count: 71
Scheduler Success Count: 70,    98 % of requests
Scheduler Failure Count: 1,     1 % of requests

Failure Details:
__________________________________________________________
Description                     Count   Total % Failure %
"Timeout was reached":          0       0 %     0 %
"Couldn't connect to server":   0       0 %     0 %
"Couldn't resolve host name":   0       0 %     0 %
"Receiving data from the peer": 0       0 %     0 %
"HTTP gateway timeout":         0       0 %     0 %
"HTTP internal server error":   0       0 %     0 %
"HTTP service unavailable":     1       1 %     100 %

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1646442 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1646521 - Posted: 25 Feb 2015, 17:24:00 UTC

Yes, it was a rocky start after the extended outage, but the kitties have managed to refill all the crunchers' caches to the brim.
So, other than our beloved AP not coming back online as originally planned, all is going well here for now.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1646521 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1646572 - Posted: 25 Feb 2015, 18:37:13 UTC - in response to Message 1646558.  

Ahh yes, AP....
Gone but not forgotten.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1646572 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 5375
Credit: 30,870,693
RAC: 1
United States
Message 1646582 - Posted: 25 Feb 2015, 18:53:26 UTC - in response to Message 1646558.  

Ah well, I will forget that we ever had anything called AP, at least until around midsummer (northern hemisphere)

Until then, I will continue to crunch MB, and picking lint out of my belly button. :-)

The lint picking seems to be epidemic.........

"Sour Grapes make a bitter Whine." <(0)>
ID: 1646582 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1646648 - Posted: 25 Feb 2015, 21:15:59 UTC - in response to Message 1646572.  

Ahh yes, AP....
Gone but not forgotten.

I have 174 pending, how many do you have?
ID: 1646648 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1646663 - Posted: 25 Feb 2015, 21:55:48 UTC

Hey - my 5 AP6 WUs that were Pending are now Validated. Haven't looked lately, so I don't know exactly when they were V'd, but it has been very recently. Might that mean we are going to be making some progress soon on all the other AP issues? (I have 439 AP7 Pending).
ID: 1646663 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 5375
Credit: 30,870,693
RAC: 1
United States
Message 1646667 - Posted: 25 Feb 2015, 22:01:55 UTC - in response to Message 1646663.  

Pending AstroPulse v6 (4)

AstroPulse v7 (692)

No recent change.

"Sour Grapes make a bitter Whine." <(0)>
ID: 1646667 · Report as offensive
Profile Fawkesguy
Volunteer tester
Avatar

Send message
Joined: 8 Jan 01
Posts: 108
Credit: 188,578,766
RAC: 0
United States
Message 1646670 - Posted: 25 Feb 2015, 22:03:54 UTC

Pending AP v7 (1308)
ID: 1646670 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 1646680 - Posted: 25 Feb 2015, 22:15:06 UTC
Last modified: 25 Feb 2015, 22:18:34 UTC

AstroPulse v6 - Validation pending (8)

My PC sent the 3 result, since ...
8 Mar 2014
20 Apr 2014
26 Jun 2014
29 Jun 2014
...
- stuck in pending.

I guess a manual script run is needed ...


AstroPulse v7 - Validation pending (102)
ID: 1646680 · Report as offensive
Previous · 1 . . . 14 · 15 · 16 · 17 · 18 · 19 · 20 . . . 22 · Next

Message boards : Number crunching : Panic Mode On (95) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.