Panic Mode On (113) Server Problems?

Message boards : Number crunching : Panic Mode On (113) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 21 · 22 · 23 · 24 · 25 · 26 · 27 . . . 37 · Next

AuthorMessage
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1959489 - Posted: 10 Oct 2018, 10:05:48 UTC - in response to Message 1959488.  
Last modified: 10 Oct 2018, 10:06:41 UTC

I don't know why the outage was so long, nor the recovery so long, but things are still very much borked.

Luck of the draw, I'm guessing, as the SSP says more than 200k units were sent out in the last 40 minutes or so.

Some of the SSP numbers are probably accurate, but others make no sense at all.
In progress should take a dive as work is returned & validated, but it's barely moved, even with all those ready-to-send WUs that have been downloaded. Validators not actually running, even though they say they are? Yet the Assimilator backlog continues to grow, which indicates things are being validated.
Even after the worst of outages in the past, I've been able to get work on at least every second or third request, unlike every 20th (or more) as is happening at the moment.

Things are still very much not right.
Edit- and the forums have back into "go slow" mode.
Grant
Darwin NT
ID: 1959489 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1959523 - Posted: 10 Oct 2018, 14:39:20 UTC

I came to check in and see how things are doing. UG! I see I'm here to commiserate with everyone about the awfulness of it all. I couldn't get my completed WUs to upload in a timely manner, so I've just shutdown communication with seti and will continue to slowly crunch what WUs I have left.

The seti techs have a busy day today sorting out this mess. Hopefully they got a good db clean and backup yesterday and that helps.
ID: 1959523 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1959537 - Posted: 10 Oct 2018, 16:00:01 UTC

could someone bring Beta back online? I still have Beta work on some of my computers that needs reporting!
.

Hello, from Albany, CA!...
ID: 1959537 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1959538 - Posted: 10 Oct 2018, 16:09:37 UTC

my systems seem to be chugging alone pretty normally.

only the stats export seems to be broken. still nothing on BoincStats or FreeDC, but maybe that's a once a day king of thing and we missed it yesterday and just need to wait until sometime later today.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1959538 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 66342
Credit: 55,293,173
RAC: 49
United States
Message 1959543 - Posted: 10 Oct 2018, 17:16:24 UTC

Pulls pin and goes from https to http, guess which works a bit better? http.

Otherwise slow as molasses here, I can cook dinner and be done before Seti is.
Savoir-Faire is everywhere!
The T1 Trust, T1 Class 4-4-4-4 #5550, America's First HST

ID: 1959543 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1959545 - Posted: 10 Oct 2018, 17:18:41 UTC - in response to Message 1959538.  

my systems seem to be chugging alone pretty normally.

only the stats export seems to be broken. still nothing on BoincStats or FreeDC, but maybe that's a once a day king of thing and we missed it yesterday and just need to wait until sometime later today.

Host stats latest is from the 6 October. The User and Team stats are current today.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1959545 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1959551 - Posted: 10 Oct 2018, 17:55:33 UTC - in response to Message 1959545.  

my systems seem to be chugging alone pretty normally.

only the stats export seems to be broken. still nothing on BoincStats or FreeDC, but maybe that's a once a day king of thing and we missed it yesterday and just need to wait until sometime later today.

Host stats latest is from the 6 October. The User and Team stats are current today.


ah good to know!
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1959551 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1959560 - Posted: 10 Oct 2018, 18:30:54 UTC

slowly things are getting better. I managed to report my completed WUs and got some new ones.

The stats page seems to be updating all the info now (bits aren't 14 hours old).
"Results returned and awaiting validation" and "Workunits waiting for assimilation" are both decreasing...very slowly, but still good they are decreasing.

I'm still having hit or miss connection timeouts to the server and these boards.
ID: 1959560 · Report as offensive
Profile lunkerlander
Avatar

Send message
Joined: 23 Jul 18
Posts: 82
Credit: 1,353,232
RAC: 4
United States
Message 1959565 - Posted: 10 Oct 2018, 19:23:11 UTC - in response to Message 1959560.  

I haven't got any seti
tasks in over 24 hours. How about you guys?
ID: 1959565 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1959566 - Posted: 10 Oct 2018, 19:26:24 UTC - in response to Message 1959565.  

Tasks are coming in just fine here.
ID: 1959566 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1959567 - Posted: 10 Oct 2018, 19:28:43 UTC - in response to Message 1959565.  

I finally got through to the servers to report around 4AM PDT this morning. I had full caches by the time I woke up. Reduce the number of task to report in the cc_config.xml file to 50 or 100 tasks and set NNT first and you should be able to report. Once you have reported all your tasks, set to allow tasks again and you should get work.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1959567 · Report as offensive
Profile lunkerlander
Avatar

Send message
Joined: 23 Jul 18
Posts: 82
Credit: 1,353,232
RAC: 4
United States
Message 1959568 - Posted: 10 Oct 2018, 19:35:48 UTC - in response to Message 1959567.  

Ok, I will try that. I allowed Einstein to send new tasks and have been running them since yesterday. I have both seti and Einstein set to send tasks. My plan was to switch back to seti once new tasks appear.
ID: 1959568 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1959570 - Posted: 10 Oct 2018, 19:39:19 UTC - in response to Message 1959568.  

I never let Einstein have free rein on task requests. I always just use a single task request for a slug of work and set NNT to prevent Einstein from commandeering the system away from my other projects.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1959570 · Report as offensive
Profile zoom3+1=4
Volunteer tester
Avatar

Send message
Joined: 30 Nov 03
Posts: 66342
Credit: 55,293,173
RAC: 49
United States
Message 1959597 - Posted: 11 Oct 2018, 0:43:09 UTC

Thanks to a SoGgy app, I was able to put the afterburner on in My Asus 970 turbo, rac's at 6,058 and climbing.
Savoir-Faire is everywhere!
The T1 Trust, T1 Class 4-4-4-4 #5550, America's First HST

ID: 1959597 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13854
Credit: 208,696,464
RAC: 304
Australia
Message 1959658 - Posted: 11 Oct 2018, 5:49:54 UTC
Last modified: 11 Oct 2018, 5:53:36 UTC

MB In-progress & Received-last-hour values still aren't updating, but at least the Assimilator mountain is (very) slowly shrinking.

Edit- and looking at my log there was 30min period of "Project has no tasks available" Scheduler responses an hour ago, so it looks like things are still not quite right on that front either.
Grant
Darwin NT
ID: 1959658 · Report as offensive
Profile lunkerlander
Avatar

Send message
Joined: 23 Jul 18
Posts: 82
Credit: 1,353,232
RAC: 4
United States
Message 1959709 - Posted: 11 Oct 2018, 12:07:03 UTC - in response to Message 1959570.  

I never let Einstein have free rein on task requests. I always just use a single task request for a slug of work and set NNT to prevent Einstein from commandeering the system away from my other projects.


I followed your advice and set Einstein to no new tasks. Then I hit "update" under SETI and new tasks came.
ID: 1959709 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1959749 - Posted: 11 Oct 2018, 15:56:56 UTC - in response to Message 1959747.  
Last modified: 11 Oct 2018, 15:58:11 UTC

Bah, I crunch just a few hours every evening, and nothing is available now.

Every attempt to get tasks just end up with:

SETI@home 2018-10-11 17:43:21 Sending scheduler request: To fetch work.
SETI@home 2018-10-11 17:43:21 Requesting new tasks for NVIDIA GPU
SETI@home 2018-10-11 17:43:27 Scheduler request completed: got 0 new tasks
SETI@home 2018-10-11 17:43:27 Project has no tasks available

Something is not right........again.....or still.....

And Beta totally down.


The same here.

But, RTS buffer shows: Results ready to send 590,336
ID: 1959749 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1959754 - Posted: 11 Oct 2018, 16:15:29 UTC - in response to Message 1959752.  

Bah, I crunch just a few hours every evening, and nothing is available now.

Every attempt to get tasks just end up with:

SETI@home 2018-10-11 17:43:21 Sending scheduler request: To fetch work.
SETI@home 2018-10-11 17:43:21 Requesting new tasks for NVIDIA GPU
SETI@home 2018-10-11 17:43:27 Scheduler request completed: got 0 new tasks
SETI@home 2018-10-11 17:43:27 Project has no tasks available

Something is not right........again.....or still.....

And Beta totally down.


The same here.

But, RTS buffer shows: Results ready to send 590,336

Yes, but that was 40 minutes (and counting) ago..... (As of*)
Agree but a 590 k buffer is expected to hold the request off all of the SETI hosts for hours (about 100-150 k WU per hr)
ID: 1959754 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1959756 - Posted: 11 Oct 2018, 16:27:12 UTC

Falling . . . falling . . . . . falling
The website never fixed the problems with the servers before the outage. And the servers still are not right. Too many tasks not validated. Impossible to look at any of your hosts because the task list never refreshes, you only see the spinner.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1959756 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1959757 - Posted: 11 Oct 2018, 16:35:55 UTC - in response to Message 1959756.  
Last modified: 11 Oct 2018, 16:38:05 UTC

And it is back working again.
Sending tasks, that is.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1959757 · Report as offensive
Previous · 1 . . . 21 · 22 · 23 · 24 · 25 · 26 · 27 . . . 37 · Next

Message boards : Number crunching : Panic Mode On (113) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.