Panic Mode On (111) Server Problems?

Message boards : Number crunching : Panic Mode On (111) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 31 · Next

AuthorMessage
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1926955 - Posted: 28 Mar 2018, 21:22:36 UTC

I woke up this morning to full caches of downloads on both rigs, but hitting the retry button on each soon had everything back to normal.

Cheers.
ID: 1926955 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1926957 - Posted: 28 Mar 2018, 21:35:39 UTC - in response to Message 1926953.  

I guess I could set <http_debug>, but that only gives me more verbose errors?
Yes, but included in the verbosity can be the actual reason for the error - and that, in turn, can guide you to the solution.
ID: 1926957 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1926959 - Posted: 28 Mar 2018, 21:44:18 UTC - in response to Message 1926893.  
Last modified: 28 Mar 2018, 21:57:33 UTC

...Time to relax and let go...LOL...


. . SETI should be playing that big hit from Frozen ... :)

Stephen

PS: when I turned this machine back on this morning, 7:30 am AEDT, Iwas able to report and managed to get a full Q of tasks but then the servers went down again. :(

<shrug>
ID: 1926959 · Report as offensive
Profile JL

Send message
Joined: 15 Apr 00
Posts: 5
Credit: 63,734,518
RAC: 38
United States
Message 1926964 - Posted: 28 Mar 2018, 21:55:24 UTC - in response to Message 1926953.  

Keep getting "transient HTTP error" when my machines try to download work.
The "Backing off" time keeps getting longer.
I'll just wait for the issues to subside, eventually I'll get them downloaded.

I suggest you read more from the recent posts to this thread.

My problem isn't reporting, it's downloading work.
I guess I could set <http_debug>, but that only gives me more verbose errors?


Downloads are working for me now.
Lesson learned ... mention the errors, and they go away :-)
ID: 1926964 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1926968 - Posted: 28 Mar 2018, 22:29:50 UTC - in response to Message 1926953.  

Keep getting "transient HTTP error" when my machines try to download work.
The "Backing off" time keeps getting longer.
I'll just wait for the issues to subside, eventually I'll get them downloaded.

I suggest you read more from the recent posts to this thread.

My problem isn't reporting, it's downloading work.
I guess I could set <http_debug>, but that only gives me more verbose errors?

Only thing I have ever found is to go to the Transfers and keep hitting the Retry button for stuck downloads.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1926968 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1926975 - Posted: 28 Mar 2018, 22:54:22 UTC

Back to unreachable servers again with download backoffs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1926975 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1926977 - Posted: 28 Mar 2018, 23:03:59 UTC - in response to Message 1926975.  

Back to unreachable servers again with download backoffs.


. . It seems they may be experimenting with settings somewhere. Or maybe they are installing the gear for Parkes ? :)

Stephen

? ?
ID: 1926977 · Report as offensive
Don

Send message
Joined: 23 Aug 17
Posts: 7
Credit: 2,418,000
RAC: 6
United States
Message 1926985 - Posted: 28 Mar 2018, 23:48:04 UTC

I also am not able to get an work for SETI@home. I haae two machines that have run out of work for SETI,
ID: 1926985 · Report as offensive
Profile Chris904395093209d Project Donor
Volunteer tester

Send message
Joined: 1 Jan 01
Posts: 112
Credit: 29,923,129
RAC: 6
United States
Message 1926990 - Posted: 29 Mar 2018, 0:41:09 UTC

Just checked my machines, looks like I missed most of the fun this week. My daily driver got 100 new tasks about 90 minutes ago, but noticed a lot of issues during the day. Just 23,000+ tasks ready to send, kind of surprising for this time of day on a Wednesday.
~Chris

ID: 1926990 · Report as offensive
Profile Ghan-buri-Ghan Mike

Send message
Joined: 27 Dec 15
Posts: 123
Credit: 92,602,985
RAC: 172
United States
Message 1926996 - Posted: 29 Mar 2018, 1:19:24 UTC - in response to Message 1926893.  

Definitely let go...and went to work.
Came back 10 hours later and its full steam ahead.
ID: 1926996 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1927006 - Posted: 29 Mar 2018, 2:34:33 UTC - in response to Message 1926996.  

The feeding frenzy at the slop trough is subsiding and the RTS buffer is beginning to build again. Wonder where the steady state level ends up this week. Hope they don't let the splitters run amok again like yesterday.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1927006 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1927021 - Posted: 29 Mar 2018, 4:34:12 UTC

Even after the extended outage, v8 WU Awaiting-deletion hasn't reduced & is now growing even larger. Previous record was 4.43 million & we're at 3.19 million and climbing.
At what point do we run out of disk space again?

And other than the overfull Ready-to-send buffer when the project came back, the splitter output isn't keeping up with demand again. Ready-to-send numbers continue to fall, as does the output from the splitters.
Grant
Darwin NT
ID: 1927021 · Report as offensive
Profile Chris904395093209d Project Donor
Volunteer tester

Send message
Joined: 1 Jan 01
Posts: 112
Credit: 29,923,129
RAC: 6
United States
Message 1927137 - Posted: 29 Mar 2018, 19:34:28 UTC

the results waiting for db purging is at 9.8 million - I think that's the highest I've ever seen it.
~Chris

ID: 1927137 · Report as offensive
bluestar

Send message
Joined: 5 Sep 12
Posts: 6995
Credit: 2,084,789
RAC: 3
Message 1927139 - Posted: 29 Mar 2018, 19:56:07 UTC - in response to Message 1927137.  

Thanks for reminding me, because almost forgot it here.
ID: 1927139 · Report as offensive
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 1927153 - Posted: 29 Mar 2018, 21:05:42 UTC

Hi everyone,

Last night/day after the outage I had to abort an AP download to get my other downloads to advance. The one was stuck and prevented me to download any WUs. (It felt so.) Then, after that, I had 1200 downloads but none of them showed any progress. I tried 'hitting retry' and 'Tools - retry pending transfers' .... With no help respect to the situation. Could not dowload any work!

At some point (in time) everything returned to normal. At that point (of time) I had a shell script running ...

#!/bin/bash

for (( ; ; ))
do

    if ./boinccmd --network_available
    then
	sleep 330
    else
	sleep 1000
    fi
done



I really did not know, and still do not, what made the difference. Everything is as it was before.
I guess I was in an another dimension, time and a place. My machine, however, was out of work.

T.b.c:d
Petri33

Petri.
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1927153 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1927163 - Posted: 29 Mar 2018, 21:25:51 UTC

Thanks for the script Petri, could come in handy.

I've been having sporadic stalled downloads this morning and afternoon. Server unreachable messages as the cause and forces the downloads into backoff. Meanwhile the caches empty fast.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1927163 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1927171 - Posted: 29 Mar 2018, 22:30:42 UTC
Last modified: 29 Mar 2018, 22:31:39 UTC

Just arrived and my cache is allmost empty, the rts buffer is at >500K WU but all requests for new work ends on:

Thu 29 Mar 2018 05:26:50 PM EST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Thu 29 Mar 2018 05:26:54 PM EST | SETI@home | Scheduler request completed: got 0 new tasks
Thu 29 Mar 2018 05:26:54 PM EST | SETI@home | Project has no tasks available

ID: 1927171 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1927173 - Posted: 29 Mar 2018, 22:37:37 UTC

At least the splitters have managed to pick up the pace, but the WUs Awaiting_deletion continue to grow, and it looks as though the MB Returned-per-hour numbers haven't updated for several hours now.
Grant
Darwin NT
ID: 1927173 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1927182 - Posted: 29 Mar 2018, 23:14:48 UTC - in response to Message 1927173.  

All machines getting no tasks are available for 4 of 5 requests. The request that finally does get any work picks up 1-5 tasks. Caches falling fast.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1927182 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1927188 - Posted: 29 Mar 2018, 23:53:12 UTC

Down 300 tasks on one machine. Will be out of gpu work shortly on all machines since nobody is getting any work on request. Just a litany of "no work is available" messages.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1927188 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 31 · Next

Message boards : Number crunching : Panic Mode On (111) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.