Panic Mode On (115) Server Problems?

Message boards : Number crunching : Panic Mode On (115) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 27 · 28 · 29 · 30 · 31 · Next

AuthorMessage
Eric Korpela Project Donor
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 3 Apr 99
Posts: 1381
Credit: 54,506,847
RAC: 60
United States
Message 1987462 - Posted: 27 Mar 2019, 17:33:48 UTC - in response to Message 1987457.  
Last modified: 27 Mar 2019, 17:36:21 UTC

We slowly climbing out of the ditch, but we still don't have a cause. It started suddenly at 7am Monday, so we're guessing that there was a change in hardware/software/DNS or routing at that point. We've got a trouble ticket in with campus networking and they are looking into it.

(My workaround was just to not let connection attempts sit in the local queues for long periods of time. Quick drops are often much better than those that hang around and prevent other connections.
That seems to have fixed the log jam, but there may still be people who can't connect.)
@SETIEric@qoto.org (Mastodon)

ID: 1987462 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1987464 - Posted: 27 Mar 2019, 18:04:09 UTC - in response to Message 1987462.  

Hello Eric, It's good to see you in here :)
I was looking a little bit ago, and the computer I seem to be having the most problem with doesn't have static DNS entries.
The other only drop a few connections, but eventually they go through on their own.
Hopefully they can sort it out.
ID: 1987464 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1987469 - Posted: 27 Mar 2019, 19:06:00 UTC - in response to Message 1987464.  
Last modified: 27 Mar 2019, 19:09:57 UTC

Thanks for the update Eric. That seems to be the case with my hosts. If the download request doesn't go through immediately, it drops out to a backoff.

[Edit] After looking at http_debug, I'm not sure if there is any DNS issue.

Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Info: Connected to boinc2.ssl.berkeley.edu (208.68.240.127) port 80 (#51041)
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: GET /sah/download_fanout/bc/26mr19ac.2101.22971.6.33.215 HTTP/1.1
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Host: boinc2.ssl.berkeley.edu
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: User-Agent: BOINC client (x86_64-pc-linux-gnu 7.15.0)
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Accept: */*
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Accept-Encoding: deflate, gzip
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Accept-Language: en_US
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server:
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42639] Info: Empty reply from server
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42639] Info: Connection #51040 to host boinc2.ssl.berkeley.edu left intact
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] HTTP error: Server returned nothing (no headers, no data)
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Info: Empty reply from server
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Info: Connection #51041 to host boinc2.ssl.berkeley.edu left intact
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] HTTP error: Server returned nothing (no headers, no data)
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Temporarily failed download of 26mr19ac.12199.21335.15.42.201: transient HTTP error
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Backing off 00:02:48 on download of 26mr19ac.12199.21335.15.42.201
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Temporarily failed download of 26mr19ac.2101.22971.6.33.215: transient HTTP error
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Backing off 00:02:00 on download of 26mr19ac.2101.22971.6.33.215
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1987469 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1987479 - Posted: 27 Mar 2019, 20:22:15 UTC

Seems to come and go. Stuck downloads for a while on all hosts that eventually clear. Then a period of no download issues. Then back to stalled downloads and backoffs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1987479 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1987498 - Posted: 27 Mar 2019, 21:45:57 UTC - in response to Message 1987464.  

Hello Eric, It's good to see you in here :)
I was looking a little bit ago, and the computer I seem to be having the most problem with doesn't have static DNS entries.
The other only drop a few connections, but eventually they go through on their own.
Hopefully they can sort it out.


Eric does know to look for the most recent Panic Mode thread, he is just a very infrequent poster.

ID: 1987498 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14531
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1987499 - Posted: 27 Mar 2019, 22:36:39 UTC - in response to Message 1987498.  

Eric does know to look for the most recent Panic Mode thread, he is just a very infrequent poster.
The Panic Mode threads started in a light-hearted way, and it's good to keep them that way. But they do serve a valuable purpose, which is one reason why I try to pounce on demonstrably false 'information' posted in them.
ID: 1987499 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5516
Credit: 528,817,460
RAC: 242
United States
Message 1987512 - Posted: 27 Mar 2019, 23:45:24 UTC - in response to Message 1987469.  

Thanks for the update Eric. That seems to be the case with my hosts. If the download request doesn't go through immediately, it drops out to a backoff.

[Edit] After looking at http_debug, I'm not sure if there is any DNS issue.

Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Info: Connected to boinc2.ssl.berkeley.edu (208.68.240.127) port 80 (#51041)
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: GET /sah/download_fanout/bc/26mr19ac.2101.22971.6.33.215 HTTP/1.1
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Host: boinc2.ssl.berkeley.edu
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: User-Agent: BOINC client (x86_64-pc-linux-gnu 7.15.0)
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Accept: */*
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Accept-Encoding: deflate, gzip
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server: Accept-Language: en_US
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Sent header to server:
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42639] Info: Empty reply from server
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42639] Info: Connection #51040 to host boinc2.ssl.berkeley.edu left intact
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] HTTP error: Server returned nothing (no headers, no data)
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Info: Empty reply from server
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] [ID#42640] Info: Connection #51041 to host boinc2.ssl.berkeley.edu left intact
Wed 27 Mar 2019 12:07:00 PM PDT | SETI@home | [http] HTTP error: Server returned nothing (no headers, no data)
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Temporarily failed download of 26mr19ac.12199.21335.15.42.201: transient HTTP error
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Backing off 00:02:48 on download of 26mr19ac.12199.21335.15.42.201
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Temporarily failed download of 26mr19ac.2101.22971.6.33.215: transient HTTP error
Wed 27 Mar 2019 12:07:01 PM PDT | SETI@home | Backing off 00:02:00 on download of 26mr19ac.2101.22971.6.33.215



I knew it...It was you Keith!!! bahahahaha....
ID: 1987512 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1987513 - Posted: 27 Mar 2019, 23:52:54 UTC
Last modified: 27 Mar 2019, 23:53:19 UTC

I knew it...It was you Keith!!! bahahahaha....

Ignoring you Z-man. Go back to your mojito. LOL.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1987513 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1987544 - Posted: 28 Mar 2019, 2:51:37 UTC

So anybody else still getting instant backoffs when tasks are requested and acknowledged? Been that way most of the day.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1987544 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 17847
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1987548 - Posted: 28 Mar 2019, 3:46:50 UTC - in response to Message 1987544.  

So anybody else still getting instant backoffs when tasks are requested and acknowledged? Been that way most of the day.

No, received 20 tasks about 10 minutes age, with no problems.
ID: 1987548 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1987552 - Posted: 28 Mar 2019, 4:07:39 UTC - in response to Message 1987544.  
Last modified: 28 Mar 2019, 4:17:12 UTC

So anybody else still getting instant backoffs when tasks are requested and acknowledged? Been that way most of the day.
I'm still getting them on All 4 machines, around a dozen backed off tasks at a time. When you go to clear them you have to hit the Update button multiple times in succession because the tasks go right back into project Backoff as soon as you hit the Update button. After about a dozen hits they are all downloaded...until the next time.

Thu 28 Mar 2019 12:10:35 AM EDT | SETI@home | Sending scheduler request: To report completed tasks.
Thu 28 Mar 2019 12:10:35 AM EDT | SETI@home | Reporting 5 completed tasks
Thu 28 Mar 2019 12:10:35 AM EDT | SETI@home | Not requesting tasks: some download is stalled
Thu 28 Mar 2019 12:10:35 AM EDT | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
Thu 28 Mar 2019 12:10:35 AM EDT | SETI@home | [sched_op] NVIDIA GPU work request: 0.00 seconds; 0.00 devices
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | Scheduler request completed
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc23_2bit_guppi_58406_03928_HIP116971_0040.29513.818.21.44.35.vlar_1
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc23_2bit_guppi_58406_04592_HIP116971_0042.29106.818.21.44.27.vlar_1
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc23_2bit_guppi_58406_04592_HIP116971_0042.29106.818.21.44.44.vlar_1
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc23_2bit_guppi_58406_20894_HIP20019_0085.29167.818.21.44.3.vlar_0
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc23_2bit_guppi_58406_04592_HIP116971_0042.29106.818.21.44.54.vlar_1
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | [sched_op] Deferring communication for 00:05:03
Thu 28 Mar 2019 12:10:36 AM EDT | SETI@home | [sched_op] Reason: requested by project
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.150
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.109
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:01 AM EDT | SETI@home | Started download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.150: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:04:08 on download of 26mr19ad.6346.237521.7.34.150
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.23220.232204.10.37.185: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:05:43 on download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.208: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:04:46 on download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.206: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:06:30 on download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.9728.235169.11.38.38: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:04:12 on download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.113: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:04:39 on download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.109: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:04:02 on download of 26mr19ad.6346.237521.7.34.109
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.23220.232204.10.37.194: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:04:16 on download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.14520.235476.14.41.221: transient HTTP error
Thu 28 Mar 2019 12:11:02 AM EDT | SETI@home | Backing off 00:04:05 on download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.150
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.109
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:04 AM EDT | SETI@home | Started download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.150: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:12:42 on download of 26mr19ad.6346.237521.7.34.150
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.23220.232204.10.37.185: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:14:49 on download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.208: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:11:01 on download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.206: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:08:46 on download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.9728.235169.11.38.38: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:10:28 on download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.113: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:12:29 on download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.23220.232204.10.37.194: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:12:26 on download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.14520.235476.14.41.221: transient HTTP error
Thu 28 Mar 2019 12:11:05 AM EDT | SETI@home | Backing off 00:12:25 on download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Finished download of 26mr19ad.6346.237521.7.34.109
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.150
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:07 AM EDT | SETI@home | Started download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.23220.232204.10.37.185: transient HTTP error
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Backing off 00:22:17 on download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.208: transient HTTP error
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Backing off 00:17:58 on download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.206: transient HTTP error
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Backing off 00:24:15 on download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.9728.235169.11.38.38: transient HTTP error
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Backing off 00:26:05 on download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.113: transient HTTP error
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Backing off 00:27:10 on download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.23220.232204.10.37.194: transient HTTP error
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Backing off 00:27:21 on download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.14520.235476.14.41.221: transient HTTP error
Thu 28 Mar 2019 12:11:08 AM EDT | SETI@home | Backing off 00:17:56 on download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Finished download of 26mr19ad.6346.237521.7.34.150
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Started download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:09 AM EDT | SETI@home | Started download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.208: transient HTTP error
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Backing off 00:54:58 on download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.6346.237521.7.34.206: transient HTTP error
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Backing off 00:57:46 on download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.9728.235169.11.38.38: transient HTTP error
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Backing off 00:51:55 on download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.23220.232204.10.37.194: transient HTTP error
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Backing off 00:56:55 on download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.14520.235476.14.41.221: transient HTTP error
Thu 28 Mar 2019 12:11:10 AM EDT | SETI@home | Backing off 00:54:08 on download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:11 AM EDT | SETI@home | Finished download of 26mr19ad.23220.232204.10.37.185
Thu 28 Mar 2019 12:11:11 AM EDT | SETI@home | Finished download of 26mr19ad.6346.237521.7.34.113
Thu 28 Mar 2019 12:11:11 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:11 AM EDT | SETI@home | Started download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:11 AM EDT | SETI@home | Started download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:11 AM EDT | SETI@home | Started download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:11 AM EDT | SETI@home | Started download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:12 AM EDT | SETI@home | Temporarily failed download of 26mr19ad.14520.235476.14.41.221: transient HTTP error
Thu 28 Mar 2019 12:11:12 AM EDT | SETI@home | Backing off 01:14:03 on download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:13 AM EDT | SETI@home | Finished download of 26mr19ad.6346.237521.7.34.208
Thu 28 Mar 2019 12:11:13 AM EDT | SETI@home | Finished download of 26mr19ad.6346.237521.7.34.206
Thu 28 Mar 2019 12:11:13 AM EDT | SETI@home | Finished download of 26mr19ad.9728.235169.11.38.38
Thu 28 Mar 2019 12:11:13 AM EDT | SETI@home | Finished download of 26mr19ad.23220.232204.10.37.194
Thu 28 Mar 2019 12:11:16 AM EDT | SETI@home | Started download of 26mr19ad.14520.235476.14.41.221
Thu 28 Mar 2019 12:11:19 AM EDT | SETI@home | Finished download of 26mr19ad.14520.235476.14.41.221
ID: 1987552 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1987554 - Posted: 28 Mar 2019, 4:49:34 UTC

I'm still getting them on All 4 machines, around a dozen backed off tasks at a time. When you go to clear them you have to hit the Update button multiple times in succession because the tasks go right back into project Backoff as soon as you hit the Update button. After about a dozen hits they are all downloaded...until the next time.

Exactly
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1987554 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1987561 - Posted: 28 Mar 2019, 5:38:22 UTC

I don't think I am getting any back offs.
A proud member of the OFA (Old Farts Association).
ID: 1987561 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13396
Credit: 208,696,464
RAC: 304
Australia
Message 1987562 - Posted: 28 Mar 2019, 5:43:51 UTC - in response to Message 1987544.  

So anybody else still getting instant backoffs when tasks are requested and acknowledged? Been that way most of the day.

Yep- instant, to near instant.
At least uploads are working now.
Grant
Darwin NT
ID: 1987562 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13396
Credit: 208,696,464
RAC: 304
Australia
Message 1987568 - Posted: 28 Mar 2019, 6:13:12 UTC

Things are improving, now it's mostly almost instant, with the occasional instant.
I think 2 secs is about the longest non-time out period so far.
Grant
Darwin NT
ID: 1987568 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13159
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1987573 - Posted: 28 Mar 2019, 6:44:52 UTC

Unfortunately, I always have a dozen or so tasks in backoff on each host each time I come in to look at BT Transfers page. Each group of tasks on each host somewhere between 2 to 5 hours in backoff. Then I spend another 15 minutes or so hammering the retry button to get them to clear. Then once they are cleared that gives the host permission to ask for more work which starts the failed download sequence and increasing backoff situation all over again. Sigh.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1987573 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13396
Credit: 208,696,464
RAC: 304
Australia
Message 1987575 - Posted: 28 Mar 2019, 6:53:06 UTC

I just had 5 Scheduler requests for work result in WUs downloading with no timeouts. Then on the 6th one I got a another almost instant timeout, then the 7th one no problems.
Maybe things are mostly fixed?
Grant
Darwin NT
ID: 1987575 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13396
Credit: 208,696,464
RAC: 304
Australia
Message 1987576 - Posted: 28 Mar 2019, 7:00:32 UTC - in response to Message 1987575.  

Maybe things are mostly fixed?

My apologies- my posting that, broke it again. Really, really broke it.
Sorry.
Grant
Darwin NT
ID: 1987576 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14531
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1987584 - Posted: 28 Mar 2019, 9:03:08 UTC

I looked around all machines as I got up, and found just one single file had failed to download. But that's enough to get some diagnostics.

The 'instant failure' mode seems to be

28/03/2019 08:52:17 | SETI@home | [http] [ID#5126] Info:  Connected to boinc2.ssl.berkeley.edu (208.68.240.119) port 80 (#7153)
28/03/2019 08:52:17 | SETI@home | [http] [ID#5126] Sent header to server: GET /sah/download_fanout/30/blc23_2bit_guppi_58406_22568_HIP20842_0090.29552.0.21.44.155.vlar HTTP/1.1
28/03/2019 08:52:17 | SETI@home | [http] [ID#5126] Sent header to server:
28/03/2019 08:52:17 | SETI@home | 
28/03/2019 08:52:17 | SETI@home | [http] [ID#5126] Info:  Empty reply from server
28/03/2019 08:52:17 | SETI@home | [http] HTTP error: Server returned nothing (no headers, no data)
28/03/2019 08:52:18 | SETI@home | Temporarily failed download of blc23_2bit_guppi_58406_22568_HIP20842_0090.29552.0.21.44.155.vlar: transient HTTP error
But a couple of seconds later, still on the same download server, it got through.

28/03/2019 08:52:20 | SETI@home | [http] [ID#5127] Received header from server: HTTP/1.1 200 OK
and we're away.
ID: 1987584 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4265
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1987622 - Posted: 28 Mar 2019, 15:58:45 UTC
Last modified: 28 Mar 2019, 16:07:52 UTC

it's been hit or miss for me.

sometimes it will download all tasks with no issues.

other times it hits immediate backoffs, requiring to hit retry several times for them to all go through.

edit, i moved my max_xfers_per_project back to the default of 2. seems to be helping prevent the stuck downloads. so far. could be placebo lol.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1987622 · Report as offensive
Previous · 1 . . . 27 · 28 · 29 · 30 · 31 · Next

Message boards : Number crunching : Panic Mode On (115) Server Problems?


 
©2022 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.