Panic Mode On (113) Server Problems?

Message boards : Number crunching : Panic Mode On (113) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 37 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1953702 - Posted: 4 Sep 2018, 2:37:59 UTC

RTS to zero. Splitters not able to keep up with demand. I at least was able to fill my caches up before the well ran dry. That should last for about an hour. I think we have finally cleaned the majority of the noise bombs out at least.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1953702 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1953705 - Posted: 4 Sep 2018, 2:43:52 UTC

The results in the field amount has been slowly climbing, so I think that the splitters are doing better than demand, but it will take a while to fill up all the caches, and then hopefully the RTS will start to fill. It isn't just current demand that needs to be met, but the backlog because it had shut down for a while. I guess like after a outage.
Hopefully all will be back to "normal" be before Tuesday's outrage in a few hours.
ID: 1953705 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1953709 - Posted: 4 Sep 2018, 4:52:40 UTC

Looks as though it died again. RTS Empty, Splitters at 5.x. Computers getting No Tasks again.
It's going to a long night.
ID: 1953709 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1953710 - Posted: 4 Sep 2018, 5:02:26 UTC - in response to Message 1953709.  

Looks as though it died again. RTS Empty, Splitters at 5.x. Computers getting No Tasks again.
It's going to a long night.


I agree :-( . Not good news before the Tuesday outage
ID: 1953710 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1953712 - Posted: 4 Sep 2018, 5:12:10 UTC

Nice Kick! Suddenly I'm getting work again.

About the same on two machines;
Tue Sep 4 01:08:26 2018 | SETI@home | Sending scheduler request: To report completed tasks.
Tue Sep 4 01:08:26 2018 | SETI@home | Reporting 11 completed tasks
Tue Sep 4 01:08:28 2018 | SETI@home | Scheduler request completed: got 71 new tasks


The page still hasn't updated for 50 minutes, but I got work.
ID: 1953712 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1953734 - Posted: 4 Sep 2018, 12:06:28 UTC

I see that the splitters still have not filled the RTS cache.
I would be expecting a long outage today to sort things.

Meow.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1953734 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1953735 - Posted: 5 Sep 2018, 0:43:23 UTC - in response to Message 1953734.  

we back??
ID: 1953735 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30651
Credit: 53,134,872
RAC: 32
United States
Message 1953737 - Posted: 5 Sep 2018, 0:51:53 UTC - in response to Message 1953735.  

we back??

Status page says no, but that might update soon
ID: 1953737 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1953738 - Posted: 5 Sep 2018, 0:56:11 UTC - in response to Message 1953737.  
Last modified: 5 Sep 2018, 1:08:47 UTC

we back??

Status page says no, but that might update soon


. . I've gotten work on 3 of the 4 crunchers, but mostly still those noisy Blc11 WUs. It is evaporating almost as quickly as it can download ... :(

. . It's going to be a while before I have stable caches for the rigs to keep working productively ...

Stephen

:(

{Edit} Work availability is surprisingly high for post outage. My caches are filling up. I would still feel better if there were not so many of the Blc11 tasks in the mix.

.
ID: 1953738 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1953742 - Posted: 5 Sep 2018, 1:01:05 UTC

Hope someone can help diagnose this server connection issue.
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | Sending scheduler request: Requested by user.
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | Reporting 100 completed tasks
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | [sched_op] CPU work request: 714052.70 seconds; 0.00 devices
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | [sched_op] NVIDIA GPU work request: 349056.00 seconds; 4.00 devices
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | [http] HTTP_OP::init_post(): http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | [http] [ID#1] Info: Hostname in DNS cache was stale, zapped
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | [http] [ID#1] Info: Trying 192.168.1.254...
Tue 04 Sep 2018 05:58:26 PM PDT | SETI@home | [http] [ID#1] Info: TCP_NODELAY set
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Info: Connected to setiboinc.ssl.berkeley.edu (192.168.1.254) port 80 (#7)
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: POST /sah_cgi/cgi HTTP/1.1
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: Host: setiboinc.ssl.berkeley.edu
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: User-Agent: BOINC client (x86_64-pc-linux-gnu 7.4.44)
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: Accept: */*
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: Accept-Encoding: deflate, gzip
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: Content-Type: application/x-www-form-urlencoded
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: Accept-Language: en_US
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: Content-Length: 560366
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server: Expect: 100-continue
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Sent header to server:
Tue 04 Sep 2018 05:58:28 PM PDT | SETI@home | [http] [ID#1] Info: Done waiting for 100-continue
Tue 04 Sep 2018 05:58:29 PM PDT | SETI@home | [http] [ID#1] Info: We are completely uploaded and fine
Tue 04 Sep 2018 05:58:29 PM PDT | SETI@home | [http] [ID#1] Received header from server: HTTP/1.1 200 OK
Tue 04 Sep 2018 05:58:29 PM PDT | SETI@home | [http] [ID#1] Received header from server: Connection: close
Tue 04 Sep 2018 05:58:29 PM PDT | SETI@home | [http] [ID#1] Received header from server: PANIC: unprotected error in call to Lua API (haserl_lualib.lua:34: attempt to index local 't' (a string value))
Tue 04 Sep 2018 05:58:30 PM PDT | SETI@home | [http] [ID#1] Info: Closing connection 7
Tue 04 Sep 2018 05:58:30 PM PDT | SETI@home | [error] No start tag in scheduler reply
Tue 04 Sep 2018 05:58:30 PM PDT | SETI@home | [sched_op] Deferring communication for 02:01:13
Tue 04 Sep 2018 05:58:30 PM PDT | SETI@home | [sched_op] Reason: can't parse scheduler reply
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1953742 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1953746 - Posted: 5 Sep 2018, 1:19:10 UTC - in response to Message 1953742.  

I have always used the static DNS entries, and they seem to be working fine.
ID: 1953746 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1953761 - Posted: 5 Sep 2018, 3:00:45 UTC

That machine eventually found the servers home. Just two machines managed to get some work just at project return when I beat out all others for the dwindling tasks in the buffer. The other 3 machines have struck out so far tonight in snagging any work.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1953761 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1953766 - Posted: 5 Sep 2018, 3:09:10 UTC

good news... they loaded more data files to be split...

The new blc16 is from the same day (58227) as the data bomb blc11 files. Is that a problem?? what does the number after the blc denote??
ID: 1953766 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1953778 - Posted: 5 Sep 2018, 3:29:59 UTC - in response to Message 1953766.  

good news... they loaded more data files to be split...
The new blc16 is from the same day (58227) as the data bomb blc11 files. Is that a problem?? what does the number after the blc denote??
Not so good news ... I just ran a sample 19 BLC16 tasks through ... 9 were not noise.

So we have a LOT more of this crap now :(
ID: 1953778 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30651
Credit: 53,134,872
RAC: 32
United States
Message 1953779 - Posted: 5 Sep 2018, 3:31:03 UTC - in response to Message 1953742.  

Hope someone can help diagnose this server connection issue.
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Info: Connected to setiboinc.ssl.berkeley.edu (192.168.1.254) port 80 (#7)

$ping setiboinc.ssl.berkeley.edu
PING setiboinc.ssl.berkeley.edu (208.68.240.126): 56 data bytes
ID: 1953779 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1953782 - Posted: 5 Sep 2018, 3:45:22 UTC - in response to Message 1953778.  

I ran another 32 blc16 tasks ... 13 good ones.
"Captain, How to you expect warp speed when all we have to feed the hamsters is tofu!"
ID: 1953782 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1953784 - Posted: 5 Sep 2018, 3:48:19 UTC

Splitters still having problems getting their act together; they get going, but struggle to maintain that output.
Grant
Darwin NT
ID: 1953784 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1953806 - Posted: 5 Sep 2018, 7:56:44 UTC - in response to Message 1953779.  

Hope someone can help diagnose this server connection issue.
Tue 04 Sep 2018 05:58:27 PM PDT | SETI@home | [http] [ID#1] Info: Connected to setiboinc.ssl.berkeley.edu (192.168.1.254) port 80 (#7)
All IP addresses in the 192.168.xxx.yyy range are for private use on internal networks only, and can't be routed over the internet.

I would guess that 192.168.1.254 is the address of Keith's home router, acting as a DHCP server allocating other IP addresses to the machines on the local network in his home. I'd expect it's also acting as his gateway to the outside world, and perhaps even as DNS server resolving server names into IP addresses.

My equivalent, using the Windows command 'ipconfig /all' (Linux will have something similar), is 192.168.173.1:

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . : home
   Description . . . . . . . . . . . : Realtek PCIe GBE Family Controller
   Physical Address. . . . . . . . . : FC-AA-14-CB-13-17
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::19ef:5e13:18be:e3b0%11(Preferred)
   IPv4 Address. . . . . . . . . . . : 192.168.173.26(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Lease Obtained. . . . . . . . . . : 01 September 2018 08:35:13
   Lease Expires . . . . . . . . . . : 06 September 2018 08:35:26
   Default Gateway . . . . . . . . . : 192.168.173.1
   DHCP Server . . . . . . . . . . . : 192.168.173.1
   DHCPv6 IAID . . . . . . . . . . . : 251439636
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1E-27-44-01-FC-AA-14-CB-13-17

   DNS Servers . . . . . . . . . . . : 192.168.173.1
   NetBIOS over Tcpip. . . . . . . . : Enabled
My guess is that there was a glitch on the local connection from Keith's router to his ISP. The router saw the port 80 request and couldn't handle it because of the glitch. The router assumed it came from a web browser, and sent back a user-friendly error message to explain what had gone wrong. Mine does that, too:


Unfortunately, BOINC doesn't know how to handle error messages when it's expecting a reply from the project server...
ID: 1953806 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1953813 - Posted: 5 Sep 2018, 8:40:25 UTC

Thanks again Jord, your trick to get a stuck upload to go through worked again :)
My Ryzen has had one stuck for 3 days, wouldn't upload or abort.

It took me a bit to find the reference info from a year ago.
https://setiathome.berkeley.edu/forum_thread.php?id=82054&postid=1895643
ID: 1953813 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1953815 - Posted: 5 Sep 2018, 8:54:24 UTC - in response to Message 1953766.  

good news... they loaded more data files to be split...

The new blc16 is from the same day (58227) as the data bomb blc11 files. Is that a problem?? what does the number after the blc denote??


. . the channel/frequency band being split from the receiver. There are lots of recorders, seems to be blc0x, blc1x to I suspect blc7x if/when they are installed. Highest I have seen to date was blc2x if memory serves me. And they are blcn0-7.

Stephen

. .
ID: 1953815 · Report as offensive
Previous · 1 . . . 8 · 9 · 10 · 11 · 12 · 13 · 14 . . . 37 · Next

Message boards : Number crunching : Panic Mode On (113) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.