Panic Mode On (100) Server Problems?

Message boards : Number crunching : Panic Mode On (100) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 17 · 18 · 19 · 20 · 21 · 22 · 23 . . . 32 · Next

AuthorMessage
Profile Oz
Avatar

Send message
Joined: 6 Jun 99
Posts: 233
Credit: 200,655,462
RAC: 212
United States
Message 1730707 - Posted: 1 Oct 2015, 17:45:19 UTC - in response to Message 1730703.  

I find it interesting that the received WUs dropped about 23k when all this started. No spikes or dips, just lower.

It makes me think that there may be a limit on the number of connections being enforced.


I noticed the same thing. It begs a more interesting question, why are ~70% of seti clients apparently still able to connect?
Member of the 20 Year Club



ID: 1730707 · Report as offensive
Profile criton
Avatar

Send message
Joined: 28 Feb 00
Posts: 131
Credit: 13,351,000
RAC: 2
United Kingdom
Message 1730709 - Posted: 1 Oct 2015, 17:48:26 UTC
Last modified: 1 Oct 2015, 17:52:32 UTC

closed boinc down and then restarted boinc straight after retried to send completed work units in and they went straight through and down loaded my cash to full again. the other pc wouldn't close down boinc so rebooted it and the same result all work sent in and cash filled up to max. les
ID: 1730709 · Report as offensive
Iztok s52d (and friends)

Send message
Joined: 12 Jan 01
Posts: 136
Credit: 393,469,375
RAC: 116
Slovenia
Message 1730713 - Posted: 1 Oct 2015, 18:02:05 UTC - in response to Message 1730703.  

I find it interesting that the received WUs dropped about 23k when all this started. No spikes or dips, just lower.

It makes me think that there may be a limit on the number of connections being enforced.


No. When I looked at hosts stats, I found it very consistent:
machines on one ISP are connecting, downloading, reporting, crunching.

Machines on another has not reported a single WU
since 30 Sep 2015, 14:09:45 UTC.

Upload works, but I can not report completed tasks.

On theory side: maybe difference is in the time to live? One ISP uses more hops?
sorry, behind firewalls, no ICMP possible on non-working ISP.

73
s52d
ID: 1730713 · Report as offensive
Iztok s52d (and friends)

Send message
Joined: 12 Jan 01
Posts: 136
Credit: 393,469,375
RAC: 116
Slovenia
Message 1730718 - Posted: 1 Oct 2015, 18:15:49 UTC - in response to Message 1730713.  


On theory side: maybe difference is in the time to live? One ISP uses more hops?
sorry, behind firewalls, no ICMP possible on non-working ISP.


No.
linux:
echo "128" >/proc/sys/net/ipv4/ip_default_ttl

and no help. Unless firewalls are modifying TTL field?

73
s52d
ID: 1730718 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1730719 - Posted: 1 Oct 2015, 18:19:04 UTC

Still intermittent for me here. This is from just a few moments ago on my laptop:

10/1/2015 11:13:17 AM | SETI@home | Sending scheduler request: To report completed tasks.
10/1/2015 11:13:17 AM | SETI@home | Reporting 1 completed tasks
10/1/2015 11:13:17 AM | SETI@home | Requesting new tasks for CPU
10/1/2015 11:13:38 AM | SETI@home | Scheduler request failed: Couldn't connect to server
10/1/2015 11:13:42 AM |  | Project communication failed: attempting access to reference site
10/1/2015 11:13:44 AM |  | Internet access OK - project servers may be temporarily down.
10/1/2015 11:15:15 AM | SETI@home | Sending scheduler request: To report completed tasks.
10/1/2015 11:15:15 AM | SETI@home | Reporting 1 completed tasks
10/1/2015 11:15:15 AM | SETI@home | Requesting new tasks for CPU
10/1/2015 11:15:17 AM | SETI@home | Scheduler request completed: got 1 new tasks
10/1/2015 11:15:19 AM | SETI@home | Started download of 02ja11ac.6328.8247.438086664203.12.35.vlar
10/1/2015 11:15:21 AM | SETI@home | Finished download of 02ja11ac.6328.8247.438086664203.12.35.vlar

It fails on the first try, then sails through without a problem on the second. (The previous scheduler contact on this machine completed the scheduler request okay, then the download failed, only to succeed a couple minutes later.)
ID: 1730719 · Report as offensive
Profile John Neale
Volunteer tester
Avatar

Send message
Joined: 16 Mar 00
Posts: 634
Credit: 7,246,513
RAC: 9
South Africa
Message 1730720 - Posted: 1 Oct 2015, 18:19:23 UTC

I'm in Outokumpu, Finland at the moment, using the hotel's wi-fi. I cannot report results, and I also cannot access the BOINC website. I am not experiencing any problems with the SETI@home website. I experienced the same behaviour earlier in the day using the wi-fi where I'm working. Here are a few traceroute results:

Tracing route to setiathome.berkeley.edu [208.68.240.110] over a maximum of 30 hops:

1 5 ms 2 ms 2 ms 192.168.XX.XXX
2 5 ms 3 ms 2 ms 10.15.10.1
3 33 ms 27 ms 96 ms 172.20.15.41
4 28 ms 66 ms 30 ms 139.97.99.136
5 35 ms 112 ms 59 ms ae1.helpa-gw1.fi.elisa.net [139.97.6.242]
6 71 ms 24 ms 99 ms ae0.bbr1.hel2.fi.eunetip.net [213.192.191.153]
7 144 ms 29 ms 110 ms 213.192.184.74
8 101 ms 60 ms 32 ms 212.73.250.181
9 262 ms 203 ms 201 ms vl-3520-ve-130.ebr1.Stockholm2.Level3.net [4.69.206.97]
10 * * * Request timed out.
11 * * 100 ms ae-101-101.ebr2.Dusseldorf1.Level3.net [4.69.141.150]
12 * 103 ms 109 ms ae-47-47.ebr1.Amsterdam1.Level3.net [4.69.143.205]
13 * * * Request timed out.
14 305 ms 247 ms 214 ms ae-45-45.ebr2.London1.Level3.net [4.69.143.70]
15 * * * Request timed out.
16 * * * Request timed out.
17 203 ms 201 ms 201 ms ae-2-2.ebr1.SanJose1.Level3.net [4.69.135.185]
18 207 ms 202 ms 241 ms ae-61-61.csw1.SanJose1.Level3.net [4.69.153.2]
19 * * * Request timed out.
20 308 ms 206 ms 212 ms CENIC.ear1.SanJose1.Level3.net [4.15.122.46]
21 291 ms 344 ms 219 ms dc-oak-agg4--svl-agg4-100ge.cenic.net [137.164.46.144]
22 225 ms 287 ms 290 ms ucb--oak-agg4-10g.cenic.net [137.164.50.31]
23 279 ms 203 ms 203 ms t2-3.inr-202-reccev.Berkeley.EDU [128.32.0.39]
24 204 ms 285 ms 210 ms e3-47.inr-310-ewdc.Berkeley.EDU [128.32.0.99]
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.

Trace complete.


Tracing route to setiboinc.ssl.berkeley.edu [208.68.240.126] over a maximum of 30 hops:

1 4 ms 2 ms 2 ms 192.168.XX.XXX
2 6 ms 2 ms 2 ms 10.15.10.1
3 123 ms 14 ms 15 ms 172.20.15.41
4 78 ms 30 ms 22 ms 139.97.99.139
5 63 ms 70 ms 110 ms ae2.helpa-gw1.fi.elisa.net [139.97.6.250]
6 59 ms 33 ms 23 ms ae0.bbr1.hel2.fi.eunetip.net [213.192.191.153]
7 49 ms 27 ms 27 ms 213.192.184.74
8 46 ms 30 ms 29 ms 212.73.250.181
9 199 ms 200 ms 324 ms vl-3517-ve-127.ebr1.Stockholm2.Level3.net [4.69.206.85]
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 107 ms 104 ms 97 ms ae-101-101.ebr2.Amsterdam1.Level3.net [4.69.203.90]
14 298 ms 288 ms 297 ms ae-47-47.ebr2.London1.Level3.net [4.69.143.78]
15 * * * Request timed out.
16 * * * Request timed out.
17 207 ms 202 ms 203 ms ae-2-2.ebr1.SanJose1.Level3.net [4.69.135.185]
18 214 ms 202 ms 253 ms ae-91-91.csw4.SanJose1.Level3.net [4.69.153.14]
19 * * * Request timed out.
20 200 ms 202 ms 351 ms CENIC.ear1.SanJose1.Level3.net [4.15.122.46]
21 334 ms 234 ms 259 ms dc-oak-agg4--svl-agg4-100ge.cenic.net [137.164.46.144]
22 274 ms 205 ms 204 ms ucb--oak-agg4-10g.cenic.net [137.164.50.31]
23 204 ms 206 ms 203 ms t2-3.inr-202-reccev.Berkeley.EDU [128.32.0.39]
24 202 ms 202 ms 273 ms et3-47.inr-311-ewdc.Berkeley.EDU [128.32.0.103]
25 et3-47.inr-311-ewdc.Berkeley.EDU [128.32.0.103] reports: Destination host unreachable.

Trace complete.


Tracing route to boinc.berkeley.edu [208.68.240.115] over a maximum of 30 hops:

1 4 ms 2 ms 1 ms 192.168.XX.XXX
2 4 ms 2 ms 7 ms 10.15.10.1
3 15 ms 15 ms 83 ms 172.20.15.41
4 133 ms 97 ms 104 ms 139.97.99.136
5 117 ms 104 ms 79 ms ae1.helpa-gw1.fi.elisa.net [139.97.6.242]
6 83 ms 24 ms 94 ms ae0.bbr1.hel2.fi.eunetip.net [213.192.191.153]
7 47 ms 93 ms 33 ms 213.192.184.74
8 * 129 ms 30 ms 212.73.250.181
9 218 ms 201 ms 344 ms vl-3519-ve-129.ebr1.Stockholm2.Level3.net [4.69.206.93]
10 169 ms 72 ms 55 ms ae-44-44.ebr1.Dusseldorf1.Level3.net [4.69.201.242]
11 59 ms 109 ms 55 ms ae-101-101.ebr2.Dusseldorf1.Level3.net [4.69.141.150]
12 * * 165 ms ae-47-47.ebr1.Amsterdam1.Level3.net [4.69.143.205]
13 57 ms 122 ms 96 ms ae-101-101.ebr2.Amsterdam1.Level3.net [4.69.203.90]
14 200 ms 201 ms 320 ms ae-46-46.ebr2.London1.Level3.net [4.69.143.74]
15 * * * Request timed out.
16 * * * Request timed out.
17 202 ms 200 ms 242 ms ae-2-2.ebr1.SanJose1.Level3.net [4.69.135.185]
18 201 ms 303 ms 206 ms ae-81-81.csw3.SanJose1.Level3.net [4.69.153.10]
19 227 ms 305 ms 204 ms ae-3-80.ear1.SanJose1.Level3.net [4.69.152.150]
20 356 ms 204 ms 304 ms CENIC.ear1.SanJose1.Level3.net [4.15.122.46]
21 201 ms 204 ms 245 ms dc-oak-agg4--svl-agg4-100ge.cenic.net [137.164.46.144]
22 203 ms 253 ms 206 ms ucb--oak-agg4-10g.cenic.net [137.164.50.31]
23 313 ms 226 ms 204 ms t2-3.inr-201-sut.Berkeley.EDU [128.32.0.37]
24 223 ms 227 ms 204 ms et3-48.inr-311-ewdc.Berkeley.EDU [128.32.0.101]
25 et3-48.inr-311-ewdc.Berkeley.EDU [128.32.0.101] reports: Destination host unreachable.

Trace complete.

ID: 1730720 · Report as offensive
grn

Send message
Joined: 23 Oct 00
Posts: 13
Credit: 2,608,907
RAC: 0
United Kingdom
Message 1730723 - Posted: 1 Oct 2015, 18:26:19 UTC

I have 3 systems that have been unable to upload results for a almost a day now and was wondering if there is any idea when the issue may be resolved. I am on holiday on Saturday and have about 50 completed WUs to upload most of which have deadlines sometime during the next week.

My machines will be shutdown early Saturday morning at the latest, so hoping it is resolved before then.

Keeping my fingers crossed.

George
ID: 1730723 · Report as offensive
Phil Burden

Send message
Joined: 26 Oct 00
Posts: 264
Credit: 22,303,899
RAC: 0
United Kingdom
Message 1730724 - Posted: 1 Oct 2015, 18:27:28 UTC - in response to Message 1730709.  

closed boinc down and then restarted boinc straight after retried to send completed work units in and they went straight through and down loaded my cash to full again. the other pc wouldn't close down boinc so rebooted it and the same result all work sent in and cash filled up to max. les


same result here too, stop & restart Boinc and all tasks reported ok, though subsequent schedule requests failed with "can't connect to server"
ID: 1730724 · Report as offensive
qbit
Volunteer tester
Avatar

Send message
Joined: 19 Sep 04
Posts: 630
Credit: 6,868,528
RAC: 0
Austria
Message 1730731 - Posted: 1 Oct 2015, 19:05:01 UTC

Once again out of work.....:-(
ID: 1730731 · Report as offensive
Bruce
Volunteer tester

Send message
Joined: 15 Mar 02
Posts: 123
Credit: 124,955,234
RAC: 11
United States
Message 1730734 - Posted: 1 Oct 2015, 19:11:15 UTC

Wish I know what was wrong. My last successful communication with the Scheduler was 7:00AM PDT on 9/30/2015. Since that time I have not reached it even once.

All I have ever gotten is:
10/1/2015 2:36:47 PM | SETI@home | Reporting 100 completed tasks
10/1/2015 2:36:47 PM | SETI@home | Requesting new tasks for NVIDIA
10/1/2015 2:37:08 PM | SETI@home | Scheduler request failed: Couldn't connect to server
10/1/2015 2:37:23 PM | | Project communication failed: attempting access to reference site
10/1/2015 2:37:24 PM | | Internet access OK - project servers may be temporarily down.

I have tried all the obvious things multiple times.
Restart Boinc
Reboot computer
Reboot router
Flush DNS

Nothing seems to work for me. I don't understand how some people can get through to the Scheduler without any trouble, and I can not reach it even a single time.

Here are the pertinent lines from the Tracert I just did:

Tracing route to setiathome.berkeley.edu [208.68.240.110]
over a maximum of 30 hops:
16 74 ms 74 ms 74 ms t2-3.inr-202-reccev.Berkeley.EDU [128.32.0.39]
17 75 ms 75 ms 75 ms e3-47.inr-310-ewdc.Berkeley.EDU [128.32.0.99]
18 * * * Request timed out.


Tracing route to setiathome.ssl.berkeley.edu [208.68.240.110]
over a maximum of 30 hops:
16 74 ms 73 ms 74 ms t2-3.inr-202-reccev.Berkeley.EDU [128.32.0.39]
17 75 ms 75 ms 74 ms e3-47.inr-310-ewdc.Berkeley.EDU [128.32.0.99]
18 * * * Request timed out.


Tracing route to setiathome.berkeley.edu [208.68.240.110]
over a maximum of 30 hops:
16 74 ms 74 ms 74 ms t2-3.inr-202-reccev.Berkeley.EDU [128.32.0.39]
17 75 ms 75 ms 75 ms e3-47.inr-310-ewdc.Berkeley.EDU [128.32.0.99]
18 * * * Request timed out.

Hope that this info might help someone to figure out what the problem is.

In the mean time, I am out of work and have a cache full of results that I can not report.

Bruce
Bruce
ID: 1730734 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1730744 - Posted: 1 Oct 2015, 19:41:05 UTC - in response to Message 1730734.  
Last modified: 1 Oct 2015, 19:42:53 UTC

Tracing route to setiathome.berkeley.edu [208.68.240.110]
Tracing route to setiathome.ssl.berkeley.edu [208.68.240.110]

Well, that looks like a DNS problem to me. Different servers, same IP address. You haven't been fiddling with a hosts file, have you?

Edit - or maybe not. I don't think "setiathome.ssl.berkeley.edu" (the second one) is a real server. Where did you get that name from?
ID: 1730744 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1730748 - Posted: 1 Oct 2015, 19:47:40 UTC - in response to Message 1730744.  

Richard, my routes come up the same.

One server can host multiple sites, so I don't think that is it.
ID: 1730748 · Report as offensive
Baiteh

Send message
Joined: 10 Sep 15
Posts: 34
Credit: 7,705,483
RAC: 0
United Kingdom
Message 1730753 - Posted: 1 Oct 2015, 19:59:44 UTC - in response to Message 1730744.  

I get this;

Thu 01 Oct 2015 20:55:11 IST | SETI@home | Requesting new tasks for CPU
Thu 01 Oct 2015 20:55:13 IST | SETI@home | Scheduler request failed: Couldn't connect to server
Thu 01 Oct 2015 20:55:14 IST | | Project communication failed: attempting access to reference site
Thu 01 Oct 2015 20:55:16 IST | | Internet access OK - project servers may be temporarily down.

I can get to the BOINC site but it's slow.

Also

PING setiathome.ssl.berkeley.edu (208.68.240.110) 56(84) bytes of data.
^C
--- setiathome.ssl.berkeley.edu ping statistics ---
67 packets transmitted, 0 received, 100% packet loss, time 66528ms
ID: 1730753 · Report as offensive
Profile Zombu2
Volunteer tester

Send message
Joined: 24 Feb 01
Posts: 1615
Credit: 49,315,423
RAC: 0
United States
Message 1730754 - Posted: 1 Oct 2015, 20:00:29 UTC

every once in a while i get master file download succeeded but thats it
I came down with a bad case of i don't give a crap
ID: 1730754 · Report as offensive
OGM

Send message
Joined: 14 Apr 15
Posts: 12
Credit: 1,001,458
RAC: 0
Portugal
Message 1730755 - Posted: 1 Oct 2015, 20:00:33 UTC

Also no server contact for over a day now on my end. Uploads seem to work, but the report and download part fail.
ID: 1730755 · Report as offensive
Profile Zombu2
Volunteer tester

Send message
Joined: 24 Feb 01
Posts: 1615
Credit: 49,315,423
RAC: 0
United States
Message 1730758 - Posted: 1 Oct 2015, 20:02:56 UTC

i abused teh hell out of it now i have a hole in the back of the screen
I came down with a bad case of i don't give a crap
ID: 1730758 · Report as offensive
Profile Oz
Avatar

Send message
Joined: 6 Jun 99
Posts: 233
Credit: 200,655,462
RAC: 212
United States
Message 1730763 - Posted: 1 Oct 2015, 20:06:30 UTC

Here is a tracert from one of my boxes that has been connecting...

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

>tracert -h 20 -w 15001 setiathome.berkeley.edu

Tracing route to setiathome.berkeley.edu [208.68.240.110]
over a maximum of 20 hops:

1 <1 ms <1 ms <1 ms xxx
2 3 ms 4 ms 4 ms xxx
3 7 ms 7 ms 7 ms xxx
4 * * * Request timed out.
5 * * * Request timed out.
6 16 ms 16 ms 15 ms 0.ae2.BR3.NYC4.ALTER.NET [140.222.229.99]
7 * * * Request timed out.
8 75 ms 74 ms 74 ms ae-3-80.ear1.SanJose1.Level3.net [4.69.152.150]
9 * * * Request timed out.
10 73 ms 73 ms 73 ms CENIC.ear1.SanJose1.Level3.net [4.15.122.46]
11 74 ms 75 ms 74 ms dc-oak-agg4--svl-agg4-100ge.cenic.net [137.164.46.144]
12 75 ms 81 ms 76 ms ucb--oak-agg4-10g.cenic.net [137.164.50.31]
13 75 ms 74 ms 74 ms t2-3.inr-201-sut.Berkeley.EDU [128.32.0.37]
14 75 ms 75 ms 74 ms et3-48.inr-311-ewdc.Berkeley.EDU [128.32.0.101]
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.

Trace complete.

So... WTF!?!
Member of the 20 Year Club



ID: 1730763 · Report as offensive
Profile Zombu2
Volunteer tester

Send message
Joined: 24 Feb 01
Posts: 1615
Credit: 49,315,423
RAC: 0
United States
Message 1730764 - Posted: 1 Oct 2015, 20:08:25 UTC

yeah same traceroute as everyone else
I came down with a bad case of i don't give a crap
ID: 1730764 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1730768 - Posted: 1 Oct 2015, 20:16:25 UTC
Last modified: 1 Oct 2015, 20:42:12 UTC

I wonder if TCP Optimizer might help some of those who haven't been able to connect at all. As I recall, I set my machines up with that before the move to the co-lo and currently I'm at least getting intermittently successfully connections.

The thread Windows TCP Settings - Follow up - Help with server communication is where I originally got the info and settings for TCP Optimizer.

EDIT: Changed the link to force it to the beginning of that thread.
ID: 1730768 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1730770 - Posted: 1 Oct 2015, 20:17:43 UTC - in response to Message 1730753.  

Also

PING setiathome.ssl.berkeley.edu (208.68.240.110) 56(84) bytes of data.
^C
--- setiathome.ssl.berkeley.edu ping statistics ---
67 packets transmitted, 0 received, 100% packet loss, time 66528ms

Pinging the servers won't help, because the ping is caught by the campus firewall. So even when Seti (and BOINC) are available, pinging them will result in zero packets returning.
ID: 1730770 · Report as offensive
Previous · 1 . . . 17 · 18 · 19 · 20 · 21 · 22 · 23 . . . 32 · Next

Message boards : Number crunching : Panic Mode On (100) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.