Panic Mode On (109) Server Problems?

Message boards : Number crunching : Panic Mode On (109) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 35 · Next

AuthorMessage
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 5375
Credit: 30,870,693
RAC: 1
United States
Message 1911531 - Posted: 7 Jan 2018, 17:42:03 UTC

Again, exposing my ignorance, I am not a code writer or analyst. I can however follow simple directions when I know where to look and what to change.

Where exactly is the 'hosts' file located, and how do I 'comment out' 208.68.240.127 ?

Some of us are not computer scientists but know enough to 'get around' and really want to contribute to the project but in posting work around and fixes sometimes the explanation assumes a level of basic knowledge that some of us just don't have........IMHO.

Thanks for any assistance.

"Sour Grapes make a bitter Whine." <(0)>
ID: 1911531 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1911534 - Posted: 7 Jan 2018, 17:47:24 UTC - in response to Message 1911531.  
Last modified: 7 Jan 2018, 17:50:45 UTC

I think most of us are using the info Richard provided in the summer about bypassing the DNS servers and going direct.
http://setiathome.berkeley.edu/forum_thread.php?id=81638&postid=1875152

EDIT: And this. http://setiathome.berkeley.edu/forum_thread.php?id=81638&postid=1875158
ID: 1911534 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1911535 - Posted: 7 Jan 2018, 17:49:32 UTC - in response to Message 1911524.  
Last modified: 7 Jan 2018, 17:58:41 UTC

@Chris, as Richard said, there is no need to do any further post-mortem on the problem. We know what the issue is with the download servers. Simple solution is to just comment out Vader server at 208.68.240.127 in the Hosts file and wait for staff to get it working next week.
Sorry, wasn't really a post about the server side problem. But more of a learning opportunity for me on how the client side works - not just with with BOINC or SETI, but with networking, specifically with DNS, load balancing, and proxy servers. All of which I had at home just for fun to see how they work but it's been awhile.
It's generic, and very basic, TCP/IP networking. Add the hosts file to your list.

The unusual thing is the way SETI uses round-robin DNS. The best way to see this is from the command prompt.

Type 'ipconfig /flushdns' to clear your browsing history from the last 24 hours - it makes life easier.

Then trigger or retry a SETI download.

Now type 'ipconfig /displaydns'. You should, among the chatter, see

Windows IP Configuration

    boinc2.ssl.berkeley.edu
    ----------------------------------------
    Record Name . . . . . : boinc2.ssl.berkeley.edu
    Record Type . . . . . : 1
    Time To Live  . . . . : 120
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 208.68.240.127


    Record Name . . . . . : boinc2.ssl.berkeley.edu
    Record Type . . . . . : 1
    Time To Live  . . . . : 120
    Data Length . . . . . : 4
    Section . . . . . . . : Answer
    A (Host) Record . . . : 208.68.240.119
Two download IP addresses. BOINC (or any other Windows program) will try the first one. In this case, Vader - and the downloads failed.

But also note the very short TTL (Time To Live - seconds). Windows effectively never caches that IP address (although BOINC does). A new download will fetch a new DNS response, and may get the IPs the other way round - and succeed.

Edit - Yup. Went back to BOINC after posting that, and clicked 'retry now' for just one file. All 40 downloaded at the first attempt.

07/01/2018 17:44:22 | SETI@home | Backing off 00:03:39 on download of blc05_2bit_guppi_57976_10703_HIP74981_0036.11026.818.22.45.128.vlar
07/01/2018 17:54:07 | SETI@home | Started download of blc05_2bit_guppi_57976_11502_HIP91971_0038.11545.818.22.45.109.vlar
07/01/2018 17:54:10 | SETI@home | Finished download of blc05_2bit_guppi_57976_11502_HIP91971_0038.11545.818.22.45.109.vlar
10 minutes would seem to be a good time to wait.
ID: 1911535 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1911539 - Posted: 7 Jan 2018, 17:58:19 UTC

This is going back to original posts made a couple of years ago, (guessing) when we had issues with the two download servers and the way they handle download requests in a round-robin order. Fix was to list the servers in your Hosts file and not rely on DNS discovery mechanisms. The Hosts file is in the C:\Windows\System32\drivers\etc directory. Use Notepad to edit it and add this entry:

208.68.240.118	setiboincdata.ssl.berkeley.edu	# upload server Oct 2016
208.68.240.119	boinc2.ssl.berkeley.edu		# Georgem download server Oct 2016
208.68.240.126	setiboinc.ssl.berkeley.edu	# scheduler Oct 2016
#208.68.240.127  vader.ssl.berkeley.edu         # Vader download server Oct 2016


The hashtag in front of the 208.68.240.127 address means the line is commented out and is not read or acted upon. Vader is the server that is causing the stalled downloads. For now, you should only use the Georgem download server.

You can do a ipconfig /flushdns from the command line to make sure the Hosts file is read and your old DNS cache is flushed. The downloads will start working again. You can check to see which download server is being used to get work by setting the http_debug flag option in the Event Log options menu.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1911539 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1911541 - Posted: 7 Jan 2018, 18:01:54 UTC - in response to Message 1911539.  
Last modified: 7 Jan 2018, 18:05:09 UTC

Use Notepad to edit it ...
Run Notepad 'As Administrator' on Windows 7 or later - otherwise you won't be able to save edits.

You can do a ipconfig /flushdns from the command line to make sure the Hosts file is read and your old DNS cache is flushed. The downloads will start working again. You can check to see which download server is being used to get work by setting the http_debug flag option in the Event Log options menu.
Very rarely needed. The new values in the hosts file are read automatically whenever you save it, and supersede any previous values - whether from the file or cached.
ID: 1911541 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 5375
Credit: 30,870,693
RAC: 1
United States
Message 1911546 - Posted: 7 Jan 2018, 18:21:39 UTC
Last modified: 7 Jan 2018, 18:29:42 UTC

OK. I successfully inserted the script Keith provided, flushed the dns cache and got confirmation in the change to the hosts file from my WInpatrol app.

I only changed one machine and will monitor it for hangs.

Thanks again for the detailed help.

JE

edit Strangely the machine I did not change has not encountered any hung downloads.

"Sour Grapes make a bitter Whine." <(0)>
ID: 1911546 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1911557 - Posted: 7 Jan 2018, 18:48:57 UTC - in response to Message 1911546.  

Glad to hear your downloads are working again. Shoutout to @Richard for pointing out Hosts is a protected access file that needs Administrator privileges to edit.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1911557 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51524
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1911564 - Posted: 7 Jan 2018, 19:00:35 UTC

Yes, thank you, Richard, for the reminder.
I had never used the hosts file in the past, but I am now, and it seems to have done the trick.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1911564 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1911571 - Posted: 7 Jan 2018, 19:14:04 UTC

07/01/2018 20:06:53 | SETI@home | Not requesting tasks: some download is stalled
07/01/2018 20:06:57 | SETI@home | Scheduler request completed
07/01/2018 20:07:59 | SETI@home | Started download of blc05_2bit_guppi_57976_10027_HIP74981_0034.26512.818.21.44.135.vlar
07/01/2018 20:08:01 | SETI@home | Temporarily failed download of blc05_2bit_guppi_57976_10027_HIP74981_0034.26512.818.21.44.135.vlar: transient HTTP error
07/01/2018 20:08:01 | SETI@home | Backing off 00:16:07 on download of blc05_2bit_guppi_57976_10027_HIP74981_0034.26512.818.21.44.135.vlar
07/01/2018 20:08:02 |  | Project communication failed: attempting access to reference site
07/01/2018 20:08:04 |  | Internet access OK - project servers may be temporarily down.

ID: 1911571 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22721
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1911572 - Posted: 7 Jan 2018, 19:17:08 UTC

I too just suffered a "bump" like that on one of my crunchers. It appears to have cleared now, but.....
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1911572 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1911574 - Posted: 7 Jan 2018, 19:24:02 UTC - in response to Message 1911564.  

Yes, thank you, Richard, for the reminder.
I had never used the hosts file in the past, but I am now, and it seems to have done the trick.
Yes, it can help.

I recommend that you (and every other new user) comment out the active lines as soon as this particular problem is over - they would cause a problem if it's GeorgeM that falls over next time, and prevent Vader taking over.

I've let Eric enjoy his weekend in peace - unless he's reading this thread - but I'm going to email him and Jeff to say that one of our servers is missing, before the lab opens for buisness tomorrow. I'll keep you posted.
ID: 1911574 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51524
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1911577 - Posted: 7 Jan 2018, 19:28:52 UTC - in response to Message 1911574.  

Yes, thank you, Richard, for the reminder.
I had never used the hosts file in the past, but I am now, and it seems to have done the trick.
Yes, it can help.

I recommend that you (and every other new user) comment out the active lines as soon as this particular problem is over - they would cause a problem if it's GeorgeM that falls over next time, and prevent Vader taking over.

I've let Eric enjoy his weekend in peace - unless he's reading this thread - but I'm going to email him and Jeff to say that one of our servers is missing, before the lab opens for buisness tomorrow. I'll keep you posted.

Will do, Richard.
I shall watch this thread for news that the problem has been fixed, and neuter the hosts file.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1911577 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1911578 - Posted: 7 Jan 2018, 19:29:06 UTC
Last modified: 7 Jan 2018, 19:29:44 UTC

It's worsening.
07/01/2018 20:27:32 | SETI@home | Started download of blc05_2bit_guppi_57976_12239_HIP91971_0040.3044.818.21.44.69.vlar
07/01/2018 20:27:32 | SETI@home | Started download of blc05_2bit_guppi_57976_11873_HIP91144_0039.2611.818.21.44.58.vlar
07/01/2018 20:27:34 |  | Project communication failed: attempting access to reference site
07/01/2018 20:27:34 | SETI@home | Temporarily failed download of blc05_2bit_guppi_57976_12239_HIP91971_0040.3044.818.21.44.69.vlar: transient HTTP error
07/01/2018 20:27:34 | SETI@home | Backing off 00:02:48 on download of blc05_2bit_guppi_57976_12239_HIP91971_0040.3044.818.21.44.69.vlar
07/01/2018 20:27:34 | SETI@home | Temporarily failed download of blc05_2bit_guppi_57976_11873_HIP91144_0039.2611.818.21.44.58.vlar: transient HTTP error
07/01/2018 20:27:34 | SETI@home | Backing off 00:03:47 on download of blc05_2bit_guppi_57976_11873_HIP91144_0039.2611.818.21.44.58.vlar
07/01/2018 20:27:36 |  | Internet access OK - project servers may be temporarily down.

I'm not going to babysit this system.
ID: 1911578 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1911583 - Posted: 7 Jan 2018, 19:40:03 UTC - in response to Message 1911578.  

I'm not going to babysit this system.
Edit, paste, save, walk away.
ID: 1911583 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1911587 - Posted: 7 Jan 2018, 19:50:55 UTC - in response to Message 1911583.  

I really don't feel like editing in the command prompt, plus I'm packing and getting ready for taxes so.... That is why I have back up projects. Yes, the RAC will fall but I got my eye on a white beach with margaritas on them. I'll check back in next week, lol...
ID: 1911587 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1911588 - Posted: 7 Jan 2018, 19:52:28 UTC - in response to Message 1911587.  
Last modified: 7 Jan 2018, 19:53:21 UTC

We just suggested Notepad. But have a nice holiday anyway.
ID: 1911588 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1911593 - Posted: 7 Jan 2018, 19:59:07 UTC - in response to Message 1911587.  

I've always liked the trick where you use the File Manager to traverse to the desired directory. Then hold down the SHIFT key and then right-click in the File Manager window pane and select the "Open command window here" option. That pops open a command window where you can work on the files in that directory. If you are the Administrator already, not necessary to elevate to Administrator privileges to begin with.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1911593 · Report as offensive
Profile mr.mac52
Avatar

Send message
Joined: 18 Mar 03
Posts: 67
Credit: 245,882,461
RAC: 0
United States
Message 1911597 - Posted: 7 Jan 2018, 20:13:52 UTC - in response to Message 1911539.  

Richard and Keith,

I was too lazy to go edit the Hosts file on my own and your two notes did it all for me.

By the time I went back into BoincTasks, it had cleared all the files in retry mode that had been sitting there.

Thanks!

John
ID: 1911597 · Report as offensive
Profile Stargate (SA)
Volunteer tester
Avatar

Send message
Joined: 4 Mar 10
Posts: 1854
Credit: 2,258,721
RAC: 0
Australia
Message 1911598 - Posted: 7 Jan 2018, 20:18:16 UTC

As I don't understand the remedy proposed I will leave the computer as is until the issue is fixed next week and just keep crunching best I can..

Cheers
ID: 1911598 · Report as offensive
Ettdus

Send message
Joined: 28 Feb 16
Posts: 4
Credit: 569,767
RAC: 0
Germany
Message 1911601 - Posted: 7 Jan 2018, 20:24:20 UTC
Last modified: 7 Jan 2018, 20:24:36 UTC

Hmm... time to modify the hosts file :/ issue with the inability to download WUs occurred again.
ID: 1911601 · Report as offensive
Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 35 · Next

Message boards : Number crunching : Panic Mode On (109) Server Problems?


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.