Panic Mode On (115) Server Problems?

Message boards : Number crunching : Panic Mode On (115) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 31 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1985624 - Posted: 17 Mar 2019, 15:32:27 UTC - in response to Message 1985611.  

Not to turn on the panic bottom but

Did anyone tried to see it's validated WU for example? In my case the page runs, runs & runs but never opens.

The web site is getting fairly slow when trying to view any tasks now. Doesn't matter the page type.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1985624 · Report as offensive
Profile -= Vyper =-
Volunteer tester
Avatar

Send message
Joined: 5 Sep 99
Posts: 1652
Credit: 1,065,191,981
RAC: 2,537
Sweden
Message 1985627 - Posted: 17 Mar 2019, 15:44:25 UTC - in response to Message 1985624.  

Not to turn on the panic bottom but

Did anyone tried to see it's validated WU for example? In my case the page runs, runs & runs but never opens.

The web site is getting fairly slow when trying to view any tasks now. Doesn't matter the page type.


Perhaps they forgot swappiness = 1 or something :P

_________________________________________________________________________
Addicted to SETI crunching!
Founder of GPU Users Group
ID: 1985627 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1985634 - Posted: 17 Mar 2019, 15:59:06 UTC - in response to Message 1985627.  

I've been watching the deletions and purges. That used to be what slowed access to your task views. But they have been very constant since the memory upgrade and servers upset recovery. Not much deviation in any of the weekly Haveland graphs. So unsure what is causing the slowdown now.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1985634 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1985832 - Posted: 18 Mar 2019, 17:53:49 UTC - in response to Message 1985822.  

The website was "snappy" when they first recovered and all the hardware was running. But the web pages for tasks are taking much longer to update or show now. Hope that after tomorrows recovery that the snappiness returns.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1985832 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24875
Credit: 3,081,182
RAC: 7
Ireland
Message 1985860 - Posted: 18 Mar 2019, 20:12:13 UTC - in response to Message 1985832.  

Clicking all the "tasks" tabs here, no lag seen.
ID: 1985860 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1985863 - Posted: 18 Mar 2019, 20:15:49 UTC - in response to Message 1985860.  

Quite a large difference between your account and mine. My database is much larger so takes much longer to read and refresh.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1985863 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1985866 - Posted: 18 Mar 2019, 20:34:13 UTC - in response to Message 1985863.  

mine should be pretty close to yours. I'm not having too much trouble loading tasks. seems about normal to me.

viewing individual host stats, all categories of tasks load withing 5 seconds.

the only time i saw a little more lag, was when viewing all host stats aggregated together (nearly 80k tasks). it still loads each category, but maybe 10-15s lag sometimes.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1985866 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1985868 - Posted: 18 Mar 2019, 20:42:36 UTC
Last modified: 18 Mar 2019, 20:45:45 UTC

In my case, try to see the pendings for example is simply impossible, you click, wait, wait & wait and nothing happening.
I just try now. Wait more than 15 secs and nothing...
But without warning sometimes it's works. Never understand why.
My host has a dedicated 160 DL /10 UL MB connection. So i imagine connection speed is out of the scope here.

<edit>Just because i spoke about, just tried again and after 10 secs the task list appears.
I imagine has something to do with the servers load.
ID: 1985868 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1985873 - Posted: 18 Mar 2019, 20:58:29 UTC

Yes, might be when I hit the database. My timing might be bad. Just now accessed my 73K tasks and it took 43 seconds to populate. I do that every day for my AP task collection for the day post. Getting an individual hosts tasks takes much shorter. But my comment about now and last week was for the same database access of total tasks on the account. The number of tasks hasn't changed but is taking longer now. I am chalking it up to a more fragmented database as a guess.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1985873 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24875
Credit: 3,081,182
RAC: 7
Ireland
Message 1985999 - Posted: 19 Mar 2019, 19:06:36 UTC

Another fast outage.
ID: 1985999 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11358
Credit: 29,581,041
RAC: 66
United States
Message 1986001 - Posted: 19 Mar 2019, 19:10:06 UTC

It's back, but connecting to upload and report is an issue.
ID: 1986001 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1986002 - Posted: 19 Mar 2019, 19:13:03 UTC - in response to Message 1986001.  

yup. uploads haven't been going through the whole outage.

large cache of uploads stacking up, will likely prevent any downloads if they can't be flushed out and reported.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1986002 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1986014 - Posted: 19 Mar 2019, 20:10:21 UTC

Master database queries/second 4,259

4000+ is just too much for the system to handle. A bunch will have to get put off in backoff timing. It might need some tweaking of how this is done and for how long to deal with after outage recovery. I try and keep my system out of communications with the server for as long as I can on Tuesdays just to stay out of the way of those who need WUs. Saw on another thread that there are others who do that too... might be worth a thread to get more people to use the "Daily Schedules" feature to limit communications on Tuesday if they aren't in the likely to run out of WUs during a maintenance shutdown group.
ID: 1986014 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1986016 - Posted: 19 Mar 2019, 20:13:39 UTC

Yes we got into trouble the first part of the outage with the upload server offline along with the rest of the project. Lots of stalled uploads.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1986016 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11358
Credit: 29,581,041
RAC: 66
United States
Message 1986025 - Posted: 19 Mar 2019, 21:32:23 UTC

2 hrs after coming back uploads are still borked.
ID: 1986025 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1986026 - Posted: 19 Mar 2019, 21:56:12 UTC - in response to Message 1986001.  

It's back, but connecting to upload and report is an issue.


. . My rigs were OK before the outage with uploads but all during the outage and now they are having big problems uploading. Constantly hanging and going into system backoffs.

Stephen

:(
ID: 1986026 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1986032 - Posted: 19 Mar 2019, 23:21:25 UTC - in response to Message 1986014.  
Last modified: 19 Mar 2019, 23:27:00 UTC

...might be worth a thread to get more people to use the "Daily Schedules" feature to limit communications on Tuesday if they aren't in the likely to run out of WUs during a maintenance shutdown group.


Ok, I will byte :)

How do you use the local manager settings for the network over-ride, to shut down communications? I am getting the impression that the "time" needs to be based off of UTC rather than the local time equivalent?

Since my local UTC is -5 or -6 and they go down about 8am and come back up about 3pm (or earlier)[on short days] by my local time, what should my "day/time" setting be?

Tom
A proud member of the OFA (Old Farts Association).
ID: 1986032 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1986042 - Posted: 20 Mar 2019, 1:02:53 UTC - in response to Message 1986032.  

The time you suspend communications should also factor in how long your machine can go before running out of WUs to work on. I will admit that I turn communications back on at 23:45 UTC so that I can upload my finished WUs to keep my daily graph pretty. I also need to adjust my time as we had daylight savings and I didn't change it. You are right though that the setting takes your local time to act on and this needs to be thought of when the outage usually hits you as it is on California time that the events happen on a Tuesday.
ID: 1986042 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1849
Credit: 268,616,081
RAC: 1,349
United States
Message 1986043 - Posted: 20 Mar 2019, 1:34:41 UTC
Last modified: 20 Mar 2019, 1:38:21 UTC

Definitely still an issue with uploads
[edit] Who'd have thought posting a message would help :) [/edit]
ID: 1986043 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1986045 - Posted: 20 Mar 2019, 1:43:49 UTC - in response to Message 1986032.  
Last modified: 20 Mar 2019, 1:49:48 UTC

...might be worth a thread to get more people to use the "Daily Schedules" feature to limit communications on Tuesday if they aren't in the likely to run out of WUs during a maintenance shutdown group.


Ok, I will byte :)

How do you use the local manager settings for the network over-ride, to shut down communications? I am getting the impression that the "time" needs to be based off of UTC rather than the local time equivalent?

Since my local UTC is -5 or -6 and they go down about 8am and come back up about 3pm (or earlier)[on short days] by my local time, what should my "day/time" setting be?

Tom

You use your local time zone in the configuration (use the 24hr format)

I am at UTC -5 too and this are my settings: (in the global_prefs_override.xml file)

<day_prefs>
      <day_of_week>2</day_of_week>
      <net_start_hour>22.00</net_start_hour>
      <net_end_hour>8.00</net_end_hour>
</day_prefs>


It stops the network around the outage start time 8 AM and return after 14 hrs at 22:00 hrs (10 PM).

But i run with the spoofed builds so my large cache handle the long delay to restore the network access.

In your case, to return at 3PM as you posted, your must use: <net_start_hour>15.00</net_start_hour>
ID: 1986045 · Report as offensive
Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 31 · Next

Message boards : Number crunching : Panic Mode On (115) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.