Panic Mode On (114) Server Problems?

Message boards : Number crunching : Panic Mode On (114) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 14 · 15 · 16 · 17 · 18 · 19 · 20 . . . 45 · Next

AuthorMessage
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1972090 - Posted: 27 Dec 2018, 1:39:06 UTC - in response to Message 1972088.  

I have been doing some playing around today and added
<max_tasks_reported>50</max_tasks_reported>
Which eliminated the HTTP internal error message, by slowing down the reporting.

I have had all my 4000+ tasks in for about 20 minutes now.
Getting a trickle come in now, with a couple of big pulls.


. . Absolutely nothing here on all 4 machines. Within the hour they will begin to fall over with no work.

Stephen

:(
ID: 1972090 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1972091 - Posted: 27 Dec 2018, 1:51:43 UTC - in response to Message 1972088.  

I've found I can usually get away with<max_tasks_reported>100</max_tasks_reported> and have the scheduler accept it with no problems. I don't have as much luck with values higher than that though. Still takes a considerable amount of connections to report all the tasks crunched during such a long outage.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1972091 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1649
Credit: 12,921,799
RAC: 89
New Zealand
Message 1972100 - Posted: 27 Dec 2018, 2:57:19 UTC - in response to Message 1971561.  
Last modified: 27 Dec 2018, 2:59:30 UTC


sometimes I find it tricky to trigger the double resend I will give it a go

I set 178 tasks free by abandoning them
ID: 1972100 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14039
Credit: 208,696,464
RAC: 304
Australia
Message 1972112 - Posted: 27 Dec 2018, 5:00:35 UTC
Last modified: 27 Dec 2018, 5:02:15 UTC

Forums have gone in to super slow motion mode, and the project is no longer issuing work (again), as of about 20min ago.
27/12/2018 14:15:08 | SETI@home | Scheduler request completed: got 0 new tasks
Grant
Darwin NT
ID: 1972112 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1972113 - Posted: 27 Dec 2018, 5:06:55 UTC - in response to Message 1972112.  

Forums have gone in to super slow motion mode, and the project is no longer issuing work (again), as of about 20min ago.
27/12/2018 14:15:08 | SETI@home | Scheduler request completed: got 0 new tasks


yup. This isn't a good recovery. Hoping things will fix themselves, but worried we might need to send out the Seti signal. It is 9pm in CA. Not sure if that is too late.
ID: 1972113 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14039
Credit: 208,696,464
RAC: 304
Australia
Message 1972117 - Posted: 27 Dec 2018, 5:47:36 UTC

And now it appears to be functioning again.
Grant
Darwin NT
ID: 1972117 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1972121 - Posted: 27 Dec 2018, 7:46:02 UTC - in response to Message 1972100.  


sometimes I find it tricky to trigger the double resend I will give it a go

I set 178 tasks free by abandoning them


. . Abandoning or aborting? I know how to abort tasks but not how to abandon a task.

Stephen

??
ID: 1972121 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1649
Credit: 12,921,799
RAC: 89
New Zealand
Message 1972122 - Posted: 27 Dec 2018, 7:55:49 UTC - in response to Message 1972121.  
Last modified: 27 Dec 2018, 7:57:38 UTC


sometimes I find it tricky to trigger the double resend I will give it a go

I set 178 tasks free by abandoning them


. . Abandoning or aborting? I know how to abort tasks but not how to abandon a task.

Stephen

??

All the tasks were ghosts so I detached from the project and reattached which caused the ghost tasks to error out. Here is an example http://setiathome.berkeley.edu/result.php?resultid=7268578562
ID: 1972122 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14039
Credit: 208,696,464
RAC: 304
Australia
Message 1972123 - Posted: 27 Dec 2018, 7:59:49 UTC - in response to Message 1972122.  

All the tasks were ghosts so I detached from the project and reattached which caused the ghost tasks to error out.

Why?
The same thing would have occurred when they reached their due dates.
Grant
Darwin NT
ID: 1972123 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1649
Credit: 12,921,799
RAC: 89
New Zealand
Message 1972124 - Posted: 27 Dec 2018, 8:38:29 UTC - in response to Message 1972123.  

All the tasks were ghosts so I detached from the project and reattached which caused the ghost tasks to error out.

Why?
The same thing would have occurred when they reached their due dates.

I was just trying to help the database not be so bloated and not making wing people having to wait till next year for credit. Just trying to be an all round good person
ID: 1972124 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 38784
Credit: 261,360,520
RAC: 489
Australia
Message 1972126 - Posted: 27 Dec 2018, 9:02:26 UTC

Did I miss another hiccup again?

Cheers.
ID: 1972126 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14039
Credit: 208,696,464
RAC: 304
Australia
Message 1972130 - Posted: 27 Dec 2018, 9:17:23 UTC - in response to Message 1972126.  

Did I miss another hiccup again?

Yep.
Grant
Darwin NT
ID: 1972130 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 14039
Credit: 208,696,464
RAC: 304
Australia
Message 1972131 - Posted: 27 Dec 2018, 9:20:20 UTC - in response to Message 1972124.  

I was just trying to help the database not be so bloated and not making wing people having to wait till next year for credit. Just trying to be an all round good person

A couple of hundred tasks out of 5million or so isn't too much of an issue.
If it's something that keeps occurring you'd want to look in to it, but for a one off thing? Not worth worrying about IMHO.
Grant
Darwin NT
ID: 1972131 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1972145 - Posted: 27 Dec 2018, 12:41:50 UTC - in response to Message 1972123.  

All the tasks were ghosts so I detached from the project and reattached which caused the ghost tasks to error out.

Why?
The same thing would have occurred when they reached their due dates.


. . So the tasks do NOT sit in limbo for 2 months and cause his wingmen undue delays ...

Stephen

? ?
ID: 1972145 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1972146 - Posted: 27 Dec 2018, 13:33:36 UTC

I thought it was a good idea Speedy. I know when I got ghosts I did what I could so my wingman wouldn't have to wait forever, and so they would clear the db sooner. The system has been having enough hiccups lately as it is, I would think every little bit would help.

To stay on topic: I will note that I think the creation rate it is a bit low. It might meet demand, or might slowly dwindle, but it needs a kick in the pants to build the RTS up to normal.
ID: 1972146 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1972148 - Posted: 27 Dec 2018, 13:50:27 UTC - in response to Message 1972087.  

Almost have all tasks reported, still about 500 to go on my last unreported host. Still getting nothing on all requests.


. . I found that when I was having trouble reporting I set my caches very low so they would not request work, all the rigs then reported AOK. I noticed this because I had already set the Linux caches low as they had sufficient tasks and they were reporting just fine but the Windows boxes were not. So I set the windows boxes to minimum and then they were able to report we well.

. . But like you, despite the fact that the RTS is overflowing there are still "no tasks available"

Stephen

:(
ID: 1972148 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1972150 - Posted: 27 Dec 2018, 14:24:07 UTC - in response to Message 1972126.  

Did I miss another hiccup again?
Cheers.
It's almost like you're shutting the system down when you go to bed ....
ID: 1972150 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1972151 - Posted: 27 Dec 2018, 14:28:28 UTC

I wonder if they set the RTS cache to 100k to see if less stress in the files system would help the server performance.
It sure seems that way to my eye.
ID: 1972151 · Report as offensive
Profile Chris904395093209d Project Donor
Volunteer tester

Send message
Joined: 1 Jan 01
Posts: 112
Credit: 29,923,129
RAC: 6
United States
Message 1972166 - Posted: 27 Dec 2018, 16:12:17 UTC - in response to Message 1972151.  

I wonder if they set the RTS cache to 100k to see if less stress in the files system would help the server performance.
It sure seems that way to my eye.


I was kinda wondering the same thing. Usually by this time the day after the weekly maint, the RTS cache is back to normal of around 500k.
~Chris

ID: 1972166 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1972189 - Posted: 27 Dec 2018, 20:27:01 UTC - in response to Message 1972150.  

Did I miss another hiccup again?
Cheers.
It's almost like you're shutting the system down when you go to bed ....


. . Here I have the opposite issue
, this machine 'shuts itself down' during the day when the heat gets too much to bear ...

Stephen

:(
ID: 1972189 · Report as offensive
Previous · 1 . . . 14 · 15 · 16 · 17 · 18 · 19 · 20 . . . 45 · Next

Message boards : Number crunching : Panic Mode On (114) Server Problems?


 
©2026 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.