Panic Mode On (109) Server Problems?

Message boards : Number crunching : Panic Mode On (109) Server Problems?
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 36 · Next

AuthorMessage
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1905986 - Posted: 9 Dec 2017, 20:20:12 UTC

The warp engines(the database) cannot stand the strain captain, we need to do something...........

ID: 1905986 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1906016 - Posted: 9 Dec 2017, 22:27:58 UTC - in response to Message 1905986.  

The warp engines(the database) cannot stand the strain captain, we need to do something...........

The assimilators are back on line and the Validation/assimilation backlogs have stopped growing. But it looks like it's going to take a while for them to start reducing & eventually clear.
Getting work from the Scheduler is still hit and miss.
Having to do the triple update several times a day now to keep the work coming.

Oh boy- received in the last hour has hit almost 130,000- the usual value (these days) is around 90-100k.
Grant
Darwin NT
ID: 1906016 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1906039 - Posted: 9 Dec 2017, 23:12:16 UTC

I'm still having trouble receiving tasks for my Three Linux machines. Now we're OUT of Arecibo files, and at this rate the GBTs will be OUT by morning. Did I mention tonight's going to be the Coldest night of the season here?
ID: 1906039 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1906043 - Posted: 9 Dec 2017, 23:20:00 UTC - in response to Message 1906039.  
Last modified: 9 Dec 2017, 23:24:12 UTC

I'm still having trouble receiving tasks for my Three Linux machines. Now we're OUT of Arecibo files, and at this rate the GBTs will be OUT by morning. Did I mention tonight's going to be the Coldest night of the season here?

You might get lucky.
Most of my current GBT work hasn't been noise bombs for the last few hours.

Edit- i'll swap some of our heat for your cold.
28°c has been about as cool as it gets overnight for the last few days. 24°c is about the coolest so far this month.
Grant
Darwin NT
ID: 1906043 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1906048 - Posted: 10 Dec 2017, 0:00:52 UTC - in response to Message 1906043.  

ID: 1906048 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906056 - Posted: 10 Dec 2017, 0:26:52 UTC

I'm still processing quite a bit of noisy guppies, though I think they are finally diminishing. My WU weather station page had a news article today about all the snow that the South got recently and the West is going for the second month of the lowest recorded moisture since 1895. Temps here these past couple of weeks are 10-15 degrees above average for the season. Makes for some really nice daytime temps and still low enough nights where you can burrow under the covers for sleep.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906056 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1906057 - Posted: 10 Dec 2017, 0:30:37 UTC - in response to Message 1906048.  

Wind Chill at above freezing ... What bloody Whoosies you are down there.
I remember trying to save our machine shed from blowing down when it was -112F. Now that was F'm COLD, about 5 minutes was about all you could stand before heading to the shop, tearing of a few clothes and stand in front of the furnace for 10 minutes ... then back out again .... BRRRR!

Right now it's 41F here and mph wing .... heck that's just long sleeve shirt weather :D
ID: 1906057 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906063 - Posted: 10 Dec 2017, 0:40:39 UTC

I'll fess up and admit it. I'm a California native and 40° F is "where's my parka" time. I had a few months exposure to Great Lakes winter temps and that was enough to provide a relative scale to what my cold weather tolerance is. NO THANK YOU. I'll stay in balmier climes.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906063 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1906064 - Posted: 10 Dec 2017, 0:49:45 UTC - in response to Message 1906057.  

That reminds me to once again tell my 88 year old bed-ridden father how smart he was to have chosen to live in Florida. If we're lucky the power won't go out and I won't have to pile a couple more blankets on him. Around here it's common to have a power outage to go along with the coldest nights, something about everyone having electric heaters and not prepared for cold weather.
ID: 1906064 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1906078 - Posted: 10 Dec 2017, 1:34:23 UTC - in response to Message 1906056.  

I'm still processing quite a bit of noisy guppies, though I think they are finally diminishing.

Not for me.
I was doing a bunch that were running all the way through, but now i'm going through a bunch of noisy ones again.
Grant
Darwin NT
ID: 1906078 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906091 - Posted: 10 Dec 2017, 2:28:39 UTC - in response to Message 1906078.  
Last modified: 10 Dec 2017, 2:29:11 UTC

I do believe you are correct. Just looked at my BoincTasks History log and it is littered with noise bombs again.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906091 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906229 - Posted: 10 Dec 2017, 17:18:34 UTC

Download servers seem to be offline. I have lots of pending downloads stacking up.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906229 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906232 - Posted: 10 Dec 2017, 17:40:52 UTC - in response to Message 1906231.  

My stalled list keeps growing and I only see one or two successful downloads every 10 or 20 minutes. So, yes the download server is sorta functional but extremely bogged down as you state. Doesn't help in getting work obviously with the stalled downloads on backoff. Caches falling as usual for me it seems.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906232 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1906233 - Posted: 10 Dec 2017, 17:44:44 UTC

I just noticed the d/l problem, too. Both my crunchers are able to upload, but no d/l goodness, even when I do manual requests for work.

Sad!

Did they have snow in Berkeley, too?
ID: 1906233 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11358
Credit: 29,581,041
RAC: 66
United States
Message 1906234 - Posted: 10 Dec 2017, 17:51:02 UTC

No joy on downloads here
ID: 1906234 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906238 - Posted: 10 Dec 2017, 17:58:03 UTC - in response to Message 1906233.  

I just noticed the d/l problem, too. Both my crunchers are able to upload, but no d/l goodness, even when I do manual requests for work.

Sad!

Did they have snow in Berkeley, too?

Snow?? We haven't had any snow in the state of California for over two weeks, even in the Sierras. We have a high pressure ridge locked over us and no precipitation anywhere.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906238 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11358
Credit: 29,581,041
RAC: 66
United States
Message 1906247 - Posted: 10 Dec 2017, 18:26:29 UTC

Downloads got unstuck for me
ID: 1906247 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906257 - Posted: 10 Dec 2017, 19:57:55 UTC - in response to Message 1906247.  

Yes, the backlog has finally cleared for me too. Am getting occasional no work messages though.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906257 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1906374 - Posted: 11 Dec 2017, 7:17:06 UTC
Last modified: 11 Dec 2017, 7:19:39 UTC

Add to the download issues; the Validation/assimilation backlog has been clearing, slowly. Very slowly. However now there is a backlog of WU (AP and MB) awaiting deletion building up. And the replica, after falling behind and catching up again is now back to falling behind, even further behind than previously. Oh, and the Ready-to-send buffer has been rather up and down as the splitters have been unable to crank out the work again.
The system isn't looking too healthy.
And if it's all to do with the current noise bombs resulting in the record received-last-hour numbers, it doesn't bode well for the future if we do get a significant boost in active cruncher numbers to process the new work.

EDIT- and I forgot to mention the application settings related issue that began late last Dec that affects some but not others is still rearing it's unwelcome head. Had to triple update (again) as my cache was running down (again).
Grant
Darwin NT
ID: 1906374 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1906388 - Posted: 11 Dec 2017, 8:18:21 UTC

I noticed the numbers on the SSP too not being in very healthy shape for some parameters and getting better for others. The site is slow as molasses and you can't look at any hosts' tasks now without the connection timing out.

Hope they can clear the cobwebs out of the system on Tuesday and that the project stays up for Monday without further damage.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1906388 · Report as offensive
1 · 2 · 3 · 4 . . . 36 · Next

Message boards : Number crunching : Panic Mode On (109) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.