Panic Mode On (99) Server Problems?

Message boards : Number crunching : Panic Mode On (99) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 20 · 21 · 22 · 23 · 24 · 25 · 26 · Next

AuthorMessage
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1714126 - Posted: 16 Aug 2015, 23:43:05 UTC

Replica is back, sort of. 94,538 seconds behind master.
ID: 1714126 · Report as offensive
Scarecrow

Send message
Joined: 15 Jul 00
Posts: 4520
Credit: 486,601
RAC: 0
United States
Message 1714132 - Posted: 16 Aug 2015, 23:53:20 UTC - in response to Message 1714126.  

Replica is back, sort of. 94,538 seconds behind master.

Heck, in just 26 hours it'll be all caught up. :)
ID: 1714132 · Report as offensive
Andrew Scharbarth
Volunteer tester

Send message
Joined: 29 May 07
Posts: 40
Credit: 5,984,436
RAC: 0
United States
Message 1714137 - Posted: 17 Aug 2015, 0:02:33 UTC
Last modified: 17 Aug 2015, 0:06:12 UTC

Hopefully it'll start cleaning up the backlog of stuff that needs to be purged now. 13.3 million database entries can't be helping server performance.
ID: 1714137 · Report as offensive
Scarecrow

Send message
Joined: 15 Jul 00
Posts: 4520
Credit: 486,601
RAC: 0
United States
Message 1714139 - Posted: 17 Aug 2015, 0:07:31 UTC

For the last 90 minutes I'm getting only...

Scheduler request completed: got 0 new tasks

Server can't open database

Project has no tasks available
ID: 1714139 · Report as offensive
Andrew Scharbarth
Volunteer tester

Send message
Joined: 29 May 07
Posts: 40
Credit: 5,984,436
RAC: 0
United States
Message 1714141 - Posted: 17 Aug 2015, 0:09:57 UTC - in response to Message 1714139.  

For the last 90 minutes I'm getting only...

Scheduler request completed: got 0 new tasks

Server can't open database

Project has no tasks available



There were only 300k tasks in the buffer and 250K+ got reported when the servers came back up. Wouldn't be surprised if they simply just got sucked up.
ID: 1714141 · Report as offensive
Profile Zombu2
Volunteer tester

Send message
Joined: 24 Feb 01
Posts: 1615
Credit: 49,315,423
RAC: 0
United States
Message 1714161 - Posted: 17 Aug 2015, 0:48:28 UTC

damn
I came down with a bad case of i don't give a crap
ID: 1714161 · Report as offensive
raydar115

Send message
Joined: 6 Oct 02
Posts: 17
Credit: 16,305,128
RAC: 0
United States
Message 1714164 - Posted: 17 Aug 2015, 0:56:37 UTC

Well I just uploaded and reported 120 tasks with no problem with the exception of it only verifying about 25 of them and still shows my maximum of 201 work units unreported? Have received about 25 new work units and still says I'm at my maximum allowed hope they show up and not been abducted by aliens
ID: 1714164 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1714166 - Posted: 17 Aug 2015, 0:59:04 UTC - in response to Message 1714164.  

That displaying of the status of the tasks is done by the replica database, which is still some 90,000+ seconds behind on the master database. Once they're somewhat in sync again, will every task show up again.
ID: 1714166 · Report as offensive
Andrew Scharbarth
Volunteer tester

Send message
Joined: 29 May 07
Posts: 40
Credit: 5,984,436
RAC: 0
United States
Message 1714186 - Posted: 17 Aug 2015, 1:34:01 UTC - in response to Message 1714166.  
Last modified: 17 Aug 2015, 1:35:15 UTC

That displaying of the status of the tasks is done by the replica database, which is still some 90,000+ seconds behind on the master database. Once they're somewhat in sync again, will every task show up again.


Well, it went down by 2000 seconds from the last update, and queries from master went up to 7,000. That should even out pretty soon.
ID: 1714186 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1714188 - Posted: 17 Aug 2015, 1:39:38 UTC - in response to Message 1714186.  

which is still some 90,000+ seconds

That's 25 hrs.
ID: 1714188 · Report as offensive
Andrew Scharbarth
Volunteer tester

Send message
Joined: 29 May 07
Posts: 40
Credit: 5,984,436
RAC: 0
United States
Message 1714189 - Posted: 17 Aug 2015, 1:46:20 UTC - in response to Message 1714188.  

which is still some 90,000+ seconds

That's 25 hrs.


Doesn't mean it will take that long to sync back up.
ID: 1714189 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1714190 - Posted: 17 Aug 2015, 1:50:25 UTC - in response to Message 1714189.  

which is still some 90,000+ seconds

That's 25 hrs.


Doesn't mean it will take that long to sync back up.

I'd be very pleased if they could do it in 12 hrs.
ID: 1714190 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1714281 - Posted: 17 Aug 2015, 6:26:07 UTC - in response to Message 1714190.  

For some reason, a few hours before the weekly outage, results started (finally) to be purged.
However since the outage, that has stopped, and now the number to be purged continues to grow at a steady rate.
Grant
Darwin NT
ID: 1714281 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1714374 - Posted: 17 Aug 2015, 10:48:25 UTC - in response to Message 1714281.  

Results waiting to be purged, still heading upwards.
Grant
Darwin NT
ID: 1714374 · Report as offensive
WezH
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 576
Credit: 67,033,957
RAC: 95
Finland
Message 1714395 - Posted: 17 Aug 2015, 11:28:51 UTC - in response to Message 1714374.  
Last modified: 17 Aug 2015, 11:33:57 UTC

Results waiting to be purged, still heading upwards.


Yep, 7:10:03 UTC it was 13,823,385, now at 11:20:03 UTC 14,162,579.

Could be 2^32 problem?

And replica is struggling to catch up, sometime ago it was about 62K sec late, now it's back up to 63.8K sec...
ID: 1714395 · Report as offensive
Profile ElricM
Volunteer tester

Send message
Joined: 4 Oct 03
Posts: 4
Credit: 607,981,200
RAC: 228
Germany
Message 1714411 - Posted: 17 Aug 2015, 12:15:33 UTC - in response to Message 1712905.  


I have only 3 out of 9 rigs running right now.
Daily driver dead due to technical issues as yet unresolved.
The other 5 are shut down due to the temps here being around 90f for the next few days, and I don't want the crunchers, the kitties, or myself melting down.


Come on, MSattler and buy a air condition :) My 6kW cooling system is struggling against 14 GPU's :)
I miss your crunching power at WoW, only playing with tbret is not enough :)

Regards
ElricM
ID: 1714411 · Report as offensive
WezH
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 576
Credit: 67,033,957
RAC: 95
Finland
Message 1714426 - Posted: 17 Aug 2015, 13:06:17 UTC - in response to Message 1714395.  

And replica is struggling to catch up, sometime ago it was about 62K sec late, now it's back up to 63.8K sec...


And bang, at 13:00:03 UTC replica is 47,933 sec behind master.

Something did happend, but will it continue?
ID: 1714426 · Report as offensive
WezH
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 576
Credit: 67,033,957
RAC: 95
Finland
Message 1714479 - Posted: 17 Aug 2015, 14:44:29 UTC - in response to Message 1714426.  
Last modified: 17 Aug 2015, 14:46:25 UTC

Reply to myself:

Yes, it's continuing, now 14:40:03 UTC 25,976 sec behind. So "only" 7.2 hours behind master database....

But as Grant said, results waiting to be purged, still heading upwards
ID: 1714479 · Report as offensive
WezH
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 576
Credit: 67,033,957
RAC: 95
Finland
Message 1714540 - Posted: 17 Aug 2015, 17:04:34 UTC - in response to Message 1714479.  

Again, reply to myself.

SSP shows us now that replica is now "on time"

But there is lots if "Burgers" to consume for "db purge: vader"
ID: 1714540 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1714554 - Posted: 17 Aug 2015, 17:34:57 UTC - in response to Message 1714540.  

Anybody see the front page news about the project shutdown this evening till Wednesday morning?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1714554 · Report as offensive
Previous · 1 . . . 20 · 21 · 22 · 23 · 24 · 25 · 26 · Next

Message boards : Number crunching : Panic Mode On (99) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.