Panic Mode On (110) Server Problems?

Message boards : Number crunching : Panic Mode On (110) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 37 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1915006 - Posted: 25 Jan 2018, 2:37:05 UTC

Is the reason that no stats have been exported today is that the replica has been stuck at 44,692 seconds since the project came back from the outage?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1915006 · Report as offensive
Kiska
Volunteer tester

Send message
Joined: 31 Mar 12
Posts: 302
Credit: 3,067,762
RAC: 0
Australia
Message 1915011 - Posted: 25 Jan 2018, 3:04:47 UTC - in response to Message 1915006.  
Last modified: 25 Jan 2018, 3:07:27 UTC

Is the reason that no stats have been exported today is that the replica has been stuck at 44,692 seconds since the project came back from the outage?


I would assume so, since stats is generated on replica instead of master
Its getting better only 43k seconds behind!

Also RTS buffer looks to be recovering nicely 330k ready to send
ID: 1915011 · Report as offensive
Profile Ghan-buri-Ghan Mike

Send message
Joined: 27 Dec 15
Posts: 123
Credit: 92,602,985
RAC: 172
United States
Message 1915111 - Posted: 25 Jan 2018, 17:30:03 UTC - in response to Message 1914978.  

At least you are still in the leaderboard in the computing tab. I have disappeared completely. RAC should have me in the 170s somewhere.
ID: 1915111 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1915112 - Posted: 25 Jan 2018, 17:34:28 UTC - in response to Message 1915111.  

Yes, the Linux cruncher does the heavy lifting. I almost fell off the first page when I was down to #20. My second biggest producer is down 30K from its normal RAC.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1915112 · Report as offensive
Profile Ghan-buri-Ghan Mike

Send message
Joined: 27 Dec 15
Posts: 123
Credit: 92,602,985
RAC: 172
United States
Message 1915130 - Posted: 25 Jan 2018, 19:48:07 UTC - in response to Message 1915112.  

I have re-appeared at # 182.....
ID: 1915130 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1915180 - Posted: 25 Jan 2018, 23:53:57 UTC - in response to Message 1915139.  
Last modified: 26 Jan 2018, 0:00:33 UTC

I've fallen, and I refuse to get up.


. . Slacker ... where's your sense of fun?

. . Overall I have dropped way back, down to page 5 at #81

. . By computer I am still on page 2 at #38 :)

Stephen

:)
ID: 1915180 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 1915254 - Posted: 26 Jan 2018, 10:12:19 UTC

I hope we get some more files loaded for the weekend. The rate we're going through work lately the current batch may not make it through.
Grant
Darwin NT
ID: 1915254 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 1915255 - Posted: 26 Jan 2018, 10:13:30 UTC - in response to Message 1915139.  

I've fallen, and I refuse to get up.

That's better than the usual "I've fallen and can't get up. Pass me my drink"
Grant
Darwin NT
ID: 1915255 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 1915330 - Posted: 26 Jan 2018, 21:01:53 UTC
Last modified: 26 Jan 2018, 21:10:00 UTC

Maybe they were coming to the end dataset? Because tape blc05_2bit_guppi_57977_43520_3C138_0006 is only 11.56 GB. As I write this tape has not started to be split. It will be interesting to see how long tasks run for. I currently have not got any tasks on board however recently on my 970 they have been running at around 7 minutes 10 seconds
ID: 1915330 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 1915332 - Posted: 26 Jan 2018, 21:13:22 UTC - in response to Message 1915254.  

I hope we get some more files loaded for the weekend. The rate we're going through work lately the current batch may not make it through.

Currently there was 5490 channels remaining to be split
ID: 1915332 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 1915352 - Posted: 26 Jan 2018, 22:30:32 UTC - in response to Message 1915332.  

I hope we get some more files loaded for the weekend. The rate we're going through work lately the current batch may not make it through.

Currently there was 5490 channels remaining to be split

Good, they loaded some extra files to get us through the weekend.
Grant
Darwin NT
ID: 1915352 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 1915503 - Posted: 27 Jan 2018, 22:24:31 UTC - in response to Message 1915332.  
Last modified: 27 Jan 2018, 22:26:03 UTC

Currently there was 5490 channels remaining to be split

Now down to 3420.
As the Americans say, we're going through them "like a bear through berries."
Roughly 2,000 every 24hrs.
Grant
Darwin NT
ID: 1915503 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1915506 - Posted: 27 Jan 2018, 22:30:07 UTC - in response to Message 1915503.  

Currently there was 5490 channels remaining to be split

Now down to 3420.
As the Americans say, we're going through them "like a bear through berries."
Roughly 2,000 every 24hrs.

Which means that the pickings might be slim Sunday night and before the staff gets in to work on Monday.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1915506 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 1915712 - Posted: 29 Jan 2018, 3:10:32 UTC

Since work is being processed so fast I wonder if we are starting to catch up or at least keep up with the incoming data flow
ID: 1915712 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1915714 - Posted: 29 Jan 2018, 4:07:19 UTC - in response to Message 1915712.  

Don't know. Best guess, we aren't. I think the amount of new data is in the petaflops range and we are only producing in the teraflops range.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1915714 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 1915721 - Posted: 29 Jan 2018, 4:33:27 UTC - in response to Message 1915714.  

Don't know. Best guess, we aren't. I think the amount of new data is in the petaflops range and we are only producing in the teraflops range.

If that is the case I completely agree with you
ID: 1915721 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1915729 - Posted: 29 Jan 2018, 5:14:42 UTC

This being true that we're awash in data needing to be processed, it'll be nice when the day finally comes where the back end/database can keep up with all the processing capacity out here, and relax the current 100 task limit somewhat. Or, somehow make the WU's 'harder', where it wouldn't be an issue any longer as they would each take longer to process. Although with the goal of processing on almost any device, that might be an issue unless changes are made which would recognize which ones could handle it, and which ones couldn't. Smells like a lot of work to me, which means it's probably going to be a while before this becomes a reality.

ID: 1915729 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1915756 - Posted: 29 Jan 2018, 11:02:44 UTC - in response to Message 1915729.  

This being true that we're awash in data needing to be processed, it'll be nice when the day finally comes where the back end/database can keep up with all the processing capacity out here, and relax the current 100 task limit somewhat. Or, somehow make the WU's 'harder', where it wouldn't be an issue any longer as they would each take longer to process. Although with the goal of processing on almost any device, that might be an issue unless changes are made which would recognize which ones could handle it, and which ones couldn't. Smells like a lot of work to me, which means it's probably going to be a while before this becomes a reality.


. . Well looking at the tapes mounted at the moment there will be famine before the lads get to the lab tomorrow ...

Stephen

:(
ID: 1915756 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 5375
Credit: 30,870,693
RAC: 1
United States
Message 1915778 - Posted: 29 Jan 2018, 15:07:16 UTC

It also appears that dear Carolyn has been having indigestion off and on the last few days......replica falls behind to upwards of a hour then catches up.

"Sour Grapes make a bitter Whine." <(0)>
ID: 1915778 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1915781 - Posted: 29 Jan 2018, 15:38:47 UTC

Right now, I am most concerned about the splitter queue running dry.
A number of GBT splitters have already shut down due to lack of work to split.
I have sent a heads up to Eric, but do not know if he is at home this week or not.

Meow!
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1915781 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 . . . 37 · Next

Message boards : Number crunching : Panic Mode On (110) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.