Panic Mode On (109) Server Problems?

Message boards : Number crunching : Panic Mode On (109) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 28 · 29 · 30 · 31 · 32 · 33 · 34 . . . 36 · Next

AuthorMessage
Profile Stargate (SA)
Volunteer tester
Avatar

Send message
Joined: 4 Mar 10
Posts: 1854
Credit: 2,258,721
RAC: 0
Australia
Message 1913266 - Posted: 15 Jan 2018, 22:19:34 UTC

Ah ok, ty for reply
ID: 1913266 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1913277 - Posted: 16 Jan 2018, 0:13:16 UTC - in response to Message 1913263.  

How does one have so many errors and invalids?


. . By being a developer (Petri) and constantly trying new things ... or in other cases by having faulty hardware or totally misconfigured apps that trash most of the WUs you receive, there are many of them around too.

Stephen

:)
ID: 1913277 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1913298 - Posted: 16 Jan 2018, 4:51:22 UTC
Last modified: 16 Jan 2018, 5:16:57 UTC

Is it just me, or are these new 02 WUs processing even faster than the previous 05s? Around 10sec on the GPUs and 4min on the CPU faster.
The servers weren't doing too well as it was, these new WUs will push them even harder.
Grant
Darwin NT
ID: 1913298 · Report as offensive
Profile Stargate (SA)
Volunteer tester
Avatar

Send message
Joined: 4 Mar 10
Posts: 1854
Credit: 2,258,721
RAC: 0
Australia
Message 1913301 - Posted: 16 Jan 2018, 5:05:05 UTC

Id say your right Grant..
ID: 1913301 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1913315 - Posted: 16 Jan 2018, 8:31:54 UTC

Looks like we've reached that critical point of returned-last-hour, WU-waiting-deletion, in-progress and splitter load.
Waiting-deletion is going up, splitter output has gone down, and Ready-to-send buffer is rapidly emptying. And with the shorter WU run times, channels left to be split is rapidly diminishing also.
Grant
Darwin NT
ID: 1913315 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22505
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1913318 - Posted: 16 Jan 2018, 9:27:22 UTC

Eric muttered something in Beta a few days ago about finalising some "new binaries" that might be loaded onto the servers this Tuesday - perhaps he wants to have a fairly low RTS before doing the update....
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1913318 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1913319 - Posted: 16 Jan 2018, 9:29:40 UTC - in response to Message 1913318.  

Eric muttered something in Beta a few days ago about finalising some "new binaries" that might be loaded onto the servers this Tuesday - perhaps he wants to have a fairly low RTS before doing the update....

I hope the implementation is smoother here than at Beta.
Grant
Darwin NT
ID: 1913319 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1913328 - Posted: 16 Jan 2018, 10:19:14 UTC - in response to Message 1913319.  

Eric muttered something in Beta a few days ago about finalising some "new binaries" that might be loaded onto the servers this Tuesday - perhaps he wants to have a fairly low RTS before doing the update....
I hope the implementation is smoother here than at Beta.
No new applications have appeared on the Beta applications page since October 2017, and those were all for Mac - so whatever the current problems are at Beta, it can't be attributed to new application binaries.

What Eric actually said was

We're testing BOINC updates, which have broken the splitters.

We'll probably promote these server binaries to the main project next week.
ID: 1913328 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22505
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1913333 - Posted: 16 Jan 2018, 10:50:28 UTC

Thanks Richard - I was on my phone and looking at two things at once on that is "problematic" - hopefully the new SERVER binaries will address some of the issues we've seen on main of late. (And Eric's note post dates the "new splitters broke Beta" announcement by a few weeks, and the more recent Beta stuff has flowed fairly smoothly....
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1913333 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1913371 - Posted: 17 Jan 2018, 0:44:41 UTC

and we are back
ID: 1913371 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1913376 - Posted: 17 Jan 2018, 1:07:04 UTC

But the splitters are offline. If Eric did deploy new server code, I hope we are not seeing the same issue that Beta had with the new server code.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1913376 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1913383 - Posted: 17 Jan 2018, 1:38:43 UTC
Last modified: 17 Jan 2018, 1:46:19 UTC

And the splitters are back online. But the SSP has not updated in 15 minutes. So who knows?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1913383 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11414
Credit: 29,581,041
RAC: 66
United States
Message 1913387 - Posted: 17 Jan 2018, 2:06:12 UTC

Current result creation rate ** 0/sec 0.0148/sec 2.5690/sec 6m

This is not gud.
ID: 1913387 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1913389 - Posted: 17 Jan 2018, 2:17:25 UTC - in response to Message 1913298.  

Is it just me, or are these new 02 WUs processing even faster than the previous 05s? Around 10sec on the GPUs and 4min on the CPU faster.
The servers weren't doing too well as it was, these new WUs will push them even harder.


. . Yes, they are processing slightly faster than those Blc05 units were. Not quite so dramatic here though they are around 30 secs faster on the 950 GPU and about one min on the i5-6400, maybe 10 secs on the 970s and 2 mins on the i5-6600, and finally about 10 to 12 secs faster on the 1050ti, the C2D doesn't crunch normally but during the outage it ran a couple of Blc02s and they were about 2 to 5 mins quicker (compared to 81 to 84 mins).

Stephen

PS:- the post outage hourly return nearly hit the 200,000 mark :(
ID: 1913389 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1913390 - Posted: 17 Jan 2018, 2:19:42 UTC - in response to Message 1913315.  

Looks like we've reached that critical point of returned-last-hour, WU-waiting-deletion, in-progress and splitter load.
Waiting-deletion is going up, splitter output has gone down, and Ready-to-send buffer is rapidly emptying. And with the shorter WU run times, channels left to be split is rapidly diminishing also.


. . On that subject (channels to be split) I am left pondering why it is that all the "tapes" (Disks) show as 52.24 GB but while some show 80 to 120 channels others show only one or two. What causes that level of discrepancy?

Stephen

??
ID: 1913390 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1913400 - Posted: 17 Jan 2018, 2:54:20 UTC - in response to Message 1913383.  
Last modified: 17 Jan 2018, 3:22:48 UTC

And the splitters are back online. But the SSP has not updated in 15 minutes. So who knows?

Splitters showing Green, and status page has updated. But still no work actually coming from the splitters.
Server code update broke our splitters as well as Betas?


Edit- over 2 hours since the project came back up, but still no splitter output.
Grant
Darwin NT
ID: 1913400 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1913402 - Posted: 17 Jan 2018, 2:56:27 UTC - in response to Message 1913390.  

. . On that subject (channels to be split) I am left pondering why it is that all the "tapes" (Disks) show as 52.24 GB but while some show 80 to 120 channels others show only one or two. What causes that level of discrepancy?

Only those in progress (dark green), or have been processed (light green), show up.
Grant
Darwin NT
ID: 1913402 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1913411 - Posted: 17 Jan 2018, 3:45:40 UTC
Last modified: 17 Jan 2018, 3:46:25 UTC

Apparently the splitters are splitting as showed by the Splitter status but where the splitted data is going?
ID: 1913411 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11414
Credit: 29,581,041
RAC: 66
United States
Message 1913412 - Posted: 17 Jan 2018, 3:50:37 UTC

I made it thru todays outrage but it looks like I will go dry this evening unless something gets fixed.
ID: 1913412 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13847
Credit: 208,696,464
RAC: 304
Australia
Message 1913414 - Posted: 17 Jan 2018, 4:09:37 UTC - in response to Message 1913411.  

Apparently the splitters are splitting as showed by the Splitter status but where the splitted data is going?

I think it's a case of the splitters are running, but they're not actually doing anything.

Apparently, the updated server code broke the splitters on Beta.
They got that fixed, but it would appear there's some differences between the splitters here & at Beta that didn't cope too well with the new code.
It could be a while before work is flowing again.
Grant
Darwin NT
ID: 1913414 · Report as offensive
Previous · 1 . . . 28 · 29 · 30 · 31 · 32 · 33 · 34 . . . 36 · Next

Message boards : Number crunching : Panic Mode On (109) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.