Panic Mode On (113) Server Problems?

Message boards : Number crunching : Panic Mode On (113) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 37 · Next

AuthorMessage
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1952600 - Posted: 28 Aug 2018, 22:39:40 UTC - in response to Message 1952599.  

And we are back...

But not getting any work. Darn it.


Getting low on work....

ID: 1952600 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1952603 - Posted: 28 Aug 2018, 22:53:09 UTC

Grrrrr, I was able to contact the scheduler to report work right when they came back. But nothing but scheduler server errors since and every contact has received the no tasks are available message.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1952603 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1952613 - Posted: 28 Aug 2018, 23:56:22 UTC

we seem to be chugging along now. some of my systems have a full queue, and all of them have at least several hours worth of work
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1952613 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1952615 - Posted: 28 Aug 2018, 23:57:54 UTC - in response to Message 1952613.  

I can't get some of my systems to contact the servers to report. I've tried all the tricks to no avail.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1952615 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1952616 - Posted: 28 Aug 2018, 23:59:27 UTC

All 4 caches full again here.

Cheers.
ID: 1952616 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1952619 - Posted: 29 Aug 2018, 0:16:35 UTC - in response to Message 1952603.  

Grrrrr, I was able to contact the scheduler to report work right when they came back. But nothing but scheduler server errors since and every contact has received the no tasks are available message.


. . I've been assigned some work but it will not download ... some kind of server problem ??

Stephen

? ?
ID: 1952619 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1952620 - Posted: 29 Aug 2018, 0:21:07 UTC - in response to Message 1952619.  

Grrrrr, I was able to contact the scheduler to report work right when they came back. But nothing but scheduler server errors since and every contact has received the no tasks are available message.


. . I've been assigned some work but it will not download ... some kind of server problem ??

Stephen

? ?

Yes, the servers are too busy and overloaded with download requests. Stuck in the same situation with download backoffs of five hours.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1952620 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1952646 - Posted: 29 Aug 2018, 2:01:36 UTC

Full caches on both machines, it took a while, but eventually got there.

ID: 1952646 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1953096 - Posted: 31 Aug 2018, 21:07:23 UTC

. . Hmmm, these new Blc11 tapes won't take long to disappear if the current rate of noise bombs persists. Unless I am a very unlucky volunteer who just happens to get all the noise bombs on all 4 machines they are currently about 50% of all WUs from these tapes.

. . Still it should get us through the weekend.

Stephen

? ?
ID: 1953096 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1953097 - Posted: 31 Aug 2018, 21:11:11 UTC - in response to Message 1953096.  

The return rate has been climbing steadily. It is now over 142k from ~90k before the BLC11 tapes.
ID: 1953097 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1953105 - Posted: 31 Aug 2018, 22:34:35 UTC - in response to Message 1953097.  

The return rate has been climbing steadily. It is now over 142k from ~90k before the BLC11 tapes.

Just noticed that myself- talk about lucky timing.
Even with the lower return rate of the longer running WUs, the Wow! event added 12-15k extra returned per hour. Not too sure how well the system would have coped with an extended period of 155-160k per hour.

It will be interesting to see if this return rate is sustained for a few days, and if the servers can handle it OK- after all the issues the servers were having with such high return rates previously, we haven't had any since the database changes were made with the hope of alleviating them. This may give us a chance to see if those changes were successful (other than significantly reducing the time needed for the weekly outages).
Grant
Darwin NT
ID: 1953105 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1953117 - Posted: 31 Aug 2018, 23:18:37 UTC - in response to Message 1953105.  

Boy, you aren't kidding about the BLC11 noise bombs. It will be a good test of the servers to handle a high return rate.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1953117 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1953120 - Posted: 31 Aug 2018, 23:28:03 UTC

And now the deleters are showing the first signs of a struggle ....
ID: 1953120 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1953139 - Posted: 1 Sep 2018, 1:31:51 UTC

FYI-Monday is a holiday. so hopefully there is enough data (probably) and the system can handle the noise
ID: 1953139 · Report as offensive
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 1953153 - Posted: 1 Sep 2018, 2:51:11 UTC - in response to Message 1953117.  

Boy, you aren't kidding about the BLC11 noise bombs. It will be a good test of the servers to handle a high return rate.


yeah i thought something was wrong when i saw all the jobs flying by in 5-10 seconds. CPU jobs even.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 1953153 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1953159 - Posted: 1 Sep 2018, 3:24:57 UTC - in response to Message 1953120.  

And now the deleters are showing the first signs of a struggle ....

They seem to be coping okayed at the moment they are down to 2.727M

I wonder if anybody will reach the max number of tasks allowed per day because of all of these noise bombs
ID: 1953159 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1953168 - Posted: 1 Sep 2018, 3:48:35 UTC - in response to Message 1953159.  

I wonder if anybody will reach the max number of tasks allowed per day because of all of these noise bombs

My maximum tasks per day are around 22,000 & 46,000. And as each one validates, your maximum number of tasks increases.
So I don't think it's likely to be a problem.
Grant
Darwin NT
ID: 1953168 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1953175 - Posted: 1 Sep 2018, 4:52:18 UTC - in response to Message 1953168.  

I wonder if anybody will reach the max number of tasks allowed per day because of all of these noise bombs

My maximum tasks per day are around 22,000 & 46,000. And as each one validates, your maximum number of tasks increases.
So I don't think it's likely to be a problem.

Fair point Grant. Enjoy crunching through them quickly. Not to me that people that crunch more than one at a time will burn through them even quicker
ID: 1953175 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1953183 - Posted: 1 Sep 2018, 6:33:53 UTC - in response to Message 1953159.  

And now the deleters are showing the first signs of a struggle ....

They seem to be coping okayed at the moment they are down to 2.727M

I wonder if anybody will reach the max number of tasks allowed per day because of all of these noise bombs


. . As I understand it, the limit is not how many tasks you can do in a day, but how many tasks can be allocated to you at any one time.

. . There are quite a few hosts doing multi thousands of tasks per day.

Stephen

.
.
ID: 1953183 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1953191 - Posted: 1 Sep 2018, 9:46:42 UTC

Wow, it's up to 166k/hr return rate now, but the system seems to be handling it fine.
Other than the rapid shortening of the files left to split.
ID: 1953191 · Report as offensive
Previous · 1 . . . 4 · 5 · 6 · 7 · 8 · 9 · 10 . . . 37 · Next

Message boards : Number crunching : Panic Mode On (113) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.