Panic Mode On (116) Server Problems?

Message boards : Number crunching : Panic Mode On (116) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 18 · 19 · 20 · 21 · 22 · 23 · 24 . . . 47 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995038 - Posted: 24 May 2019, 17:46:10 UTC - in response to Message 1995034.  

There is a ton of "noise bombs" going through currently. Likely the reason for the higher return rate.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995038 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19072
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1995053 - Posted: 24 May 2019, 20:32:47 UTC - in response to Message 1995038.  

There is a ton of "noise bombs" going through currently. Likely the reason for the higher return rate.

I looked at my returned tasks just after you posted, and at that time about 25% were noisy.
Just checked again and now its about 50%.
Server stats are also up.
Results received in last hour ** 146,414 0m
ID: 1995053 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1995054 - Posted: 24 May 2019, 20:41:08 UTC

I've been checking the stats periodically in between chores, and it hit a peak around 150k/ hour and appears to be slowly falling now.
ID: 1995054 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13744
Credit: 208,696,464
RAC: 304
Australia
Message 1995082 - Posted: 24 May 2019, 23:01:38 UTC - in response to Message 1995038.  

There is a ton of "noise bombs" going through currently. Likely the reason for the higher return rate.

Yep.
Noticed a lot in my results, and still quite a few in my cache.

And of course the random Credits being allocated- Anything from 0.3 to 5 depending on wingmate.
Grant
Darwin NT
ID: 1995082 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1995097 - Posted: 25 May 2019, 0:19:14 UTC

The results returned per hour number is on the rise again, guessing that means more noise bombs. Still in the 140k -150k range, so no panic.
ID: 1995097 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1995137 - Posted: 25 May 2019, 6:48:51 UTC
Last modified: 25 May 2019, 6:49:39 UTC

Thanks to "noise bombs" we have completed over 1000 channels today. No need to panic
ID: 1995137 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995138 - Posted: 25 May 2019, 7:09:52 UTC - in response to Message 1995097.  
Last modified: 25 May 2019, 8:09:14 UTC

The results returned per hour number is on the rise again, guessing that means more noise bombs. Still in the 140k -150k range, so no panic.

Up to 184K185K187K188K now.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995138 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19072
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1995139 - Posted: 25 May 2019, 7:17:18 UTC
Last modified: 25 May 2019, 7:26:09 UTC

Only a few noisy bombs, average time to complete tasks is 310s (5m:10s)

25/05/2019 08:00:07 | SETI@home | Sending scheduler request: To fetch work.
25/05/2019 08:00:07 | SETI@home | Reporting 1 completed tasks
25/05/2019 08:00:07 | SETI@home | Requesting new tasks for NVIDIA GPU
25/05/2019 08:00:09 | SETI@home | Scheduler request completed: got 1 new tasks

5m:10s later

25/05/2019 08:05:17 | SETI@home | Sending scheduler request: To fetch work.
25/05/2019 08:05:17 | SETI@home | Reporting 7 completed tasks
25/05/2019 08:05:17 | SETI@home | Requesting new tasks for NVIDIA GPU
25/05/2019 08:05:19 | SETI@home | Scheduler request completed: got 6 new tasks

5m:10s later

25/05/2019 08:10:27 | SETI@home | Sending scheduler request: To fetch work.
25/05/2019 08:10:27 | SETI@home | Reporting 7 completed tasks
25/05/2019 08:10:27 | SETI@home | Requesting new tasks for NVIDIA GPU
25/05/2019 08:10:29 | SETI@home | Scheduler request completed: got 7 new tasks

Up a bit more
Results received in last hour ** 185,875
ID: 1995139 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13744
Credit: 208,696,464
RAC: 304
Australia
Message 1995140 - Posted: 25 May 2019, 7:21:06 UTC - in response to Message 1995138.  
Last modified: 25 May 2019, 7:21:19 UTC

The results returned per hour number is on the rise again, guessing that means more noise bombs. Still in the 140k -150k range, so no panic.

Up to 184K now.

And the WU awaiting Validation & Assimilation continue to climb, Ready-to-send continues to fall, and the splitter output continues to decline.
Looks like we've found the servers limits again.
Grant
Darwin NT
ID: 1995140 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995141 - Posted: 25 May 2019, 7:21:25 UTC

Ha ha. I have to scroll through 15 pages to find a page that isn't less than 50% noise bombs. Don't know whether they have had a receiver go bad at Green Bank or they just recorded when there was an extraneous noise source in the antenna's aperture.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995141 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13744
Credit: 208,696,464
RAC: 304
Australia
Message 1995142 - Posted: 25 May 2019, 7:26:04 UTC

Just watched one of my GPUs go through 10 noise bombs in a row.
Grant
Darwin NT
ID: 1995142 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19072
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1995143 - Posted: 25 May 2019, 7:28:13 UTC - in response to Message 1995141.  

Ha ha. I have to scroll through 15 pages to find a page that isn't less than 50% noise bombs. Don't know whether they have had a receiver go bad at Green Bank or they just recorded when there was an extraneous noise source in the antenna's aperture.

My 1st page of "Valid" has 17 noise bombs.
ID: 1995143 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995145 - Posted: 25 May 2019, 7:35:20 UTC

I think we may have found the "fail-over" point in the servers. All my hosts are not able to download tasks but go straight to backoffs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995145 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19072
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1995146 - Posted: 25 May 2019, 7:43:23 UTC - in response to Message 1995145.  
Last modified: 25 May 2019, 7:43:56 UTC

I think we may have found the "fail-over" point in the servers. All my hosts are not able to download tasks but go straight to backoffs.

not seen here. times BST
25/05/2019 08:42:15 | SETI@home | Sending scheduler request: To fetch work.
25/05/2019 08:42:15 | SETI@home | Reporting 1 completed tasks
25/05/2019 08:42:15 | SETI@home | Requesting new tasks for NVIDIA GPU
25/05/2019 08:42:17 | SETI@home | Scheduler request completed: got 1 new tasks
25/05/2019 08:42:19 | SETI@home | Started download of blc25_2bit_guppi_58340_62915_HIP43542_0100.25780.0.20.29.226.vlar
25/05/2019 08:42:22 | SETI@home | Finished download of blc25_2bit_guppi_58340_62915_HIP43542_0100.25780.0.20.29.226.vlar
ID: 1995146 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13744
Credit: 208,696,464
RAC: 304
Australia
Message 1995147 - Posted: 25 May 2019, 7:48:10 UTC - in response to Message 1995145.  

I think we may have found the "fail-over" point in the servers. All my hosts are not able to download tasks but go straight to backoffs.

No such issues here, yet.
Grant
Darwin NT
ID: 1995147 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995148 - Posted: 25 May 2019, 7:49:05 UTC

A server request for one task is a lot more likely to go through vice the several hundred I ask for at each connection.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995148 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13744
Credit: 208,696,464
RAC: 304
Australia
Message 1995150 - Posted: 25 May 2019, 7:59:41 UTC - in response to Message 1995148.  

A server request for one task is a lot more likely to go through vice the several hundred I ask for at each connection.

I've had my work reporting limited to 60 for ages now due to that issue.
Grant
Darwin NT
ID: 1995150 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1995151 - Posted: 25 May 2019, 8:04:53 UTC - in response to Message 1995150.  

A server request for one task is a lot more likely to go through vice the several hundred I ask for at each connection.

I've had my work reporting limited to 60 for ages now due to that issue.

But even with reporting limited to 60, I can have 5 hosts times 60 = 300 tasks being requested. And that is if I wasn't down several hundred tasks per host already because of the download backoffs, so each host is going to ask for whatever the RTS buffer has in it each time to try and rebuild my cache.

I have my hosts reporting 100 tasks which works fine when the servers aren't under extreme duress like they are now.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1995151 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13744
Credit: 208,696,464
RAC: 304
Australia
Message 1995152 - Posted: 25 May 2019, 8:11:24 UTC - in response to Message 1995151.  
Last modified: 25 May 2019, 8:23:51 UTC

But even with reporting limited to 60, I can have 5 hosts times 60 = 300 tasks being requested.

?
The WU reported limit is a per host limit, the number of hosts reporting from a particular connection wouldn't make any difference (unless you're talking about a significant number of the total number of active hosts for the entire project).



Results received in last hour is almost 190k now.
Splitter output has picked up a bit, but still behind the present load, so Ready-to-send continues to plummet (just a bit slower than is was).
Edit-
And a few Validators & Assimilators are disabled, which wouldn't be helping clear the surge in returned work.
Grant
Darwin NT
ID: 1995152 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1995153 - Posted: 25 May 2019, 8:14:17 UTC - in response to Message 1995145.  

I think we may have found the "fail-over" point in the servers. All my hosts are not able to download tasks but go straight to backoffs.


. . I have been assigned new work but the downloads are going into backoff ...

Stephen

:(
ID: 1995153 · Report as offensive
Previous · 1 . . . 18 · 19 · 20 · 21 · 22 · 23 · 24 . . . 47 · Next

Message boards : Number crunching : Panic Mode On (116) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.