Panic Mode On (106) Server Problems?

Message boards : Number crunching : Panic Mode On (106) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 29 · Next

AuthorMessage
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1867690 - Posted: 17 May 2017, 12:02:09 UTC - in response to Message 1867679.  
Last modified: 17 May 2017, 12:03:08 UTC

2018 Arecibo tapes seems never coming to an end..


. . Aaaah! A cruncher from the future :)

. . Although I know it is very important to process all the more current data from GBT I do like the way the 2008 Arecibo tasks flow through my GPUs :)

. . But as has been explained. Those old 'tapes' were originally processed with the older apps and they did not have the resolution of the more recent apps, so we are crunching them for a second time to get more detail. That is the nature of a search for something which is akin to the proverbial needle in a haystack. In fact I would be hard pressed to think of a much more appropriate application of that saying.

. . And for what it is worth I am getting more GBT work than Arecibo work of late.

Stephen

:)
ID: 1867690 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1867738 - Posted: 17 May 2017, 16:17:51 UTC

Except for Numbskull who continues to get an exclusive diet of Arecibo VLARs for some reason. I am still perplexed by the difference in task mix between the Ryzen system and the FX systems.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1867738 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1867747 - Posted: 17 May 2017, 17:08:42 UTC - in response to Message 1867738.  

Except for Numbskull who continues to get an exclusive diet of Arecibo VLARs for some reason. I am still perplexed by the difference in task mix between the Ryzen system and the FX systems.


. . Server behaviour is inscrutable. 3 of my rigs fill the caches whenever they upload/report completed work. But Mi_Burrito only gets 27 or 28 new tasks regardless of how many it has done. Strange and a bit of a drag. No reason I can think of why this is so ...

Stephen

??
ID: 1867747 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1867759 - Posted: 17 May 2017, 18:24:08 UTC - in response to Message 1867747.  


. . Server behaviour is inscrutable.
??

Yes, inscrutable in spades. The only way I can get BLC tasks onto the Ryzen CPU is with the rescheduler. The kick in the n*ts is that the CPU processes the BLC tasks the fastest at greatest efficiency. Wish the server could figure that out. {sigh}
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1867759 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1867800 - Posted: 17 May 2017, 22:34:42 UTC - in response to Message 1867759.  


. . Server behaviour is inscrutable.
??

Yes, inscrutable in spades. The only way I can get BLC tasks onto the Ryzen CPU is with the rescheduler. The kick in the n*ts is that the CPU processes the BLC tasks the fastest at greatest efficiency. Wish the server could figure that out. {sigh}


. . Exactly, if the configuration option was available I would select only BLC and Arecibo VLAR tasks for the CPUs and all the other Arecibo work for the GPUs. The CPUs love the GBT stuff.

Stephen

:)
ID: 1867800 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1867841 - Posted: 18 May 2017, 2:06:45 UTC - in response to Message 1867679.  

2018 Arecibo tapes seems never coming to an end..

2018 tapes? Are you seeing tapes from the future? (May 17, 2017) ;-)
.

Hello, from Albany, CA!...
ID: 1867841 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1867875 - Posted: 18 May 2017, 5:24:17 UTC

Web site is here, then it's gone. Forums work, then they don't. Then they do, but it takes forever. Things here are a bit erratic at the moment.
Grant
Darwin NT
ID: 1867875 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1867881 - Posted: 18 May 2017, 6:13:31 UTC - in response to Message 1867875.  

Web site is here, then it's gone. Forums work, then they don't. Then they do, but it takes forever. Things here are a bit erratic at the moment.


. . Yep, definitely, and this rig can only get 10 jobs at a time .... :(

. . Oh well ...

Stephen

<shrug>
ID: 1867881 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1868042 - Posted: 18 May 2017, 23:51:33 UTC

. . @ All,

. . I am surprised but I am getting absolutely no work on 4 rigs, all saying "Project has no work available" and not a single comment in here. It makes me wonder if I am the only one experiencing this. :(

Stephen

. . Within the hour the Linux boxes will be shut down because of empty caches and the windows boxes have a few more hours of work cached from 8 hours ago.
ID: 1868042 · Report as offensive
Profile scocam
Avatar

Send message
Joined: 28 Feb 17
Posts: 27
Credit: 15,120,999
RAC: 0
United States
Message 1868046 - Posted: 19 May 2017, 0:06:19 UTC - in response to Message 1868042.  
Last modified: 19 May 2017, 0:12:20 UTC

. . I am surprised but I am getting absolutely no work on 4 rigs, all saying "Project has no work available" and not a single comment in here. It makes me wonder if I am the only one experiencing this. :(


Stephen, I just saw your message and I'm down to 114 104 96 tasks on my main machine. Hmmm. I'll be out of tasks in no time at all. Server status says several of the splitters are down and the Result creation rate is way low... but I'm not that knowledgeable regarding the servers and status so I'll defer to the experts.

scocam
ID: 1868046 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1868048 - Posted: 19 May 2017, 0:10:32 UTC

The downloads have been struggling for a couple hours now, I picked up a few along the way but I'm also low.
ID: 1868048 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1868052 - Posted: 19 May 2017, 0:24:39 UTC - in response to Message 1868046.  

. . I am surprised but I am getting absolutely no work on 4 rigs, all saying "Project has no work available" and not a single comment in here. It makes me wonder if I am the only one experiencing this. :(


Stephen, I just saw your message and I'm down to 114 104 96 tasks on my main machine. Hmmm. I'll be out of tasks in no time at all. Server status says several of the splitters are down and the Result creation rate is way low... but I'm not that knowledgeable regarding the servers and status so I'll defer to the experts.

scocam


. . Hi Scocam,

. . Like you I know little about the server structure so I am not too sure what the stats mean but clearly then there is an issue. I am just surprised that I was the first to mention it. Normally by the time I look in here there are a string of messages about it :)

. . I have Einstein at Home set up as a backup project for occasions such as this so it looks like they may be getting some more crunching done today :(

Stephen

<shrug>
ID: 1868052 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1868058 - Posted: 19 May 2017, 0:43:28 UTC - in response to Message 1868042.  

No, you are not alone in this regard. I meant to post to the thread about the lack of work available but got side-tracked. My Windows7 computers have been getting nothing but "no work is available" for most of the afternoon. They might be making a recovery starting now. Haven't had the issue on the Ryzen Windows 10 system because it continues to get almost exclusively nothing but Arecibo shorties for the CPU. I can then reschedule them to the GPU's to keep them busy.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1868058 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1868065 - Posted: 19 May 2017, 2:20:00 UTC - in response to Message 1868058.  

No, you are not alone in this regard. I meant to post to the thread about the lack of work available but got side-tracked. My Windows7 computers have been getting nothing but "no work is available" for most of the afternoon. They might be making a recovery starting now. Haven't had the issue on the Ryzen Windows 10 system because it continues to get almost exclusively nothing but Arecibo shorties for the CPU. I can then reschedule them to the GPU's to keep them busy.



. . Hi Keith,

. . That makes me wish I still had CPU crunching on these two Linux machines so that I would have some work to cannibalise for the GPUs. But the schedulers seem to be in "rationing" mode. They are dolling out just enough work to keep going. Both Linux machines are teetering along with a queue of 4 or 5 tasks. Oh well, at least they are still crunching.

Stephen

:)
ID: 1868065 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1868068 - Posted: 19 May 2017, 3:02:37 UTC

Yeah, I took a look at your Linux machines. Pretty poor, one with no tasks and the other with 9. Big whup. I am only down from 50-80 tasks across all three machines with my 300 task allotment. The rescheduler is definitely helping keeping the GPUs fed. I wonder what the issue is with the scheduler. They recovered to about 70 tasks/sec around 7 PM and are now running about where they normally are at just under 30 tasks/sec. I can understand if they are just feeding out nothing but VLARs and they won't go to Nvidia cards. But I am down to about 25 tasks on the Ryzen CPU and it should be sent everything and ANYTHING that is available. It is coming up with more 'tasks aren't available' messages than the 3 or 4 tasks it gets every half hour or so. That effects me the most on the Ryzen since it crunches through CPU tasks as fast as a mid-range GPU.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1868068 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1868073 - Posted: 19 May 2017, 3:55:13 UTC
Last modified: 19 May 2017, 4:24:13 UTC

Looking at the work in progress numbers, there are lot of people other than just us unable to get work- there were 4.62million WUs in progress, but it's been steadily dropping for several hours now. Down to 4.3 and still falling.
Scheduler/feeder needs a kick maybe?


EDIT- and add to that the forums are slow, slow, slow, slow and getting slower...


EDIT- mucking around with Application preferences doesn't help either. No downloads to my i7 system for over 3 hours now.
Grant
Darwin NT
ID: 1868073 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1868080 - Posted: 19 May 2017, 4:33:20 UTC - in response to Message 1868068.  

Yeah, I took a look at your Linux machines. Pretty poor, one with no tasks and the other with 9.


. . Yeah, after my last post La-Bamba stopped getting any work at all. Every request is "No tasks available".

. . But E@H is doing well out of it :)

Stephen

:(
ID: 1868080 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1868082 - Posted: 19 May 2017, 4:50:07 UTC

On the upside, the RTS cache is 100k overfull, meaning they are all resends just waiting for me to instantly validate them when I get them :D
ID: 1868082 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 1868091 - Posted: 19 May 2017, 5:49:58 UTC
Last modified: 19 May 2017, 5:58:10 UTC

At least somebody should be able to kick the tires tomorrow morning Berkely time, in the roughly 9 hours time

Edit: things could be on the improve I noticed the ready to send his dropped to about 680 K it was around 712,000 before
ID: 1868091 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 1868096 - Posted: 19 May 2017, 6:03:49 UTC - in response to Message 1868091.  
Last modified: 19 May 2017, 6:05:29 UTC

Edit: things could be on the improve I noticed the ready to send his dropped to about 680 K it was around 712,000 before

Some people picked up some work, unfortunately my i7 isn't one of those lucky systems.

EDIT- the rate of decline for In progress work did slow down for a short while, but it appears to be dropping at it's earlier rate again.
Just a temporary pause on it's way down.
Grant
Darwin NT
ID: 1868096 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 29 · Next

Message boards : Number crunching : Panic Mode On (106) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.