Panic Mode On (107) Server Problems?

Message boards : Number crunching : Panic Mode On (107) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 29 · Next

AuthorMessage
Profile Kissagogo27 Special Project $75 donor
Avatar

Send message
Joined: 6 Nov 99
Posts: 711
Credit: 8,032,827
RAC: 62
France
Message 1890887 - Posted: 20 Sep 2017, 8:45:22 UTC

noisy and 3330 app don't want to crunch them ... ( no app cmd line )


https://setiathome.berkeley.edu/results.php?userid=92221&offset=0&show_names=0&state=6&appid=
ID: 1890887 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5516
Credit: 528,817,460
RAC: 242
United States
Message 1890907 - Posted: 20 Sep 2017, 11:34:22 UTC - in response to Message 1890875.  

Can't say I'm seeing all that many... A few here and there but that is about it.
ID: 1890907 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891548 - Posted: 23 Sep 2017, 16:36:34 UTC

Woke up this morning to no gpu tasks on the linux machine. Don't know what's up with that. Is there nothing but Arecibo VLARs coming from the splitters for the last 4 hours? The other machines have work but they don't clear their caches as fast. Going through the wake up the servers procedure right now.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891548 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891550 - Posted: 23 Sep 2017, 16:54:30 UTC

The wakeup started retrieving work again but in small batches. Notice that the website is very slow this morning to refresh any new page or access the database.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891550 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1891583 - Posted: 23 Sep 2017, 22:25:37 UTC - in response to Message 1891550.  

The wakeup started retrieving work again but in small batches. Notice that the website is very slow this morning to refresh any new page or access the database.


. . The pages are slow and jerky to load/refresh. One of my 2 Linux machines is getting work fairly consistently. The other keeps going AWOL as far as the servers are concerned and I have to give them the "kick in the pants" several times each day :( It is the one with the 970s, so maybe the servers lose interest in Linux machines running 970s :P

Stephen

:)
ID: 1891583 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891588 - Posted: 23 Sep 2017, 23:51:30 UTC - in response to Message 1891583.  

The wakeup started retrieving work again but in small batches. Notice that the website is very slow this morning to refresh any new page or access the database.


. . The pages are slow and jerky to load/refresh. One of my 2 Linux machines is getting work fairly consistently. The other keeps going AWOL as far as the servers are concerned and I have to give them the "kick in the pants" several times each day :( It is the one with the 970s, so maybe the servers lose interest in Linux machines running 970s :P

Stephen

:)

Seems to be that way .... doesn't it?? :-}
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891588 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891619 - Posted: 24 Sep 2017, 6:27:14 UTC

I can't believe there are still tons of noise bombs being distributed. I would have thought they'd would have cleared out by now.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891619 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1635
Credit: 12,921,799
RAC: 89
New Zealand
Message 1891624 - Posted: 24 Sep 2017, 8:00:32 UTC - in response to Message 1891619.  

I can't believe there are still tons of noise bombs being distributed. I would have thought they'd would have cleared out by now.

I have recently picked up about 90 tasks however none of them were GBT tasks.
ID: 1891624 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891626 - Posted: 24 Sep 2017, 8:21:38 UTC - in response to Message 1891624.  

I can't believe there are still tons of noise bombs being distributed. I would have thought they'd would have cleared out by now.

I have recently picked up about 90 tasks however none of them were GBT tasks.

Yeah, the splitters seem to have been spitting out nothing but Arecibo shorties for the last couple of hours mostly. At least that's what I have seen on my four crunchers. We seem to be getting more of that lately, either a majority of Arecibo or GBT tasks. Not many cases with an evenly balanced mix of tasks from both telescopes. Too bad they don't have some sort of randomizer function in the splitters. I do believe they work on a FIFO order.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891626 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1891640 - Posted: 24 Sep 2017, 15:29:28 UTC - in response to Message 1891619.  

I can't believe there are still tons of noise bombs being distributed. I would have thought they'd would have cleared out by now.


. . Yep, still seeing lots of them ...

Stephen

??
ID: 1891640 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1891643 - Posted: 24 Sep 2017, 15:38:51 UTC - in response to Message 1891624.  

I can't believe there are still tons of noise bombs being distributed. I would have thought they'd would have cleared out by now.

I have recently picked up about 90 tasks however none of them were GBT tasks.


. . Well, after a prolonged period of "No tasks available" I just gave the servers the "kick" and received 106 tasks, 98 were GBT tasks. Aint I the lucky one :)

Stephen

:)
ID: 1891643 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891647 - Posted: 24 Sep 2017, 16:09:21 UTC - in response to Message 1891643.  

Thought I was going to have to resort to that but some heavy rescheduling produced enough deficit that the servers finally filled me up. Had been lots of no work is available replies.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891647 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1891658 - Posted: 24 Sep 2017, 16:49:18 UTC - in response to Message 1891647.  

Thought I was going to have to resort to that but some heavy rescheduling produced enough deficit that the servers finally filled me up. Had been lots of no work is available replies.


. . Well the i5 with the 970s now has a full GPU Q with 200 WUs, 130 of them GBT tasks. :( Oh well.

Stephen

<shrug>
ID: 1891658 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1891850 - Posted: 26 Sep 2017, 1:32:07 UTC

Hmmm...kind of an interesting change to the BLC files being split. All the blc05 files that were still in the queue have been pulled and a dozen more Tabby's Star files have been loaded. They're blc02 this time, instead of blc04, but same date and time frame.
ID: 1891850 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891851 - Posted: 26 Sep 2017, 1:40:09 UTC - in response to Message 1891850.  

I noticed the BLC04 files that had the splitters pulled halfway and moved onto other tapes too. I don't know how or whether that has impacted my inability to keep the crunchers caches topped up. I've been doing the ghost recovery protocol on my two heavy hitters all day today to try and keep them fed. Lots of no tasks available messages. Always 80-100 tasks down all day from full.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891851 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891852 - Posted: 26 Sep 2017, 1:53:46 UTC

Doesn't help when you get HTTP: Internal server error messages either. Makes kinda hard to get any work.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891852 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1891854 - Posted: 26 Sep 2017, 1:58:12 UTC - in response to Message 1891851.  

Wish I could offer you some advice on that problem, as my 3 big boxes stay pretty well topped up. This afternoon, I stockpiled as usual on a Monday, in advance of the Tuesday outage, and had no trouble getting the GPU queues reloaded to their max within one or two scheduler requests. Interestingly, though, about 99% of the new tasks were Arecibo. Perhaps that was when they were pulling the blc05 files and loading up the new blc02 ones.
ID: 1891854 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891859 - Posted: 26 Sep 2017, 2:26:00 UTC - in response to Message 1891854.  

Yes, the great mystery of the century ....... why do I constantly have troubles getting work on request when there are ~600,000 tasks in the buffer. I don't usually try stocking up till later in the evening since if I started earlier I would crunch through the majority of my overload in the early hours of the night before the outage starts while I'm sleeping and not shepherding the systems. If I could depend on getting work normally or on time, then I could build my overstock earlier in the day.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891859 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1891861 - Posted: 26 Sep 2017, 2:33:04 UTC - in response to Message 1891854.  

Wish I could offer you some advice on that problem, as my 3 big boxes stay pretty well topped up. This afternoon, I stockpiled as usual on a Monday, in advance of the Tuesday outage, and had no trouble getting the GPU queues reloaded to their max within one or two scheduler requests. Interestingly, though, about 99% of the new tasks were Arecibo. Perhaps that was when they were pulling the blc05 files and loading up the new blc02 ones.


. . FWIW here I am getting swamped with GBT work.

Stephen

<shrug>
ID: 1891861 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13157
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1891863 - Posted: 26 Sep 2017, 2:48:45 UTC - in response to Message 1891861.  

And I'm getting just the opposite when I use the kick the servers protocol. Mostly Arecibo non-VLARs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1891863 · Report as offensive
Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 29 · Next

Message boards : Number crunching : Panic Mode On (107) Server Problems?


 
©2022 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.