Panic Mode On (116) Server Problems?

Message boards : Number crunching : Panic Mode On (116) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 31 · 32 · 33 · 34 · 35 · 36 · 37 . . . 47 · Next

AuthorMessage
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1998031 - Posted: 13 Jun 2019, 14:33:04 UTC

I've noticed the different ratio of MB to GreenBank (blc) data too . I think it partially has to do with this file, 18dc09aa, being stuck. It is taking around 8 of the 11 channels in progress and nothing is coming out. Those 8 channels aren't doing anything. So there are only 3 channels working on splitting MB data.
ID: 1998031 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1998121 - Posted: 14 Jun 2019, 0:20:28 UTC

No Panic yet, but could be a problem. The stuck Aricebo file needs to be removed.

It is nice to have two sets of data being split as if one goes into a shorty storm, then hopefully the other dataset helps stabilize the system.

It looks like they gave us a small amount of Green Bank data. blc41_2bit_guppi_58543_...
or were those files there before and I just didn't notice?
ID: 1998121 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1998127 - Posted: 14 Jun 2019, 1:02:28 UTC - in response to Message 1998121.  
Last modified: 14 Jun 2019, 1:39:49 UTC

No Panic yet, but could be a problem. The stuck Aricebo file needs to be removed.
It is nice to have two sets of data being split as if one goes into a shorty storm, then hopefully the other dataset helps stabilize the system.
It looks like they gave us a small amount of Green Bank data. blc41_2bit_guppi_58543_...
or were those files there before and I just didn't notice?

. . No, they are definitely new, and they are relatively recent data from 1st March 2019 (at least they are more recent than 15th October 2018). I have never seen a 4 series blc task before so they may have commissioned another bank of recorders at GBT. I expect they will take about the same time as a blc33 task. Personally I will be more excited though when I see a new server called amigos and tasks with a Parkes identifier in their name :)

Stephen

:)
ID: 1998127 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1998132 - Posted: 14 Jun 2019, 1:30:16 UTC

The new Green Bank data is dropping my RAC quite nicely, no reason to panic because IMO it is generally better data because it is targeted rather than the random data from Arecibo telescope.
ID: 1998132 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1998134 - Posted: 14 Jun 2019, 1:35:52 UTC

. . Be careful what you wish for ....

. . I am now getting a big flow of Arecibo tasks, but they are all .VLAR WUs. That will definitely slow things down ...

Stephen

:(
ID: 1998134 · Report as offensive
Profile Kissagogo27 Special Project $75 donor
Avatar

Send message
Joined: 6 Nov 99
Posts: 716
Credit: 8,032,827
RAC: 62
France
Message 1998203 - Posted: 14 Jun 2019, 19:54:51 UTC

https://setiathome.berkeley.edu/results.php?hostid=8572161

. . . . . .


i don't expect anything about this host, but how can the server can let it downloading all theses WU ?
ID: 1998203 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1998205 - Posted: 14 Jun 2019, 20:13:05 UTC - in response to Message 1998203.  

Wow, I wonder if that is a record for most tasks sent incorrectly by the schedulers. Definitely broken code.

But there is something else strange about the host. It has two timed out tasks that apparently only had a 3 week deadline.

Normally our MB tasks have had a 6 to 7 week deadline. That made me look at my tasks and I see now that all my tasks have a much shorter 3 week deadline too.

So when did this change occur? Does everybody else have a much shorter task deadline in their cache also?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1998205 · Report as offensive
JohnDK Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 1222
Credit: 451,243,443
RAC: 1,127
Denmark
Message 1998208 - Posted: 14 Jun 2019, 20:40:05 UTC

Those shortie WUs have always had a 3 week deadline.
ID: 1998208 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1998210 - Posted: 14 Jun 2019, 20:49:38 UTC - in response to Message 1998208.  

But I am seeing all my tasks with a 3 week deadline. Including BLC VLARs which have never been considered shorties. I call "shorties" the Arecibo VHAR tasks with AR greater than the standard 0.40- 0.44 AR range. Some of them lately have been in the .7-1.1 range and they process very fast of course. But the BLC tasks have all been normal 6-7 week deadlines.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1998210 · Report as offensive
JohnDK Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 1222
Credit: 451,243,443
RAC: 1,127
Denmark
Message 1998213 - Posted: 14 Jun 2019, 20:56:21 UTC

Except for the shorties, most of my tasks on all 3 hosts have a deadline of august 6/7.
ID: 1998213 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1998215 - Posted: 14 Jun 2019, 21:05:18 UTC - in response to Message 1998213.  

I am also seeing a lot of various short deadlines, both July 5 and July 25. There are a lot of the normal Aug 6/7 as well.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1998215 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1998219 - Posted: 14 Jun 2019, 21:34:15 UTC - in response to Message 1998213.  

But that is exactly what I am talking about. All my 27,700 tasks issued Jun 13-14 have a deadline date of the first week of August. So just about exactly a 3 week deadline.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1998219 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1998223 - Posted: 14 Jun 2019, 21:48:03 UTC - in response to Message 1998219.  
Last modified: 14 Jun 2019, 21:49:53 UTC

It's not universal. Across my 7 crunchers, I have a block of 4/5 July deadlines (3 weeks, certainly Arecibo, presumed VHAR, but not inspected internally). A few middling Arecibos, but every VLAR - both Arecibo and BLC - has a 6/7 August deadline as normal.

Edit
issued Jun 13-14 have a deadline date of the first week of August.
I think you've blanked July out of your calendar!
ID: 1998223 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1998224 - Posted: 14 Jun 2019, 21:56:08 UTC - in response to Message 1998223.  

Ohh, darn. I guess I did forget about the month of July. Still frazzled in my thinking I guess. Sorry about the interruption . . . . nothing to see here.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1998224 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1998229 - Posted: 14 Jun 2019, 22:20:57 UTC - in response to Message 1998205.  
Last modified: 14 Jun 2019, 22:28:43 UTC

Wow, I wonder if that is a record for most tasks sent incorrectly by the schedulers. Definitely broken code.
But there is something else strange about the host. It has two timed out tasks that apparently only had a 3 week deadline.
Normally our MB tasks have had a 6 to 7 week deadline. That made me look at my tasks and I see now that all my tasks have a much shorter 3 week deadline too.
So when did this change occur? Does everybody else have a much shorter task deadline in their cache also?


. . I don't normally check deadlines, I have that column disabled in my manager's task screen to make room. But I just did a random survey of more than half a dozen tasks and they are all normal deadlines (7th August).

{edit} . . OK, making a mental note to send Keith a new calendar ...

Stephen

??
ID: 1998229 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1998272 - Posted: 15 Jun 2019, 7:28:32 UTC - in response to Message 1998032.  
Last modified: 15 Jun 2019, 8:04:37 UTC

I've noticed the different ratio of MB to GreenBank (blc) data too . I think it partially has to do with this file, 18dc09aa, being stuck. It is taking around 8 of the 11 channels in progress and nothing is coming out. Those 8 channels aren't doing anything. So there are only 3 channels working on splitting MB data.

Yes, that particular file/tape got stuck when it was being split 9 years ago too.
It's a bad one, evil even :-) They had to remove it back then, and I guess the same will happen now.
Let's just hope it doesn't hang around for as long this time.

See: 18DC09AA Has Been Stuck More Than A Month

No need for panic more a sense of relief the multibeam tape mentioned above has been removed from the queue. I am unsure if that had human intervention, if that did thank you to whoever moved it on. Apologies people I got it wrong

I am also looking forward to the day when we see the 180.04 gig tape start to get it processed I noticed the name ended in 13. I'm not superstitious by any means I wonder is this a sign
ID: 1998272 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1998273 - Posted: 15 Jun 2019, 7:54:12 UTC

No need for panic more a sense of relief the multibeam tape mentioned above has been removed from the queue.

? ? ? ? I still see it loaded. It's been stuck for days.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1998273 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1998275 - Posted: 15 Jun 2019, 8:05:23 UTC - in response to Message 1998273.  

No need for panic more a sense of relief the multibeam tape mentioned above has been removed from the queue.

? ? ? ? I still see it loaded. It's been stuck for days.

Thanks Keith. I have adjusted my original post
ID: 1998275 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1998526 - Posted: 17 Jun 2019, 8:58:28 UTC

18dc09aa is still hanging in there, still no work from it.
Grant
Darwin NT
ID: 1998526 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1998530 - Posted: 17 Jun 2019, 12:22:24 UTC
Last modified: 17 Jun 2019, 12:23:32 UTC

@ Unixchick

. . Hey guess what? We are now crunching the last couple of 'groundhog day' tapes, and the other older GBT tapes in blc34 and blc22 are following. By morning here I expect to be crunching the newer blc41 tapes. After 8 months we are about to see the end of them ... YYYAAAAAAAYYYY!

. . All I need now is to see Parkes tapes begin to appear ...

Stephen

:)
ID: 1998530 · Report as offensive
Previous · 1 . . . 31 · 32 · 33 · 34 · 35 · 36 · 37 . . . 47 · Next

Message boards : Number crunching : Panic Mode On (116) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.