Panic Mode On (110) Server Problems?

Message boards : Number crunching : Panic Mode On (110) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 37 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1917481 - Posted: 7 Feb 2018, 17:22:34 UTC - in response to Message 1917435.  

There was also a shorty AO tape from yesterday 05Feb18 loaded briefly

The Arecibo data from the observation run Saturday is paying out nicely, like the Arecibo data of old.

Task 2848185979

Run time 1 hours 52 min 23 sec
CPU time 1 hours 50 min 22 sec
Validate state Valid
Credit 104.63


. . I wish I could say the same, most of the limited Arecibo work I am getting is on GPUs and is getting scores more like the GBT work. An occasional task will rise to the rarefied heights of high seventies. Aint technology wonderful .....

Stephen

:(

Yes, I would say those few tasks that were awarded in the 100's are displaying the symptom I have seen before. Any new work dispensed for the first time to the first crunchers gets awarded higher credit. It is like the scheduler and assimilators need to train the credit award mechanism for the first tasks returned and award higher than the normal long-term credit.

My later Arecibo tasks were awarded the same as yours, mid 70-80's.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1917481 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 1917524 - Posted: 7 Feb 2018, 22:24:41 UTC - in response to Message 1917435.  


. . I wish I could say the same, most of the limited Arecibo work I am getting is on GPUs and is getting scores more like the GBT work. An occasional task will rise to the rarefied heights of high seventies. Aint technology wonderful .....

Stephen

:(

When I filled my cache about 14 else to go I only got GBT data good on you if you are getting work from Arecibo
ID: 1917524 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1917543 - Posted: 8 Feb 2018, 0:31:15 UTC - in response to Message 1917524.  
Last modified: 8 Feb 2018, 0:47:04 UTC


. . I wish I could say the same, most of the limited Arecibo work I am getting is on GPUs and is getting scores more like the GBT work. An occasional task will rise to the rarefied heights of high seventies. Aint technology wonderful .....

Stephen

:(

When I filled my cache about 14 else to go I only got GBT data good on you if you are getting work from Arecibo


. . They got in a couple (or 3) new tapes from Arecibo and split them over the weekend, there was relatively little work generated overall but many of us crunchers were lucky enough to score a few of them. They are now long gone but for me they included several (less than 6) VLAR tasks and a handful more of normal AR units. Sorry if you didn't see any of them, it was a bit of a lucky dip really. Now it is back to chewing on older GBT WUs.

. . You have to laugh. Having written that I just checked my Q's over 3 machines and I have 6 old 2007 resends, 2 VLARs and 4 normals, but that is out of 520 tasks. Oh, and 5 APs that have been sitting on the one machine since the weekend because they will not run on it despite my efforts to get them to. But at the most they only ever represented a very small percentage of the work I was getting.

Stephen

:(
ID: 1917543 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1917558 - Posted: 8 Feb 2018, 1:21:16 UTC - in response to Message 1917524.  

I think I was the very first person to pull the newly loaded Arecibo tape from the RTS buffer on Monday. I think I scored somewhere around 140 Arecibo tasks. The first ones paid out like in olden dayz.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1917558 · Report as offensive
Profile JaundicedEye
Avatar

Send message
Joined: 14 Mar 12
Posts: 5375
Credit: 30,870,693
RAC: 1
United States
Message 1917610 - Posted: 8 Feb 2018, 4:33:08 UTC

Well this pack of WUs are paying an average of around 75 credits each despite running steadily 24/7.......credit screw is very discouraging.

"Sour Grapes make a bitter Whine." <(0)>
ID: 1917610 · Report as offensive
Profile Pierre A Renaud
Avatar

Send message
Joined: 3 Apr 99
Posts: 998
Credit: 9,101,544
RAC: 65
Canada
Message 1917647 - Posted: 8 Feb 2018, 11:35:37 UTC - in response to Message 1917304.  
Last modified: 8 Feb 2018, 11:41:45 UTC

. . and I was interested to see that they are Voyager1 related ...
Regarding those intriguing gbt bl VOYAGER1 files, this page explains to anyone interested why and how previous similar telemetric data was dealt with.

Breakthrough Listen: Voyager 1 Observations

Voyager 1 is the most distant man-made object from Earth. Launched by NASA in 1977, it has travelled at fantastic speed (roughly 17,000 m/s), past the outer boundaries of our Solar System and into interstellar space (>12.5 billion miles from the Sun).

Remarkably, 38 years on, Voyager 1 is still sending telemetry data from the depths of interstellar space. This makes it a great systems test for the Breakthrough Listen signal processing pipeline.

In this tutorial, we load, read, and plot some Breakthrough Listen (BL) observations of Voyager 1

(...)

If you've gotten this far, take a moment to pause and reflect. This faint signal has travelled billions of miles through space, back to us here on Earth. Launching this probe out of the Solar System was an immense feat, and the fact that we can detect it still, almost 40 years after its launch, is even more impressive.

The signals from probes such as Voyager are very different from the signals we measure from astrophysical objects. This is one of the approaches Breakthrough Listen is using in its approach toward detecting extraterrestrial signals.
End of the parenthesis, and now returning to the regular programming =:)
Apr 3, 1999 - May 3, 2020
ID: 1917647 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1917652 - Posted: 8 Feb 2018, 12:57:18 UTC - in response to Message 1917610.  

Well this pack of WUs are paying an average of around 75 credits each despite running steadily 24/7.......credit screw is very discouraging.


. . I am tending towards the theory that CreditScrew is an Anarchist plot to erode the number of volunteers crunching for SETI.

. . It's a good thing many/most of us are here for the comeraderie :)

. . OK, so mostly we are here because we hope to find some little green men ... :)

Stephen

:)
ID: 1917652 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13855
Credit: 208,696,464
RAC: 304
Australia
Message 1917818 - Posted: 9 Feb 2018, 4:52:24 UTC - in response to Message 1917652.  

. . I am tending towards the theory that CreditScrew is an Anarchist plot to erode the number of volunteers crunching for SETI.

It would certainly help reduce the server load if the number of active users dropped by 50%
Grant
Darwin NT
ID: 1917818 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1917822 - Posted: 9 Feb 2018, 5:18:49 UTC - in response to Message 1917818.  

If the "active" users were heavy-duty crunchers, maybe no difference in project throughput.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1917822 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13855
Credit: 208,696,464
RAC: 304
Australia
Message 1917839 - Posted: 9 Feb 2018, 6:35:08 UTC - in response to Message 1917822.  

If the "active" users were heavy-duty crunchers, maybe no difference in project throughput.

Yep.
If would depend on how much the top 2% of active users contribute to the total Returned-per-hour number.
Grant
Darwin NT
ID: 1917839 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13855
Credit: 208,696,464
RAC: 304
Australia
Message 1917843 - Posted: 9 Feb 2018, 7:42:51 UTC

Hmmm.
The daily glitch appears to have moved about 45min to an hour later.
Grant
Darwin NT
ID: 1917843 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1917847 - Posted: 9 Feb 2018, 7:47:39 UTC - in response to Message 1917843.  

Seems like it hits here at the same time. Just posted in the lounge the site went dark and no connections were going through. Notice at Haveland they just dumped a boatload of wu deletions.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1917847 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 1917859 - Posted: 9 Feb 2018, 13:28:03 UTC - in response to Message 1917847.  

Seems like it hits here at the same time. Just posted in the lounge the site went dark and no connections were going through. Notice at Haveland they just dumped a boatload of wu deletions.


. . Well looking at the tapes splitting it looks like being a dry old weekend ...

Stephen

:(
ID: 1917859 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1917864 - Posted: 9 Feb 2018, 13:59:20 UTC

I've been seeing the servers get tied in knots lately every night when I get home at midnight CST.
Hard to contact the servers or the forum pages at all.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1917864 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1917884 - Posted: 9 Feb 2018, 16:02:46 UTC

I found it strange when Grant mentioned he believe the project communications strangeness happened 45 minutes earlier than normal and for me it was exactly the same.

For me the project becomes unreachable or slow at 11:30 - 12:00 Midnight PST every night. That points to something locally happening at each of our locations since we are in very different time zones.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1917884 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1917885 - Posted: 9 Feb 2018, 16:10:50 UTC - in response to Message 1917859.  


. . Well looking at the tapes splitting it looks like being a dry old weekend ...

Stephen

:(

Looks like they added a bit more data overnight. When I went to bed the splitters only had about 1250 channels left to do. Now the unsplit channels are over 2000.

We are going to need more than that though to make it through to Monday.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1917885 · Report as offensive
Profile Ghan-buri-Ghan Mike

Send message
Joined: 27 Dec 15
Posts: 123
Credit: 92,602,985
RAC: 172
United States
Message 1917916 - Posted: 9 Feb 2018, 20:49:33 UTC

BOINC permits users via their cc_config files to report all results immediately. That means many more, albeit much shorter, queries to the servers (as opposed to say an hourly update). Would restricting (or asking users not to use ) this setting help or hurt the server load issue? Could hourly updates be staggered by sorting users (maybe alphabetically by user name or some other criteria) to even our the server load? Just a thought.....
ID: 1917916 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13855
Credit: 208,696,464
RAC: 304
Australia
Message 1917925 - Posted: 9 Feb 2018, 21:40:03 UTC - in response to Message 1917884.  
Last modified: 9 Feb 2018, 21:40:33 UTC

I found it strange when Grant mentioned he believe the project communications strangeness happened 45 minutes earlier than normal and for me it was exactly the same.

Later than usual.

On the Haveland graphs it used to be a bit after 06:00. The last couple of days it's been around 07:00.
Grant
Darwin NT
ID: 1917925 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13855
Credit: 208,696,464
RAC: 304
Australia
Message 1917927 - Posted: 9 Feb 2018, 21:45:17 UTC - in response to Message 1917916.  

BOINC permits users via their cc_config files to report all results immediately. That means many more, albeit much shorter, queries to the servers (as opposed to say an hourly update). Would restricting (or asking users not to use ) this setting help or hurt the server load issue? Could hourly updates be staggered by sorting users (maybe alphabetically by user name or some other criteria) to even our the server load? Just a thought.....

Better to have things spread out over a 24 hour period than lots of short, sharp, heavy demands on the system.
Getting work after an outage can be difficult- having groups of people all trying to report & get work at the same time will be lots of such events throughout the day. Add to the fact that I have to limit the number of results I report at time, otherwise i get nothing but Scheduler errors after an outage means such a system would just lead to other problems. Lots of smaller Scheduler requests through out the day are better than lots of large ones at one time.
Grant
Darwin NT
ID: 1917927 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1917932 - Posted: 9 Feb 2018, 22:21:49 UTC - in response to Message 1917927.  

I have to limit the number of results I report at time, otherwise i get nothing but Scheduler errors ...
Having a less verbose std_err report per task in the sched_request file might help with that one.
ID: 1917932 · Report as offensive
Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 37 · Next

Message boards : Number crunching : Panic Mode On (110) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.