Panic Mode On (116) Server Problems?

Message boards : Number crunching : Panic Mode On (116) Server Problems?
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 36 · 37 · 38 · 39 · 40 · 41 · 42 . . . 47 · Next

AuthorMessage
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1999743 - Posted: 26 Jun 2019, 15:28:08 UTC - in response to Message 1999725.  
Last modified: 26 Jun 2019, 15:29:14 UTC

While we can't be Tron, it does look like things are getting better. There are WUs in the RTS queue. I'm guessing some people have just shut off their machines as the Results out in the Field is much lower than usual.

The results received in the last hour is also down to 150k . Still high but better than when it is in the 200k+ range.

edit: I see it isn't people turning off their machines, but an upload problem that is causing the numbers .
ID: 1999743 · Report as offensive
Profile Siran d'Vel'nahr
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 7379
Credit: 44,181,323
RAC: 238
United States
Message 1999747 - Posted: 26 Jun 2019, 15:34:01 UTC

Greetings,

My Winders PC is OUT OF WORK! But my Linux box and Pis all have full caches. What gives? Why does my Winders PC have such a hard time uploading, but hardly a problem with my Linux box. Very aggravating...

Have a great day, if you can! :)

Siran
CAPT Siran d'Vel'nahr - L L & P _\\//
Winders 11 OS? "What a piece of junk!" - L. Skywalker
"Logic is the cement of our civilization with which we ascend from chaos using reason as our guide." - T'Plana-hath
ID: 1999747 · Report as offensive
Profile Shannon Lester
Avatar

Send message
Joined: 27 Jul 09
Posts: 83
Credit: 12,388,119
RAC: 140
United States
Message 1999751 - Posted: 26 Jun 2019, 15:51:10 UTC

Well I'll be out of work soon too... looks like a lot of my wu's were junk. :(

All of this has happened before and will happen again -Battlestar Galactica
ID: 1999751 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1999754 - Posted: 26 Jun 2019, 16:06:48 UTC

Fun and games on the left coast I see. Upload server having a very difficult time handling the huge volume of uploads from the noise bombs. Out of cpu work again, gpu cache continues to drop because stuck uploads prevent downloading of any new work.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1999754 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1999755 - Posted: 26 Jun 2019, 16:09:04 UTC - in response to Message 1999743.  
Last modified: 26 Jun 2019, 16:27:44 UTC

edit: I see it isn't people turning off their machines, but an upload problem that is causing the numbers .
Yes, a double (or even treble) whammy.

Even a noise bomb creates an upload file: BOINC won't automatically report or request new work while some upload is stalled: and even if you retry them all, BOINC won't request new work while there are too many uploads in the queue. I'm working the retry button on a machine which has a cold GPU because of that lot. They're going through slowly.

Edit - got it down to the magic number, and got a decent refill - but now the b***ers won't download, either!

Edit2 - and the few which do download are bombing out faster than they download!
ID: 1999755 · Report as offensive
Loren Datlof

Send message
Joined: 24 Jan 14
Posts: 73
Credit: 19,652,385
RAC: 0
United States
Message 1999759 - Posted: 26 Jun 2019, 16:18:16 UTC - in response to Message 1999755.  

A few of my machines ran out of GPU tasks which I presume is due to noise bombs and stalled uploads. They are now crunching some Einstein tasks. I am surprised at how often these machines crunch Einstein tasks considering that Einstein's resource share is set to zero in BOINC.
ID: 1999759 · Report as offensive
Loren Datlof

Send message
Joined: 24 Jan 14
Posts: 73
Credit: 19,652,385
RAC: 0
United States
Message 1999765 - Posted: 26 Jun 2019, 16:39:54 UTC - in response to Message 1999760.  

Interesting Wednesday indeed.
I managed to upload and report enough tasks to get new work, but then downloading is borked too:-)


On one of my machines was able to clear its uploads and actually download some GPU tasks which so far have all turned out to be noise bombs. Oh well...
ID: 1999765 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1999766 - Posted: 26 Jun 2019, 16:46:47 UTC

What a mess! It is almost 10am in California, and it looks like from a previous post that a message has been sent to Eric. Now to see if we just have to plow through all this noisy data to process the few valid WUs, or if they will pull these datafiles (to possibly feed back to us in smaller chunks ??)

It is a big mess when my slow machine has problems. It did take a while to download some new WUs, which I hope is more good than noise.
ID: 1999766 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1999772 - Posted: 26 Jun 2019, 17:29:48 UTC

I have one host that was able to clear its uploads and get new work. The other hosts still can't clear their uploads.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1999772 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1999776 - Posted: 26 Jun 2019, 17:50:10 UTC

I didn't look at the SSP page earlier so don't know when the troublesome BLC24 noise bombs either finished or were pulled. Gone now. Good BLC34 and the long running BLC41 tasks are ready to split again.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1999776 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1999779 - Posted: 26 Jun 2019, 18:17:03 UTC
Last modified: 26 Jun 2019, 18:21:55 UTC

Well, that's a novel way to reduce the load on the download servers- take out the upload server.

And I notice the splitters are back to having output issues. Probably take a while to cache the new data files (hopefully that's all it is).
Grant
Darwin NT
ID: 1999779 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1999781 - Posted: 26 Jun 2019, 18:19:54 UTC

I have at least two machines with stalled uploads.
Wed 26 Jun 2019 01:17:45 PM CDT |  | Project communication failed: attempting access to reference site
Wed 26 Jun 2019 01:17:45 PM CDT | SETI@home | Temporarily failed upload of blc24_2bit_guppi_58340_60032_HIP25577_0092.8550.834.19.28.21.vlar_2_r1112692488_0: transient HTTP error

A proud member of the OFA (Old Farts Association).
ID: 1999781 · Report as offensive
Profile Siran d'Vel'nahr
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 7379
Credit: 44,181,323
RAC: 238
United States
Message 1999783 - Posted: 26 Jun 2019, 18:22:44 UTC
Last modified: 26 Jun 2019, 18:29:20 UTC

Winders PC STILL cannot get uploads off to California. :(

No problem with Linux or Pis. :)

I'm now wondering if my Winders PC is about to byte the BIG one and go belly up. :|

[edit]
blc24_2bit_guppi_58340_58652_HIP25191_0088.20089.409.20.29.125.vlar_0_r817257672_0
6/26/2019 1:23:28 PM | | Project communication failed: attempting access to reference site
6/26/2019 1:23:29 PM | | Internet access OK - project servers may be temporarily down.
6/26/2019 1:24:27 PM | SETI@home | [sched_op] Starting scheduler request
6/26/2019 1:24:27 PM | SETI@home | Sending scheduler request: To report completed tasks.
6/26/2019 1:24:27 PM | SETI@home | Reporting 22 completed tasks
6/26/2019 1:24:27 PM | SETI@home | Not requesting tasks: too many uploads in progress
6/26/2019 1:24:27 PM | SETI@home | [sched_op] CPU work request: 0.00 seconds; 0.00 devices
6/26/2019 1:24:27 PM | SETI@home | [sched_op] NVIDIA GPU work request: 0.00 seconds; 0.00 devices
6/26/2019 1:24:29 PM | SETI@home | Scheduler request completed
6/26/2019 1:24:29 PM | SETI@home | [sched_op] Server version 709
6/26/2019 1:24:29 PM | SETI@home | Project requested delay of 303 seconds
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task 02my19ab.16804.11928.14.41.231_2
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_57981_HIP25152_0086.10523.836.20.29.98.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_57981_HIP25152_0086.10468.836.19.28.7.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_57981_HIP25152_0086.10523.836.20.29.73.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_57981_HIP25152_0086.12325.409.19.28.54.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_60032_HIP25577_0092.18997.834.20.29.61.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_59696_HIP26335_0091.24855.409.19.28.253.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc32_2bit_guppi_58406_30334_HIP20482_0113.29065.818.21.44.145.vlar_2
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_58317_HIP25878_0087.30783.818.20.29.123.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_62249_HIP43339_0098.3623.933.20.29.108.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_57306_HIP25121_0084.6156.0.20.29.54.vlar_2
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_60032_HIP25577_0092.9859.420.19.28.88.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_59696_HIP26335_0091.9902.825.19.28.58.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_62249_HIP43339_0098.11734.933.20.29.198.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_58317_HIP25878_0087.12620.0.19.28.216.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_57645_HIP25878_0085.29782.0.19.28.23.vlar_2
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_58317_HIP25878_0087.12638.0.20.29.2.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_62249_HIP43339_0098.13896.933.20.29.107.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_61914_HIP44127_0097.14021.0.20.29.103.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_61571_HIP43335_0096.18046.416.19.28.226.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_61571_HIP43335_0096.18046.416.19.28.209.vlar_0
6/26/2019 1:24:29 PM | SETI@home | [sched_op] handle_scheduler_reply(): got ack for task blc24_2bit_guppi_58340_61571_HIP43335_0096.18046.416.19.28.89.vlar_1
6/26/2019 1:24:29 PM | SETI@home | [sched_op] Deferring communication for 00:05:03
6/26/2019 1:24:29 PM | SETI@home | [sched_op] Reason: requested by project

CAPT Siran d'Vel'nahr - L L & P _\\//
Winders 11 OS? "What a piece of junk!" - L. Skywalker
"Logic is the cement of our civilization with which we ascend from chaos using reason as our guide." - T'Plana-hath
ID: 1999783 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1999784 - Posted: 26 Jun 2019, 18:23:21 UTC - in response to Message 1999781.  

I have at least two machines with stalled uploads.

Took me 20min of Retries to get mine to clear.
Grant
Darwin NT
ID: 1999784 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 1999789 - Posted: 26 Jun 2019, 18:59:31 UTC

The out in the field number keeps falling, so I know there is still a download problem.

It looks like when they took away the blc24s, that were a noisy mess, that they also added in some new blc42s.

blc42_2bit_guppi_58543_61810_3C147_0007 thru 0025
ID: 1999789 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1999792 - Posted: 26 Jun 2019, 19:06:18 UTC - in response to Message 1999789.  
Last modified: 26 Jun 2019, 19:07:37 UTC

The out in the field number keeps falling, so I know there is still a download problem.

Yep, probably even more of an issue that it was before. Even more retrying is needed to get anything to upload.
But at least I've got a couple of hours work now which will help warm up the house.
Grant
Darwin NT
ID: 1999792 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13164
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1999794 - Posted: 26 Jun 2019, 19:24:18 UTC - in response to Message 1999784.  

I have at least two machines with stalled uploads.

Took me 20min of Retries to get mine to clear.

I'd love to know the trick to clear the stalled uploads. Nothing I have tried has allowed them to upload. Several thousand uploads pending on each host.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1999794 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11361
Credit: 29,581,041
RAC: 66
United States
Message 1999796 - Posted: 26 Jun 2019, 19:32:41 UTC

I expect my backup project will get some attention if doesn't get resolved soon.
ID: 1999796 · Report as offensive
Profile B. Ahmet KIRAN

Send message
Joined: 19 Oct 14
Posts: 77
Credit: 36,140,903
RAC: 140
Turkey
Message 1999797 - Posted: 26 Jun 2019, 19:45:32 UTC

10 hours and counting... When is this TORTURE going to end? Anybody any idea?
ID: 1999797 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19062
Credit: 40,757,560
RAC: 67
United Kingdom
Message 1999799 - Posted: 26 Jun 2019, 20:00:40 UTC

I managed to clear my uploads, but then got some blc24 _2's. AGGGGHHHHH.
ID: 1999799 · Report as offensive
Previous · 1 . . . 36 · 37 · 38 · 39 · 40 · 41 · 42 . . . 47 · Next

Message boards : Number crunching : Panic Mode On (116) Server Problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.