Panic Mode On (110) Server Problems?

Author	Message
Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921557 - Posted: 28 Feb 2018, 3:02:02 UTC Well, Server Status page is now green, although Scheduler responses (no work available) are taking a minute or more to arrive. Grant Darwin NT ID: 1921557 ·

juan BFP Volunteer tester Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799	Message 1921558 - Posted: 28 Feb 2018, 3:03:32 UTC - in response to Message 1921554. Sounds likely. 422,146 hours backlog is something over 48 years, which takes us back to early 1970. Sound familiar? Unix time zero? I think you have likely deduced the issue Richard. +1 BTW After the outage the updates are ending on: Tue 27 Feb 2018 10:00:44 PM EST \| SETI@home \| Scheduler request failed: HTTP internal server error ID: 1921558 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921559 - Posted: 28 Feb 2018, 3:03:48 UTC - in response to Message 1921443. Glad the staff was able to fix that without more upset. Very helpful to know the creation rate, RTS buffer and tasks returned per hour to know whether the splitters are able to keep up with demand. There is still something wrong with the transitioner though. Haveland transitioner graph has been flatlined for a day. And the WU deleters don't seem capable of keeping up any more, so that's probably impacting on splitter output as well. Grant Darwin NT ID: 1921559 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921562 - Posted: 28 Feb 2018, 3:19:14 UTC - in response to Message 1921558. Last modified: 28 Feb 2018, 3:19:44 UTC BTW After the outage the updates are ending on: Tue 27 Feb 2018 10:00:44 PM EST \| SETI@home \| Scheduler request failed: HTTP internal server error I'm still getting Scheduler responses, just taking ages for them to finally arrive. And still no work from any of them. Grant Darwin NT ID: 1921562 ·

Iona Send message Joined: 12 Jul 07 Posts: 790 Credit: 22,438,118 RAC: 0	Message 1921563 - Posted: 28 Feb 2018, 3:23:34 UTC I was down to my last GPU task and I've just got some more work, but not a full quota . The downloads are mighty slow, though. Don't take life too seriously, as you'll never come out of it alive! ID: 1921563 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921565 - Posted: 28 Feb 2018, 3:27:56 UTC - in response to Message 1921563. I was down to my last GPU task and I've just got some more work, but not a full quota . I think you might be the only person- the Ready-to-send buffer has actually increased (slightly) since the outage. Usually it hits the ground like a stone. Hopefully the servers will sort them selves out in the not too distant future. The downloads are mighty slow, though. I'm finding the forums a bit sluggish at the moment. Takes a while for the page to come up & the thread to start to load. Grant Darwin NT ID: 1921565 ·

Chris904395093209d Volunteer tester Send message Joined: 1 Jan 01 Posts: 112 Credit: 29,923,129 RAC: 6	Message 1921569 - Posted: 28 Feb 2018, 3:46:20 UTC Dang it, my fastest PC has been out of CPU tasks for a few hours at least. Normally I can get new tasks as soon as the servers are back and before the pool runs dry. But not tonight. SETI was kind enough to send 1 GPU task for this PC though :-( ~Chris ID: 1921569 ·

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1921572 - Posted: 28 Feb 2018, 4:03:56 UTC Lots of task in the download queue. None of them make it down. Backoffs around 3 1/2 hours. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1921572 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921574 - Posted: 28 Feb 2018, 4:07:05 UTC - in response to Message 1921572. Last modified: 28 Feb 2018, 4:07:42 UTC Lots of task in the download queue. None of them make it down. Backoffs around 3 1/2 hours. Yep. Gone from not getting any, to not being able to get them to the system once I have got them allocated. The recovery after outages would appear to be getting worse, not better. Grant Darwin NT ID: 1921574 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921581 - Posted: 28 Feb 2018, 4:53:11 UTC Still no joy with downloads. It's times like these I really miss the Cricket graphs. Grant Darwin NT ID: 1921581 ·

Stephen "Heretic" Volunteer tester Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628	Message 1921583 - Posted: 28 Feb 2018, 5:10:51 UTC - in response to Message 1921574. Lots of task in the download queue. None of them make it down. Backoffs around 3 1/2 hours. Yep. Gone from not getting any, to not being able to get them to the system once I have got them allocated. The recovery after outages would appear to be getting worse, not better. . . Same here allocated 40 tasks (or so) and it took over half an hour to download them :( Stephen :( ID: 1921583 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921585 - Posted: 28 Feb 2018, 5:27:22 UTC - in response to Message 1921583. . . Same here allocated 40 tasks (or so) and it took over half an hour to download them :( Once the download server came back to life they downloaded quickly. Now all we need is for the splitters to start cranking up their output. Grant Darwin NT ID: 1921585 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921587 - Posted: 28 Feb 2018, 6:07:57 UTC - in response to Message 1921534. I wonder if the transitioner backlog error was somehow tied to the missing creation rate values for most of yesterday. Those got fixed late in the day but the transitioner reporting is still out to lunch. Looks like the Transitioner updating is still broken. Grant Darwin NT ID: 1921587 ·

Wiggo Send message Joined: 24 Jan 00 Posts: 36390 Credit: 261,360,520 RAC: 489	Message 1921591 - Posted: 28 Feb 2018, 8:06:01 UTC - in response to Message 1921563. I was down to my last GPU task and I've just got some more work, but not a full quota . The downloads are mighty slow, though. I was out for a few hours before I could get a handful on 1 rig, but I did smash some extra CPU tasks during that time. But after 2hrs at the pub things here were all good again (proof again that beer will fix everything), but that will hurt the old RAC again. Cheers. ID: 1921591 ·

Kissagogo27 Send message Joined: 6 Nov 99 Posts: 716 Credit: 8,032,827 RAC: 62	Message 1921616 - Posted: 28 Feb 2018, 10:29:20 UTC about the same here this morning 28-Feb-2018 09:51:40 [SETI@home] Requesting new tasks for CPU and AMD/ATI GPU 28-Feb-2018 09:51:43 [SETI@home] Scheduler request completed: got 2 new tasks 28-Feb-2018 09:51:46 [SETI@home] Started download of ap_14fe18aa_B6_P1_00286_20180228_07134.wu 28-Feb-2018 09:52:51 [SETI@home] Finished download of ap_14fe18aa_B6_P1_00286_20180228_07134.wu 28-Feb-2018 09:52:51 [SETI@home] Started download of ap_15au16aa_B2_P1_00209_20180228_09870.wu 28-Feb-2018 09:53:48 [SETI@home] Finished download of ap_15au16aa_B2_P1_00209_20180228_09870.wu 28-Feb-2018 09:56:50 [SETI@home] Sending scheduler request: To fetch work. 28-Feb-2018 09:56:50 [SETI@home] Requesting new tasks for CPU and AMD/ATI GPU 28-Feb-2018 09:56:53 [SETI@home] Scheduler request completed: got 1 new tasks 28-Feb-2018 09:56:55 [SETI@home] Started download of ap_17au16ac_B1_P1_00287_20180228_07159.wu 28-Feb-2018 10:01:56 [SETI@home] Temporarily failed download of ap_17au16ac_B1_P1_00287_20180228_07159.wu: transient HTTP error 28-Feb-2018 10:01:56 [SETI@home] Backing off 00:02:18 on download of ap_17au16ac_B1_P1_00287_20180228_07159.wu 28-Feb-2018 10:01:59 [---] Project communication failed: attempting access to reference site 28-Feb-2018 10:02:02 [---] Internet access OK - project servers may be temporarily down. 28-Feb-2018 10:02:57 [SETI@home] Computation for task blc13_2bit_guppi_58137_40992_HIP64241_0041.3597.0.21.44.77.vlar_1 finished 28-Feb-2018 10:02:57 [SETI@home] Starting task 12fe18aa.7416.215582.15.42.231_1 28-Feb-2018 10:03:05 [SETI@home] Started upload of blc13_2bit_guppi_58137_40992_HIP64241_0041.3597.0.21.44.77.vlar_1_r1107740128_0 28-Feb-2018 10:03:21 [SETI@home] Finished upload of blc13_2bit_guppi_58137_40992_HIP64241_0041.3597.0.21.44.77.vlar_1_r1107740128_0 28-Feb-2018 10:04:16 [SETI@home] Started download of ap_17au16ac_B1_P1_00287_20180228_07159.wu 28-Feb-2018 10:04:22 [SETI@home] Sending scheduler request: To fetch work. 28-Feb-2018 10:04:22 [SETI@home] Reporting 1 completed tasks 28-Feb-2018 10:04:22 [SETI@home] Requesting new tasks for CPU and AMD/ATI GPU 28-Feb-2018 10:04:24 [SETI@home] Scheduler request completed: got 0 new tasks 28-Feb-2018 10:04:24 [SETI@home] Project has no tasks available 28-Feb-2018 10:05:01 [SETI@home] Finished download of ap_17au16ac_B1_P1_00287_20180228_07159.wu transient slow down in data transfers, easy to see with AP wu ^^ ID: 1921616 ·

Grant (SSSF) Volunteer tester Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304	Message 1921760 - Posted: 28 Feb 2018, 22:43:25 UTC File deleters & WU purges still not keeping up, and most likely as a direct result of that is the limited splitter output. Grant Darwin NT ID: 1921760 ·

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1921923 - Posted: 1 Mar 2018, 17:47:32 UTC - in response to Message 1921760. Database is very slow now. Can't pull up anybody's tasks, just watch the spinner endlessly. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1921923 ·

Bernie Vine Volunteer moderator Volunteer tester Send message Joined: 26 May 99 Posts: 9956 Credit: 103,452,613 RAC: 328	Message 1921924 - Posted: 1 Mar 2018, 17:51:03 UTC Well seems to work OK here, just had a look at a couple of your tasks OK ID: 1921924 ·

Keith Myers Volunteer tester Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873	Message 1921926 - Posted: 1 Mar 2018, 17:59:09 UTC - in response to Message 1921924. Last modified: 1 Mar 2018, 18:02:14 UTC I just pulled up some of your tasks and they got displayed quickly. Looking at anyone else with several thousand tasks in any category is painfully slow, if the page ever populates. [Edit] I'm specifically trying to look at Petri's Titan V results. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) ID: 1921926 ·

Zalster Volunteer tester Send message Joined: 27 May 99 Posts: 5517 Credit: 528,817,460 RAC: 242	Message 1921934 - Posted: 1 Mar 2018, 18:37:16 UTC - in response to Message 1921926. specifically trying to look at Petri's Titan V results. While it shows in the computer, I'm not seeing any work being done on that card. ID: 1921934 ·

©2024 University of California

SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.