The Server Issues / Outages Thread - Panic Mode On! (119)

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (119)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 61 · 62 · 63 · 64 · 65 · 66 · 67 . . . 107 · Next

AuthorMessage
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13819
Credit: 208,696,464
RAC: 304
Australia
Message 2043099 - Posted: 5 Apr 2020, 2:37:10 UTC - in response to Message 2043095.  

Will slow down the crunch but at least the work will be reported and the new task will allow to flow.
Good luck with that.
I've picked up 18 resends since the last of the splitter output cleared from the Ready-to-send buffer. Around 4 per day.
Grant
Darwin NT
ID: 2043099 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2043106 - Posted: 5 Apr 2020, 3:04:58 UTC - in response to Message 2043099.  

Will slow down the crunch but at least the work will be reported and the new task will allow to flow.
Good luck with that.
I've picked up 18 resends since the last of the splitter output cleared from the Ready-to-send buffer. Around 4 per day.


. . Yep I would say about the same ...

Stephen

:(
ID: 2043106 · Report as offensive     Reply Quote
Ian&Steve C.
Avatar

Send message
Joined: 28 Sep 99
Posts: 4267
Credit: 1,282,604,591
RAC: 6,640
United States
Message 2043108 - Posted: 5 Apr 2020, 3:22:01 UTC - in response to Message 2043099.  

I've picked up 18 resends since the last of the splitter output cleared from the Ready-to-send buffer. Around 4 per day.


where do you see this info locally on the system? since the replica is so far behind I can't really see what tasks might have been given to my systems since the project ended.
Seti@Home classic workunits: 29,492 CPU time: 134,419 hours

ID: 2043108 · Report as offensive     Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13819
Credit: 208,696,464
RAC: 304
Australia
Message 2043112 - Posted: 5 Apr 2020, 3:29:04 UTC - in response to Message 2043108.  

I've picked up 18 resends since the last of the splitter output cleared from the Ready-to-send buffer. Around 4 per day.
where do you see this info locally on the system? since the replica is so far behind I can't really see what tasks might have been given to my systems since the project ended.
I went searching though stdoutdae.txt for "download started" after my last big batch of work on the 2/04 my time (1/04 in the US).
The Number of resends could actually be lower as the first group of 5 i included after my last big score could have had a few initial issue Tasks.
Grant
Darwin NT
ID: 2043112 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2043129 - Posted: 5 Apr 2020, 7:15:00 UTC - in response to Message 2043108.  
Last modified: 5 Apr 2020, 7:31:14 UTC

I've picked up 18 resends since the last of the splitter output cleared from the Ready-to-send buffer. Around 4 per day.


where do you see this info locally on the system? since the replica is so far behind I can't really see what tasks might have been given to my systems since the project ended.


. . just review your boinc logs and you can see when it has sent you tasks ... On this machine (C2Duo) I received 12 tasks on Thursday and on the occasions when I could connect since then nothing but 'no tasks available'.

Stephen

:)
ID: 2043129 · Report as offensive     Reply Quote
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14666
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2043138 - Posted: 5 Apr 2020, 9:07:45 UTC

With the SSP unreliable, I've been trying to pick the bones out of my task list.

The bare stats are

State: All (41295) · In progress (3599) · Validation pending (7689) · Validation inconclusive (1130) · Valid (28625) · Invalid (252) · Error (0)
with the most recent task visible sent at 31 Mar 2020, 8:45:51 UTC - so almost exactly 5 days delayed by the replica.

All (41295) ·
In progress (3599) · probably right for that time. Should be down to zero in two or three days time.
Validation pending (7689) · high, but not a source of concern at this stage.
Validation inconclusive (1130) · Too high. Still some bad hosts out there.
Valid (28625) · Absurdly high. This is what's holding us up.
Invalid (252) · gone up significantly since yesterday. May be that batch which had multiple replications created by mistake. Can't access the detail list.
Error (0) · Reassuring!
ID: 2043138 · Report as offensive     Reply Quote
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14666
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2043139 - Posted: 5 Apr 2020, 9:18:07 UTC - in response to Message 2043138.  

Invalid (252) · Can't access the detail list.
Got it now. Long list of both:

Validate error
Completed, can't validate

so it is that bad batch. Nothing we can do about it - project error, probably trying out solutions under too much time pressure and not checking things carefully enough. I hope they're getting a good, restful, weekend.
ID: 2043139 · Report as offensive     Reply Quote
Profile Kissagogo27 Special Project $75 donor
Avatar

Send message
Joined: 6 Nov 99
Posts: 716
Credit: 8,032,827
RAC: 62
France
Message 2043166 - Posted: 5 Apr 2020, 14:35:21 UTC

05-Apr-2020 04:36:54 [SETI@home] Scheduler request completed: got 2 new tasks
05-Apr-2020 06:06:39 [SETI@home] Scheduler request completed: got 3 new tasks
05-Apr-2020 13:07:40 [SETI@home] Scheduler request completed: got 1 new tasks
05-Apr-2020 13:19:11 [SETI@home] Scheduler request completed: got 1 new tasks
05-Apr-2020 13:33:02 [SETI@home] Scheduler request completed: got 1 new tasks
05-Apr-2020 13:43:14 [SETI@home] Scheduler request completed: got 1 new tasks

ID: 2043166 · Report as offensive     Reply Quote
Sleepy
Volunteer tester
Avatar

Send message
Joined: 21 May 99
Posts: 219
Credit: 98,947,784
RAC: 28,360
Italy
Message 2043188 - Posted: 5 Apr 2020, 18:10:42 UTC

I read about the lengthening of the delay between WU requests and I believe it (I also made some checks and it seems so also to me).

But when I receive the "No tasks available" answer, there is no more the message about the fact work has been asked too early and this makes me think. Whether I wait 1 second or 10 minutes.

Is it like this also by you?

Thank you!
ID: 2043188 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2043189 - Posted: 5 Apr 2020, 18:13:03 UTC - in response to Message 2043188.  

. . Yep, even when I manually initiate an early request for work there is no "last request too recent" message. Just 'no tasks available'.

Stephen

. .
ID: 2043189 · Report as offensive     Reply Quote
AllgoodGuy

Send message
Joined: 29 May 01
Posts: 293
Credit: 16,348,499
RAC: 266
United States
Message 2043243 - Posted: 6 Apr 2020, 1:11:44 UTC

And I'm back. Took a couple days away and went to pick up my pew pews in case things get stupid in California. Come back to find all my machines not doing anything, so I'll catch up with y'all. Looks like you've all had some issues, so I probably didn't miss an awful lot. I'll grind out what I have left over the next 2 or so days, and see where it takes. us. Looked over my logs and found a lot of hit and miss, small pickups, but nothing major. Looks like the SSP still reporting 20 million waiting validation, and 7.5 million assimilations.

Guy
ID: 2043243 · Report as offensive     Reply Quote
Ville Saari
Avatar

Send message
Joined: 30 Nov 00
Posts: 1158
Credit: 49,177,052
RAC: 82,530
Finland
Message 2043298 - Posted: 6 Apr 2020, 11:09:25 UTC

You get that complaint about requesting too soon only when the server was trying to send you work. So if your host didn't ask any or the server didn't have any, you don't get it.

I have exploited this in the past during the post downtime rushes to get rid of my huge pile of completed results crunched during the dt. I just manually clicked new requests as soon as the previous one had completed. Reporting 100 results in each. While having NNT set.
ID: 2043298 · Report as offensive     Reply Quote
Sleepy
Volunteer tester
Avatar

Send message
Joined: 21 May 99
Posts: 219
Credit: 98,947,784
RAC: 28,360
Italy
Message 2043315 - Posted: 6 Apr 2020, 13:12:12 UTC - in response to Message 2043298.  

You get that complaint about requesting too soon only when the server was trying to send you work. So if your host didn't ask any or the server didn't have any, you don't get it.
Thank you!
Never noticed it before, not even in the dire times when no work was available.
Maybe because I usually use scripts to handle work requests at the right time.

In any case, I tested it. With double delay I get some trickle of WUs. With old normal delay nothing in hours. Enough for me to switch to double delay.

Thank you again!
ID: 2043315 · Report as offensive     Reply Quote
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2043317 - Posted: 6 Apr 2020, 13:18:48 UTC - in response to Message 2043315.  

Thank you!
Never noticed it before, not even in the dire times when no work was available.
Maybe because I usually use scripts to handle work requests at the right time.
In any case, I tested it. With double delay I get some trickle of WUs. With old normal delay nothing in hours. Enough for me to switch to double delay.
Thank you again!


. . Better off than me, I get zero WUs no matter what delay I use.

Stephen

:(
ID: 2043317 · Report as offensive     Reply Quote
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14666
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2043367 - Posted: 6 Apr 2020, 19:15:58 UTC - in response to Message 2043342.  

And I finally got down to bedrock, with an 'oldest pending' dating back to 1st January. All that page seem to be into their second timeout, so probably won't get completed until late April or even May. Still a long haul ahead of us.
ID: 2043367 · Report as offensive     Reply Quote
AllgoodGuy

Send message
Joined: 29 May 01
Posts: 293
Credit: 16,348,499
RAC: 266
United States
Message 2043369 - Posted: 6 Apr 2020, 19:28:04 UTC - in response to Message 2043367.  
Last modified: 6 Apr 2020, 19:30:03 UTC

And I finally got down to bedrock, with an 'oldest pending' dating back to 1st January. All that page seem to be into their second timeout, so probably won't get completed until late April or even May. Still a long haul ahead of us.

My oldest just timed out on 13 Mar. It was issued 12 Jan. Luckily, the wingman at least contacted the servers on 5 April. It times out again on 13 May.
ID: 2043369 · Report as offensive     Reply Quote
AllgoodGuy

Send message
Joined: 29 May 01
Posts: 293
Credit: 16,348,499
RAC: 266
United States
Message 2043370 - Posted: 6 Apr 2020, 19:35:08 UTC
Last modified: 6 Apr 2020, 19:36:05 UTC

My laptop is down to 2 AP gpu tasks. Final CPU task is running on my 2018 Mac mini, has enough intel_gpu tasks to last roughly 3 days. 2014 Mac mini will be working for about a week on both CPU and GPU tasks, GPU disabled until the CPU runs out in about 3 days. Then I start saving about $100-150/mo in electric costs.

Edit: temperature in my room already down at least 10 degrees Fahrenheit.
ID: 2043370 · Report as offensive     Reply Quote
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 2043373 - Posted: 6 Apr 2020, 19:52:05 UTC - in response to Message 2043370.  

Meanwhile I've been running Three computers at Einstein, for Three days, and my RAC is already over 1000000, https://einsteinathome.org/account/289866
Now if I could just figure out why this AMD 570 doesn't want to work on my Hackintosh I'd be semi-happy.
ID: 2043373 · Report as offensive     Reply Quote
AllgoodGuy

Send message
Joined: 29 May 01
Posts: 293
Credit: 16,348,499
RAC: 266
United States
Message 2043375 - Posted: 6 Apr 2020, 20:03:57 UTC - in response to Message 2043373.  
Last modified: 6 Apr 2020, 20:17:40 UTC

Meanwhile I've been running Three computers at Einstein, for Three days, and my RAC is already over 1000000, https://einsteinathome.org/account/289866
Now if I could just figure out why this AMD 570 doesn't want to work on my Hackintosh I'd be semi-happy.

What OS version are you running? Should theoretically work with versions above I want to say 10.12 or 10.13ish.

Edit: that is straight from the OS level. Prior to that, might require a trusted Kext.

Edit 2: Be wary that 10.14 requires 64 bit compliance for software to run, so...there is that.
ID: 2043375 · Report as offensive     Reply Quote
Profile Kissagogo27 Special Project $75 donor
Avatar

Send message
Joined: 6 Nov 99
Posts: 716
Credit: 8,032,827
RAC: 62
France
Message 2043379 - Posted: 6 Apr 2020, 20:36:44 UTC

05-Apr-2020 16:26:59 [SETI@home] Scheduler request completed: got 6 new tasks
05-Apr-2020 17:26:02 [SETI@home] Scheduler request completed: got 1 new tasks
05-Apr-2020 18:40:19 [SETI@home] Scheduler request completed: got 1 new tasks
05-Apr-2020 23:43:43 [SETI@home] Scheduler request completed: got 2 new tasks
06-Apr-2020 08:29:09 [SETI@home] Scheduler request completed: got 1 new tasks
06-Apr-2020 14:35:44 [SETI@home] Scheduler request completed: got 2 new tasks
06-Apr-2020 16:24:45 [SETI@home] Scheduler request completed: got 3 new tasks


69 to do ...
ID: 2043379 · Report as offensive     Reply Quote
Previous · 1 . . . 61 · 62 · 63 · 64 · 65 · 66 · 67 . . . 107 · Next

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (119)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.