The Server Issues / Outages Thread - Panic Mode On! (118)

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 94 · Next

AuthorMessage
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 2026745 - Posted: 8 Jan 2020, 1:42:12 UTC - in response to Message 2026741.  
Last modified: 8 Jan 2020, 1:49:43 UTC

Off to let Einstein torture me for a bit. Sure wish 7.16.3 hadn't fubared the scheduler ... used to be able to share work on my terms ...


I have opened up my World Community Grid load some. Maybe I will take it off the leash till tomorrow. Looks like I have E@H configured to behave more nicely. Guess I will let it munch.

Tom
A proud member of the OFA (Old Farts Association).
ID: 2026745 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 38114
Credit: 261,360,520
RAC: 489
Australia
Message 2026747 - Posted: 8 Jan 2020, 1:57:06 UTC

Did someone forget to open the gates?

RTS is now over 900K.

Cheers.
ID: 2026747 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13946
Credit: 208,696,464
RAC: 304
Australia
Message 2026757 - Posted: 8 Jan 2020, 3:27:33 UTC - in response to Message 2026717.  

So far the smoothest recovery I've seen in quite a while. Was able to report all work immediately, and got a small download. Guess we'll see how it goes; hope my optimism doesn't jinx anything :)
You jinxed it.
I reported & picked up some work early on, but have since run out of GPU work on my Linux system as "Project has no tasks available" has been the response for around 40min now. My Windows system for some reason has been able to pick up work several times in that period, but not on every request. Probably every 3-4.
Grant
Darwin NT
ID: 2026757 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 2026761 - Posted: 8 Jan 2020, 3:46:04 UTC

It looks as though we are finally getting some downloads, however, just as Always the Splitters have failed to ramp up. It would be nice if someone can persuade the Splitters to start working before the RTS reaches Zero, as it seems they Always do. Otherwise, we will be Out of Work in a few hours again,
Results ready to send = 336,224
Current result creation rate = 4.7576/sec
ID: 2026761 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2026764 - Posted: 8 Jan 2020, 4:09:07 UTC - in response to Message 2026739.  

So far the smoothest recovery I've seen in quite a while. Was able to report all work immediately, and got a small download. Guess we'll see how it goes; hope my optimism doesn't jinx anything :)


. . If it does we'll just blame you :)

Stephen

:)

Blame accepted:) Haven't gotten squat since ...


. . the same old SNAFU after an outage. I wonder what it will be like if they get the 'new' server working (Muarae2).

. . We can but daydream ... :)

Stephen

:)
ID: 2026764 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13946
Credit: 208,696,464
RAC: 304
Australia
Message 2026765 - Posted: 8 Jan 2020, 4:13:33 UTC

And on those somewhat rare occasions I do pick up work, the elapsed time ticks away on the download, but no bits are moving.
And as I typed that, after over a minute of nothing happening, they managed to start (and finish) downloading.
Grant
Darwin NT
ID: 2026765 · Report as offensive
Profile Unixchick Project Donor
Avatar

Send message
Joined: 5 Mar 12
Posts: 815
Credit: 2,361,516
RAC: 22
United States
Message 2026770 - Posted: 8 Jan 2020, 5:10:07 UTC

out in the field is 6.7 million and it is usually around 7.3, so there is quite a hole to fill. I wish the server could have a better recovery algorithm, so it could give those who have none some before topping off the caches of those who have plenty.


I'm going to hang out in the wilds for a bit, so I've emptied my machine, so I'm not competing for WUs with you right now. I'll do some howling at the moon and dancing around the fire in hopes that these silly rituals help keep the seti machines working well :-).
ID: 2026770 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 38114
Credit: 261,360,520
RAC: 489
Australia
Message 2026771 - Posted: 8 Jan 2020, 5:13:39 UTC

My caches are finally full again. :-)

Cheers.
ID: 2026771 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13946
Credit: 208,696,464
RAC: 304
Australia
Message 2026774 - Posted: 8 Jan 2020, 5:38:15 UTC - in response to Message 2026771.  

My caches are finally full again. :-)
One cache full, the other still filling. Most responses are still "Project has no tasks available", but when it does get work, it's getting a lot. So it's cache is slowly managing to refill. 5 steps forward, 4 steps back.
Grant
Darwin NT
ID: 2026774 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2026786 - Posted: 8 Jan 2020, 10:23:32 UTC - in response to Message 2026770.  

out in the field is 6.7 million and it is usually around 7.3, so there is quite a hole to fill. I wish the server could have a better recovery algorithm, so it could give those who have none some before topping off the caches of those who have plenty.


I'm going to hang out in the wilds for a bit, so I've emptied my machine, so I'm not competing for WUs with you right now. I'll do some howling at the moon and dancing around the fire in hopes that these silly rituals help keep the seti machines working well :-).


. . Have fun and don't bring any stray coyotes home :)

Stephen

:)
ID: 2026786 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2026787 - Posted: 8 Jan 2020, 10:24:59 UTC - in response to Message 2026774.  

My caches are finally full again. :-)
One cache full, the other still filling. Most responses are still "Project has no tasks available", but when it does get work, it's getting a lot. So it's cache is slowly managing to refill. 5 steps forward, 4 steps back.


. . I would describe it more as 4 steps back and a big leap forward to catch up.

Stephen

:(
ID: 2026787 · Report as offensive
Profile Siran d'Vel'nahr
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 7381
Credit: 44,181,323
RAC: 238
United States
Message 2026793 - Posted: 8 Jan 2020, 11:38:29 UTC

Greetings,

If there was no such thing as artificial "spoofing" perhaps those users that can only get a few WUs and can't, after a maintenance cycle, could. Just sayin'... :)

Yeah I know, I'm kinda spoofing too, but I have no control over it. BOINC sees 2 GPUs and gets WUs accordingly, even though only one GPU is running tasks. The other drives my monitor.

Have a great day! :)

Siran
CAPT Siran d'Vel'nahr - L L & P _\\//
Winders 11 OS? "What a piece of junk!" - L. Skywalker
"Logic is the cement of our civilization with which we ascend from chaos using reason as our guide." - T'Plana-hath
ID: 2026793 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 2026795 - Posted: 8 Jan 2020, 12:02:12 UTC - in response to Message 2026786.  

out in the field is 6.7 million and it is usually around 7.3, so there is quite a hole to fill. I wish the server could have a better recovery algorithm, so it could give those who have none some before topping off the caches of those who have plenty.


I'm going to hang out in the wilds for a bit, so I've emptied my machine, so I'm not competing for WUs with you right now. I'll do some howling at the moon and dancing around the fire in hopes that these silly rituals help keep the seti machines working well :-).


. . Have fun and don't bring any stray coyotes home :)

Stephen

:)


What about "friendly" stray Coyotes?
A proud member of the OFA (Old Farts Association).
ID: 2026795 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5126
Credit: 276,046,078
RAC: 462
Message 2026796 - Posted: 8 Jan 2020, 12:03:33 UTC

Got up this morning. Turned E@H off and as soon as I cleared out some of my cache of E@H started getting downloads on both my "normal" projects.

Tom
A proud member of the OFA (Old Farts Association).
ID: 2026796 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14690
Credit: 200,643,578
RAC: 874
United Kingdom
Message 2026799 - Posted: 8 Jan 2020, 12:07:58 UTC

Referring back to the server issue of 20 December (Anonymous Platform failure after upgrade), I have written up the story so far at #3419. Nils Høimyr of LHC has produced some useful diagnostics, but LHC now feel that they have to refer the problem back to David Anderson.
ID: 2026799 · Report as offensive
Stephen "Heretic" Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Sep 12
Posts: 5557
Credit: 192,787,363
RAC: 628
Australia
Message 2026806 - Posted: 8 Jan 2020, 13:18:37 UTC - in response to Message 2026795.  

I'm going to hang out in the wilds for a bit. I'll do some howling at the moon and dancing around the fire in hopes that these silly rituals help keep the seti machines working well :-).


. . Have fun and don't bring any stray coyotes home :)
Stephen


What about "friendly" stray Coyotes?


. . If she is running about in the woods howling you just never know ...

Stephen

? ?
ID: 2026806 · Report as offensive
jdzukley Project Donor

Send message
Joined: 6 Apr 11
Posts: 19
Credit: 26,357,809
RAC: 74
United States
Message 2026828 - Posted: 8 Jan 2020, 14:59:43 UTC

From my perspective, and while I do not want to discount that there are issues and problems out there... If you look at the current server status page lately and currently, we have maxed out the site. We are all crunching all the work the site can give to us. The ready to send que is very low, and generally is, and the tasks counts being created are high. This is all great stuff as far as I am concerned!
ID: 2026828 · Report as offensive
Kiska
Volunteer tester

Send message
Joined: 31 Mar 12
Posts: 302
Credit: 3,067,762
RAC: 0
Australia
Message 2026830 - Posted: 8 Jan 2020, 15:19:43 UTC - in response to Message 2026828.  

From my perspective, and while I do not want to discount that there are issues and problems out there... If you look at the current server status page lately and currently, we have maxed out the site. We are all crunching all the work the site can give to us. The ready to send que is very low, and generally is, and the tasks counts being created are high. This is all great stuff as far as I am concerned!


Munin graphs kinda confirm this:






ID: 2026830 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19691
Credit: 40,757,560
RAC: 67
United Kingdom
Message 2026891 - Posted: 9 Jan 2020, 0:34:48 UTC - in response to Message 2026859.  

Beta is back, or at least there's life there. It's slowly coming to life, but don't expect that everything works yet,
It's been down since Dec 20, so it will take time to recover.

Reported 50 tasks, got two new tasks from Beta.
ID: 2026891 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13946
Credit: 208,696,464
RAC: 304
Australia
Message 2026911 - Posted: 9 Jan 2020, 2:49:47 UTC - in response to Message 2026859.  

Beta is back, or at least there's life there. It's slowly coming to life, but don't expect that everything works yet,
It's been down since Dec 20, so it will take time to recover.
Curious as to what the issue turned out to be.
Grant
Darwin NT
ID: 2026911 · Report as offensive
Previous · 1 . . . 19 · 20 · 21 · 22 · 23 · 24 · 25 . . . 94 · Next

Message boards : Number crunching : The Server Issues / Outages Thread - Panic Mode On! (118)


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.