Have we just crashed?
Just had a bunch of stalled downloads on one system (cleared after 3-4min), and now both systems are getting "Project has no tasks available" Scheduler responses, and not a WU in sight.


Edit- eventually got some work, on both systems. And neither can download it; one system's been going for 4min & counting with not a bit transferred (and it's just gone in to timeout). The other keeps timing out on the retries.

Definitely strangeness going on. What I've never seen before is that on my Win box the stalled downloads are in excess to my full cache. e.g. with 2 GPUs I'm limited to 200 GPU tasks, I have 200 tasks in process and it's stalled downloading another 51.
Something threw a wobblie ....
[edit]
A quick peek also shows this:

All tasks for computer 8699207

State: All (872) · In progress (350) ...

With 2 GPUs and CPU max should indeed be 300 in progress,

---

Have we just crashed?
Just had a bunch of stalled downloads on one system (cleared after 3-4min), and now both systems are getting "Project has no tasks available" Scheduler responses, and not a WU in sight.


Edit- eventually got some work, on both systems. And neither can download it; one system's been going for 4min & counting with not a bit transferred (and it's just gone in to timeout). The other keeps timing out on the retries.

Definitely strangeness going on. What I've never seen before is that on my Win box the stalled downloads are in excess to my full cache. e.g. with 2 GPUs I'm limited to 200 GPU tasks, I have 200 tasks in process and it's stalled downloading another 51.
Something threw a wobblie ....
[edit]
A quick peek also shows this:

All tasks for computer 8699207

State: All (872) · In progress (350) ...

With 2 GPUs and CPU max should indeed be 300 in progress,

All tasks for computer 8699207

State: All (1042) · In progress (518)

Just keeps getting weirder ...

---

Has Xmas come early?

Both of my rigs have 490 In progress tasks each. :-O

Cheers.

I've set myself to no new task for a while, until the situation gets better. I hope everyone who needs WUs can get them.

Set myself to APs only for a while, given that the Win box which is entitled to 200 GPU tasks now has nearly 500, and a Linux box has over 150 CPU tasks. Presumably they'll figure out what happened to the scheduler check on max tasks per client device.

---

I've set myself to no new task for a while, until the situation gets better. I hope everyone who needs WUs can get them.

I did that with my 2500K rig when it hit 750 in progress.

Cheers.

Well, there's the usual & not so usual after outage weirdness going on.
My Linux system is chewing through work as fast as it can get it, most Scheduler requests resulting in "Project has no tasks available". However the Windows system on the other hand is getting work on almost every other request, and has gone on a rampage. It's usual limit is 300 WUs, yet it's already up to 469.



I'm thinking that whatever broke, might have taken out the Server side limits.
It might be worth someone letting those in charge know- as much as I would like a larger cache to get through outages such as this most recent one, I don't want the system falling over due to the amount of work in the system bringing the database to a grinding halt.

---

Grant
Darwin NT

Wow - this reminds me of before the CPU/GPU limits were imposed, and one of my machines had ~3500 WUs on board at one point...Yay!

Right now, both are well over their limits, but one has 0 CPU WUs waiting (with ~700 GPU), and the other has 750 GPU and also 170+ CPU onboard.
Both have 2 GPUs, so should max out at 22 (+100 CPU)

---

All tasks for computer 8699207

State: All (1042) · In progress (518)

Just keeps getting weirder ...

. . At this point it is safe to say something is broken. Hopefully it will only require a reset of some task/s.

Stephen

<crossing fingers>

Other than not getting work on almost all requests, the only thing out of the ordinary is that I have 111 cpu tasks on the Xeon host which should only ever get 100. The Ryzen 3900X is almost out of cpu tasks since my hosts always ask for gpu work first to fill up their cache first before ever asking for cpu work. All hosts are down on gpu work because of the scheduler sending 0 tasks upon request.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

Now I have another host with more than 100 cpu tasks. But the host that has been empty for hours of cpu work never gets any if requested. And then sets a 1400 second backoff timer. Strange.

Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | choose_project: scanning
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | can fetch CPU
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | CPU needs work - buffer low
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | checking CPU
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | [work_fetch] set_request() for CPU: ninst 15 nused_total 4191.00 nidle_now 0.00 fetch share 1.00 req_inst 0.00 req_secs 505762.90
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | CPU set_request: 505762.895780
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | checking NVIDIA GPU
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | [work_fetch] set_request() for NVIDIA GPU: ninst 48 nused_total 4191.00 nidle_now 0.00 fetch share 1.00 req_inst 0.00 req_secs 1637296.40
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | NVIDIA GPU set_request: 1637296.401672
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | [sched_op] Starting scheduler request
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | [work_fetch] request: CPU (505762.90 sec, 0.00 inst) NVIDIA GPU (1637296.40 sec, 0.00 inst)
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | Sending scheduler request: To fetch work.
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | Reporting 17 completed tasks
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | [sched_op] CPU work request: 505762.90 seconds; 0.00 devices
Sat 07 Dec 2019 12:57:32 AM PST | SETI@home | [sched_op] NVIDIA GPU work request: 1637296.40 seconds; 0.00 devices
Sat 07 Dec 2019 12:57:37 AM PST | | [work_fetch] Request work fetch: application exited
Sat 07 Dec 2019 12:57:37 AM PST | SETI@home | Computation for task 05mr11ad.28881.3339.3.30.90_0 finished
Sat 07 Dec 2019 12:57:37 AM PST | SETI@home | Starting task 05mr11ad.11989.2112.6.33.83_0
Sat 07 Dec 2019 12:57:39 AM PST | SETI@home | Started upload of 05mr11ad.28881.3339.3.30.90_0_r114699300_0
Sat 07 Dec 2019 12:57:41 AM PST | SETI@home | Finished upload of 05mr11ad.28881.3339.3.30.90_0_r114699300_0
Sat 07 Dec 2019 12:57:41 AM PST | | [work_fetch] Request work fetch: project finished uploading
Sat 07 Dec 2019 12:57:51 AM PST | SETI@home | Scheduler request completed: got 41 new tasks
Sat 07 Dec 2019 12:57:51 AM PST | SETI@home | [sched_op] Server version 709
Sat 07 Dec 2019 12:57:51 AM PST | SETI@home | Project requested delay of 303 seconds
Sat 07 Dec 2019 12:57:51 AM PST | SETI@home | [sched_op] estimated total CPU task duration: 0 seconds
Sat 07 Dec 2019 12:57:51 AM PST | SETI@home | [sched_op] estimated total NVIDIA GPU task duration: 1743 seconds
~~
Sat 07 Dec 2019 12:57:51 AM PST | SETI@home | [work_fetch] backing off CPU 1400 sec
Sat 07 Dec 2019 12:57:51 AM PST | SETI@home | [sched_op] Deferring communication for 00:05:03

And still no cpu tasks.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

I have 553 WUs in progress, 192 are for the CPU, the rest are for the GPU.
My alottment is 100+100, so I have work galore and uploads are working fine (so far.....)
I've set NNT and will just crunch away.
Could have been worse !

---

Humans may rule the world...but bacteria run it...

Just tried backing the work cache down to .5 day with .5 additional, as suggested. We'll see if it has any effect.
At present, my Win box has 500 GPU tasks for 2x750tis, and the little Linux box has 190 cpu and 900ish gpu.
Guess we'll see if limits throttle it.
Meanwhile, the Big Boy is on a lean diet and can't get fed.

---

Just tried backing the work cache down to .5 day with .5 additional, as suggested. We'll see if it has any effect.

Better yet, try 0.02 for the additional setting (the larger the additional number, the less frequently it asks for work).

---

Grant
Darwin NT

Just tried backing the work cache down to .5 day with .5 additional, as suggested. We'll see if it has any effect.

Better yet, try 0.02 for the additional setting (the larger the additional number, the less frequently it asks for work).

Just waiting for a log file entry telling me I've reached a limit on tasks. Until then, all else seems irrelevant ... appears that the server isn't telling the client to quit asking for work as I think it normally does.

---

My three machines actually seem to have the amount of work actually requested. i.e my Linux machine has around 800 tasks and is set for 1 day, and 800 is about what it gets through.

One Windows machine has 139 GPU and 109 CPU, and the other has 77 GPU and 62 CPU again about correct for the cache I have set so for me there is no real problem

My three machines actually seem to have the amount of work actually requested. i.e my Linux machine has around 800 tasks and is set for 1 day, and 800 is about what it gets through.

One Windows machine has 139 GPU and 109 CPU, and the other has 77 GPU and 62 CPU again about correct for the cache I have set so for me there is no real problem

Yes, it is looking like the limits are being respected. Will look a bit weird when I get up, I suspect. Near as I can tell, my 980s tend to do about 1000 MBs per day each, so I could end up with 5000 and 7000 GPU tasks respectively for the Linux boxes. Fortunately each machine has about 100gb free disk space.
The Win box is no longer asking for GPU work, and has filled its CPU to a reasonable level.

---

Greetings,

I just set my older Linux PC to No New Tasks. This host should only get 200 WUs and it's got 600 in progress. Looks like something barfed all over the place. ;)

Have a great day! :)

Siran

---

CAPT Siran d'Vel'nahr - L L & P _\\//
Winders 11 OS? "What a piece of junk!" - L. Skywalker
"Logic is the cement of our civilization with which we ascend from chaos using reason as our guide." - T'Plana-hath

So, is everybody saying that "We've got what we wished for" - cache sizes controlled only by our own request preferences, not by a rigid server cap?

Use it wisely. I hope we don't come to regret this.

So, is everybody saying that "We've got what we wished for" - cache sizes controlled only by our own request preferences, not by a rigid server cap?

Use it wisely. I hope we don't come to regret this.

Hi Richard,

Are you saying that the powers-that-be undid the 100 max WUs?

It kinda doesn't make sense. My newer, faster Linux PC should have WAY more WUs than my older one then. It's got less than half what the older one has. I also noticed that the RTS is down under 100 WUs. I don't think the splitters are keeping up. ;)

Have a great day! :)

Siran

---

CAPT Siran d'Vel'nahr - L L & P _\\//
Winders 11 OS? "What a piece of junk!" - L. Skywalker
"Logic is the cement of our civilization with which we ascend from chaos using reason as our guide." - T'Plana-hath

I hope we don't come to regret this.

That's the million dollar question. Have we been given what we wished for, or is it just because something's broken?
If we have been given it, I would have thought it would be a case of gradually increasing the limits to make sure the system doesn't fall over in a screaming heap. To just remove them seems rather, risky. Especially considering how sluggish the system has been over the last few weeks (although now without the Haveland graphs we can't actually see that occurring anymore).

---

Grant
Darwin NT