Message boards :
Number crunching :
Panic Mode On (104) Server Problems?
Message board moderation
Previous · 1 . . . 10 · 11 · 12 · 13 · 14 · 15 · 16 . . . 42 · Next
Author | Message |
---|---|
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
Server Status page shows everything's good, but the cache on my main cruncher is running down again. Most requests for work result in "Project has no tasks available" messages and those that do get work don't get enough to make up the developing deficit. Grant Darwin NT |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
Server Status page shows everything's good, but the cache on my main cruncher is running down again. Most requests for work result in "Project has no tasks available" messages and those that do get work don't get enough to make up the developing deficit. Not sure what the problem could be there. I am just a few WUs short of my 2600 cache limit across my 8 rigs. There are still a lot of the Aerecibo shorties out in the field, so lots of work requests to fill. I suspect it's just the luck of the draw to hit the scheduler when it has a full bucket. "Time is simply the mechanism that keeps everything from happening all at once." |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
Hmm. Results received in the last hour is at 111,780. At the height of the Arecibo only work I think I noticed it hit around 15,250/hr- it could be that all the systems that ran out of work now have some, so their Manager backoffs are no longer stopping them from requesting work, hence increasing the demand on the server even though the amount of work being returned has dropped off. It could be we're looking at another system limit. The project wants more crunchers to process the work, but if it can't continue to support the current load that's just going to lead to more unhappy crunchers that won't hang around and head for greener pastures (ie more work, and more credit). Time for another Feeder, or increase the size of the present one (if it's possible)? Or is it just a case of something else on Synergy soaking up CPU cycles & inhibiting it's output? EDIT- just looked at my Event Log and where Scheduler requests usually get a response in 3-4 seconds, at the moment it can take as long as 10 seconds, and has been that way for at least the last 8 hours (as far back as my Event Log goes). Grant Darwin NT |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
The other thing that sometimes happens is the work being split creates almost all VLAR tasks, which are suppressed from being sent to nvidia GPUs. So the ready to send cache can be full, but will not be sent to those cards. "Time is simply the mechanism that keeps everything from happening all at once." |
Wiggo Send message Joined: 24 Jan 00 Posts: 36556 Credit: 261,360,520 RAC: 489 |
The other thing that sometimes happens is the work being split creates almost all VLAR tasks, which are suppressed from being sent to nvidia GPUs. I'll amend that to, "Arecibo VLAR's won't be sent to GPU's", as GBT VLAR's are sent to GPU's. ;-) Cheers. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
The other thing that sometimes happens is the work being split creates almost all VLAR tasks, which are suppressed from being sent to nvidia GPUs. Yep. But it's been almost a day or so since I got an Arecibo VLAR on my CPU. Since my initial post, my system picked up enough work to fill the cache, but then the next 6 requests for work have resulted in none so it's not as full as it was... I've done a speed test on my net connection, and it's not as healthy as it should be- uploads are actually faster than downloads at the moment and there's quite a bit of latency & jitter which doesn't help things. Grant Darwin NT |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
The other thing that sometimes happens is the work being split creates almost all VLAR tasks, which are suppressed from being sent to nvidia GPUs. That would be correct. Thanks. "Time is simply the mechanism that keeps everything from happening all at once." |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
Really struggling to get work here. 15/01/2017 14:43:25 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 14:43:25 | SETI@home | Reporting 3 completed tasks 15/01/2017 14:43:25 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 14:43:32 | SETI@home | Scheduler request completed: got 6 new tasks 15/01/2017 14:48:38 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 14:48:38 | SETI@home | Reporting 2 completed tasks 15/01/2017 14:48:38 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 14:48:46 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 14:48:46 | SETI@home | Project has no tasks available 15/01/2017 14:53:51 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 14:53:51 | SETI@home | Reporting 2 completed tasks 15/01/2017 14:53:51 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 14:53:59 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 14:53:59 | SETI@home | Project has no tasks available 15/01/2017 14:59:05 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 14:59:05 | SETI@home | Reporting 2 completed tasks 15/01/2017 14:59:05 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 14:59:14 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 14:59:14 | SETI@home | Project has no tasks available 15/01/2017 15:04:20 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:04:20 | SETI@home | Reporting 9 completed tasks 15/01/2017 15:04:20 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:04:27 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:04:27 | SETI@home | Project has no tasks available 15/01/2017 15:09:32 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:09:32 | SETI@home | Reporting 2 completed tasks 15/01/2017 15:09:32 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:09:40 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:09:40 | SETI@home | Project has no tasks available 15/01/2017 15:14:46 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:14:46 | SETI@home | Reporting 3 completed tasks 15/01/2017 15:14:46 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:14:56 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:14:56 | SETI@home | Project has no tasks available 15/01/2017 15:20:01 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:20:01 | SETI@home | Reporting 3 completed tasks 15/01/2017 15:20:01 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:20:08 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:20:08 | SETI@home | Project has no tasks available 15/01/2017 15:25:14 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:25:14 | SETI@home | Reporting 3 completed tasks 15/01/2017 15:25:14 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:25:22 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:25:22 | SETI@home | Project has no tasks available 15/01/2017 15:30:27 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:30:27 | SETI@home | Reporting 1 completed tasks 15/01/2017 15:30:27 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:30:33 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:30:33 | SETI@home | Project has no tasks available 15/01/2017 15:35:38 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:35:38 | SETI@home | Reporting 3 completed tasks 15/01/2017 15:35:38 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:35:45 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:35:45 | SETI@home | Project has no tasks available 15/01/2017 15:40:51 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:40:51 | SETI@home | Reporting 3 completed tasks 15/01/2017 15:40:51 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:40:59 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:40:59 | SETI@home | Project has no tasks available 15/01/2017 15:46:04 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:46:04 | SETI@home | Reporting 3 completed tasks 15/01/2017 15:46:04 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:46:13 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:46:13 | SETI@home | Project has no tasks available 15/01/2017 15:51:19 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:51:19 | SETI@home | Reporting 4 completed tasks 15/01/2017 15:51:19 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:51:27 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:51:27 | SETI@home | Project has no tasks available 15/01/2017 15:56:33 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 15:56:33 | SETI@home | Reporting 2 completed tasks 15/01/2017 15:56:33 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 15:56:41 | SETI@home | Scheduler request completed: got 0 new tasks 15/01/2017 15:56:41 | SETI@home | Project has no tasks available Finally! 15/01/2017 16:01:51 | SETI@home | Sending scheduler request: To fetch work. 15/01/2017 16:01:51 | SETI@home | Reporting 4 completed tasks 15/01/2017 16:01:51 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU 15/01/2017 16:01:55 | SETI@home | Scheduler request completed: got 51 new tasks Still not enough to fill the cache, but less likely to run out in the very near future. Grant Darwin NT |
Wiggo Send message Joined: 24 Jan 00 Posts: 36556 Credit: 261,360,520 RAC: 489 |
I wonder if it's just bad timing or your BOINC version as I've been having no such issues here Grant. Cheers. |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
I wonder if it's just bad timing or your BOINC version as I've been having no such issues here Grant. . . FWIW I am running 7.6.33 and while I was having similar problems earlier right now it seems to be OK. Stephen :) [edit] touch wood |
Wiggo Send message Joined: 24 Jan 00 Posts: 36556 Credit: 261,360,520 RAC: 489 |
I wonder if it's just bad timing or your BOINC version as I've been having no such issues here Grant. It could very well be those excessive back off times that later versions than mine have causing your problems. Between that, replacing the Messages tab with a Notice tab and the flakiness of many later versions I decided to stick with what works best for me (I hate being a guinea pig). [edit] It'll take at least the first 2 fixed and many good reports on the 3rd to convince me to think about upgrading. Cheers. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
It could very well be those excessive back off times that later versions than mine have causing your problems. Not backoff related as I haven't (yet) run out of work. But for some reason I ask for work, and there is none available. The last hour or so it hasn't taken nearly as many requests to finally get work, but it's certainly not available with every request the way it usually is. Normally I can report 2, get 2. Report 5, get 5. Report 1, get 1. Grant Darwin NT |
Wiggo Send message Joined: 24 Jan 00 Posts: 36556 Credit: 261,360,520 RAC: 489 |
It could very well be those excessive back off times that later versions than mine have causing your problems. Well I can't think of any other reason Grant, as I just havn't suffered depleted caches except for the regular outrages here. Cheers. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
Well I can't think of any other reason Grant, as I just havn't suffered depleted caches except for the regular outrages here. Yeah. It's odd. My download speed is way off what it should be, and jitter & latency way up, but with those sort of issues if they cause a problem usually result in Scheduler timeouts, not just slightly longer than usual waits for a response, and the response being "No work available." Grant Darwin NT |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
I wonder if it's just bad timing or your BOINC version as I've been having no such issues here Grant. . . Again, FWIW I can't say I was aware of any problems with 3.6.22, but 3.6.33 is maybe a little flaky at times, though I can't put my finger on it :) Stephen :) |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
Grant, it's nothing to worry about, the scheduler is 1). Just full of vlar's at that time, 2). I was just there 2 seconds before you and emptied it :) |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
Grant, it's nothing to worry about, the scheduler is I haven't had any Arecibo VLARs today at all. Even in Arecibo VLAR storms it hasn't taken more than about 6 requests to get work. 2). I was just there 2 seconds before you and emptied it :) For the last 1hr 30min? I've got 2 WUs in that time. Before that it was just over an hour before getting any WUs. Grant Darwin NT |
Brent Norman Send message Joined: 1 Dec 99 Posts: 2786 Credit: 685,657,289 RAC: 835 |
yea, about and hour ago I was 1/2 empty, then poof, 111 tasks, now all is fine. It's just luck of the draw when I need 3-9 per request. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13841 Credit: 208,696,464 RAC: 304 |
yea, about and hour ago I was 1/2 empty, then poof, 111 tasks, now all is fine. It's just luck of the draw when I need 3-9 per request. At least i'm not the only one having problems getting regular work. Grant Darwin NT |
Bruce Send message Joined: 15 Mar 02 Posts: 123 Credit: 124,955,234 RAC: 11 |
yea, about and hour ago I was 1/2 empty, then poof, 111 tasks, now all is fine. It's just luck of the draw when I need 3-9 per request. Grant you are not the only one having trouble getting work! I'm down to about 30% of work units in my cache, that is only about a couple of hours worth. I keep getting the same as you. 0 tasks sent project has no tasks available. Even if there is a flood of Arecibo VLARs, where is all of the GBT work? The RTS says that it is full. I don't get it. According to SSP it doesn't look like there should be a problem. Bruce |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.