Message boards :
Number crunching :
Panic Mode On (110) Server Problems?
Message board moderation
Previous · 1 . . . 30 · 31 · 32 · 33 · 34 · 35 · 36 . . . 37 · Next
Author | Message |
---|---|
kittyman Send message Joined: 9 Jul 00 Posts: 51478 Credit: 1,018,363,574 RAC: 1,004 |
Hmmm....the kitties are detecting a bit of a lack of nVidia GPU work for the servers to send. All 6 of my rigs have been slowly losing cache. Getting some here and there. Just a heads up from the kitty crew. Meow. "Time is simply the mechanism that keeps everything from happening all at once." |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Woke this morning to find all machines severely down on gpu work. It appears that the old problem of only Arecibo tasks coming out of the scheduler has appeared and thus a shortage sent to Nvidia machines that only pick up a few Arecibo non-VLAR's. Don't see a single BLC task in the last 4 hours in any of my machines Event Logs. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
kittyman Send message Joined: 9 Jul 00 Posts: 51478 Credit: 1,018,363,574 RAC: 1,004 |
That's what I was seeing. All Arecibo or nothing at all. I did just get a hit on this rig of some 30 tasks and it was almost all BLC work. So maybe the Arecibo splitting is almost over and caches will start to recover. "Time is simply the mechanism that keeps everything from happening all at once." |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
I just receive a lot of blc01 WU. What is strange are the temps on the GPU when crunch this WU, even the GPU usage still at almost 98-100% the temps drops about 5 C from the normal range 48-55C now at 43-51C only. Only blc01 are crunching on the GPUs at this time. CPUs temp are normal. |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Yes, I have received a mix of Arecibo/Green Banks tasks now and the caches are refilling. Don't know whether we just hit a pocket of exclusive Arecibo in the RTS buffer or whether the elves were in the lab this morning to massage the scheduler. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Still having issues getting full caches. Seeing a lot of "This computer has reached a limit on tasks in progress" messages. I know it has to with the Project Preferences settings. Started getting that when I set the toggle for AP work and now there is no AP work, it is preventing getting MB work. Can't seem to get the schedulers to ignore that request even though I have tried the settings update and Triple Update tricks. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
Try.... Run only the selected applications AstroPulse v7: yes SETI@home v8: yes If no work for selected applications is available, accept work from other applications? no works here normally. |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
That's what I normally use. When the issue crops up for me I have to keep toggling the accept other work off and on to wake up the schedulers. Or turn off AP work. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
I see that the Haveland site renewed their security certificate. It was expired yesterday and the site wouldn't load. I see that others must be having issues getting work too as the Tasks in Progress is steadily dropping. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
Keith Myers Send message Joined: 29 Apr 01 Posts: 13164 Credit: 1,160,866,277 RAC: 1,873 |
Caches dropping again. Nothing but Arecibo work. Seti@Home classic workunits:20,676 CPU time:74,226 hours A proud member of the OFA (Old Farts Association) |
kittyman Send message Joined: 9 Jul 00 Posts: 51478 Credit: 1,018,363,574 RAC: 1,004 |
Well, there's no more splitting as of right now. This rig actually ran out of GPU work. "Time is simply the mechanism that keeps everything from happening all at once." |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
The request for new works end on: Sun 18 Mar 2018 04:41:47 PM EST | SETI@home | This computer has reached a limit on tasks in progress The same thing happens the last week. Seems like a pattern when the Arecibo tapes are done. On my host: Nvidia WU Max tasks per day 2190 Number of tasks today 851 CPU WU Max tasks per day 734 Number of tasks today 36 So makes no sense the msg. |
JohnDK Send message Joined: 28 May 00 Posts: 1222 Credit: 451,243,443 RAC: 1,127 |
I´ve seen that message many times over time and not just now. |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
I´ve seen that message many times over time and not just now. Could be but that is not normal in a host like mine who crunch 1000's of WU per day with almost no WU with error. The next call for works picks 54 Arecibo WU (no blc?), in the following one the msg returns. Something is happening on the server side. |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
Hmmm....the kitties are detecting a bit of a lack of nVidia GPU work for the servers to send. . . It seems to be another Arecibo VLAR logjam stuffing up the schedulers. I am getting 100% Arecibo VLAR's on my CPUs and having trouble getting work for the GPUs. When I do it is Arecibo work and that is despite the fact the Arecibo tapes finished splitting a while ago. It seems the splitters are hung again on the Greenbank tapes as the number of channels left to do is not decreasing very fast. Stephen ? ? |
Stephen "Heretic" Send message Joined: 20 Sep 12 Posts: 5557 Credit: 192,787,363 RAC: 628 |
Still having issues getting full caches. Seeing a lot of "This computer has reached a limit on tasks in progress" messages. I know it has to with the Project Preferences settings. Started getting that when I set the toggle for AP work and now there is no AP work, it is preventing getting MB work. Can't seem to get the schedulers to ignore that request even though I have tried the settings update and Triple Update tricks. . . I don't think that is the culprit in this case. Because "Bertie" can no longer run the AP app I have turned off AP in preferences so I should not be requesting any of those but I am having the same symptoms as you. GPU cache is 47/100 but scheduler response is "you have reached your limit". I give the schedulers a kick and then I get some work but always less than a full allotment. Aint life grand :) Stephen ? ? |
Wiggo Send message Joined: 24 Jan 00 Posts: 36870 Credit: 261,360,520 RAC: 489 |
At least you know that your CPU caches are full. ;-) Cheers. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14679 Credit: 200,643,578 RAC: 874 |
If your CPU caches are full before your GPU caches, you're probably asking for too much work. |
Wiggo Send message Joined: 24 Jan 00 Posts: 36870 Credit: 261,360,520 RAC: 489 |
Nah, they're just burning through their GPU caches too fast Richard. :-D Cheers. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14679 Credit: 200,643,578 RAC: 874 |
Yeah, there are both VLAR Arecibo tasks and VHAR tasks in the batch that has just finished splitting. They do mess with your cache calculations. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.