Message boards :
Number crunching :
Panic Mode On (83) Server Problems?
Message board moderation
Previous · 1 . . . 11 · 12 · 13 · 14 · 15 · 16 · 17 . . . 21 · Next
Author | Message |
---|---|
Sutaru Tsureku Send message Joined: 6 Apr 07 Posts: 7105 Credit: 147,663,825 RAC: 5 |
It looks like no new AP WU since 3 days? Is there a reason why the AP splitters are: Not Running Program failed or ran out of work (or the project is down) Maybe the currently tapes have no useful AP data in it for crunching? * Best regards! :-) * Philip J. Fry, team seti.international founder. * Optimize your PC for higher RAC. * SETI@home needs your help. * |
Donald L. Johnson Send message Joined: 5 Aug 02 Posts: 8240 Credit: 14,654,533 RAC: 20 |
if you can't even get CPU work units (and I am having the same problem here) where are all the work units going. I thought it was the Scheduler that assigned the tasks to the appropriate resource, depending on what was requested by your BOINC. If both CPU and GPU tasks are requested, usually the GPU is the more efficient resource so that request gets filled first, then the CPU request, to the extent there are tasks available in the feeder. The one exception is VLARs are not sent to NVidia GPUs, since many of the older models do not handle those tasks very well, and the Scheduler has not been updated to make that distinction. Donald Infernal Optimist / Submariner, retired |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
As I understand it, the splitters are running at about the same rate as they were before the move, Nah, not even close. Before the move (and for a week or 2 after it) the splitters were able to produce 50+/s. That would allow them to provide work as well as build up a ready-to-send buffer. Once the buffer was full (around 300,000 WUs) they'd shut down. For the last couple of weeks, they've only been able to produce 30/s (which is barely enough when the work is predominately VLARs, put just a few shorties in there & it's no where near enough. Shorty storm- forget it). There was actually a spike there today of 60/s, but that was just a glitch. 30/s (or less) has been it for a couple of weeks now. Grant Darwin NT |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
Actually all kidding aside on this, if you can't even get CPU work units (and I am having the same problem here) where are all the work units going. The outbound pipe is running at twice the capacity of before yet many of us are starved for work. Anyone care to speculate or am I missing something obvious here? Ah. I'll go with Joe, it's just Seti traffic. Grant Darwin NT |
Donald L. Johnson Send message Joined: 5 Aug 02 Posts: 8240 Credit: 14,654,533 RAC: 20 |
It looks like no new AP WU since 3 days? Yes, per the Server Status Page, all the tapes available to the splitters have already been processed for AP tasks. Until more tapes are loaded, only AP resends will be available. Donald Infernal Optimist / Submariner, retired |
juan BFP Send message Joined: 16 Mar 07 Posts: 9786 Credit: 572,710,851 RAC: 3,799 |
Please Help! need some GPU WU to crunch, it´s cold here and the GPU´s are running empty... |
Donald L. Johnson Send message Joined: 5 Aug 02 Posts: 8240 Credit: 14,654,533 RAC: 20 |
As I understand it, the splitters are running at about the same rate as they were before the move, Okay, I haven't been paying that close attention to the production rate. Maybe that is one of the "throttles" Matt spoke of. Yes, I do wish he or Jeff or Eric would confirm that, so we can stop speculating and arguing about things we can't be certain about .... Donald Infernal Optimist / Submariner, retired |
Sutaru Tsureku Send message Joined: 6 Apr 07 Posts: 7105 Credit: 147,663,825 RAC: 5 |
Philip J. Fry wrote: It looks like no new AP WU since 3 days? Donald L. Johnson wrote: Yes, per the Server Status Page, all the tapes available to the splitters have already been processed for AP tasks. Until more tapes are loaded, only AP resends will be available. But it looks strange. Everytime I look to the server status page, the AP splitter are 'Not Running'. My BOINC don't get new AP WUs since 3 days. Normally every day ~ 5 AP WUs. * Best regards! :-) * Philip J. Fry, team seti.international founder. * Optimize your PC for higher RAC. * SETI@home needs your help. * |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
As I understand it, the splitters are running at about the same rate as they were before the move, Unfortunately the resolution of the long term graphs makes it impossible to see the spikes & troughs that used to be the splitter output (run till buffer full, then shut down). But what can be seen is the average turn around time. Can anyone rememeber when the server side limits came in? The average turn around time used to be 80-100 hours. Around Dec it dropped to around 50hrs. Lately it's been around 30hrs. In the past, those sustained 50+/s outputs from the splitters were only for a couple of hours, then the buffer would be full & they'd shut down. Not knowing the underlying mechanism for splitting work, it's possible the present sustained load is resulting in the splitter chocking (as someone posted previously there are issues with disk I/O limitations). But even allowing for that, to me it still doen't explain why we don't at least get several hours of high output after the outages. Grant Darwin NT |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
But it looks strange. You will only get work if work is being produced. For work to be produced the splitters have to be running. They can't run if there isn't any data for them to process. There isn't any data for them to process, so they're not running. Grant Darwin NT |
Sutaru Tsureku Send message Joined: 6 Apr 07 Posts: 7105 Credit: 147,663,825 RAC: 5 |
Philip J. Fry wrote: But it looks strange. Grant (SSSF) wrote: You will only get work if work is being produced. And why there isn't any data for them to process? ;-) The tapes don't have useful AP data in it since 3 days? * Best regards! :-) * Philip J. Fry, team seti.international founder. * Optimize your PC for higher RAC. * SETI@home needs your help. * |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
The tapes don't have useful AP data in it since 3 days? Nope. New ones have yet to be loaded. Grant Darwin NT |
Mike Send message Joined: 17 Feb 01 Posts: 34257 Credit: 79,922,639 RAC: 80 |
AP splitter 2 and 3 are running but all tapes are already marked as done. That means they was sent as soon they were beeing split. With each crime and every kindness we birth our future. |
Wiggo Send message Joined: 24 Jan 00 Posts: 34744 Credit: 261,360,520 RAC: 489 |
Philip J. Fry wrote:But it looks strange. Philip, for all the time that you have been here I thought that you would be able to read the Server Status page and there for understand the reason without having to ask. AP's are split off the same files that MB's are but the AP's are split off that much faster than the MB's are being that there are a lot less AP's coming from 1 file then there are MB's. If you had a look at the files to be split the last few days you would of noticed a lot of files there that had already finished splitting AP's but were still waiting to complete MB's splitting. Now that those files are almost finished now I expect that new files will start to be loaded again and AP's will be split very quickly again and will have to wait for the MB's to catch up again. Cheers. |
rob smith Send message Joined: 7 Mar 03 Posts: 22190 Credit: 416,307,556 RAC: 380 |
Each tape holds the same theoretical number of APs, but the AP splitters get through a tape much faster than the MB splitters for a number of reasons, including the split is simpler - there is substantially less overlap (non?), and the bandwidth per WU is much higher, (Which means there are fewer AP WU are split from each tape). The new MB splitters appear to much slower than the old ones - are they stress testing them? The last tape load was "quite substantial", and there haven't been any new tapes loaded for a few days - much the same happened last week, where the splitters were down to the very tail of the last tape before any new tapes were loaded. At least with the 24/7 support in the coloc there should be someone around to load the new tapes when the time is due (provided the project has sent a few down the hill). Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
Wiggo Send message Joined: 24 Jan 00 Posts: 34744 Credit: 261,360,520 RAC: 489 |
Personally I think that they could drop 2 AP splitters and move them over to MB splitting which would help balance the production of the 2 a bit better. Cheers. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
Personally I think that they could drop 2 AP splitters and move them over to MB splitting which would help balance the production of the 2 a bit better. Good idea. Grant Darwin NT |
Keith White Send message Joined: 29 May 99 Posts: 392 Credit: 13,035,233 RAC: 22 |
This last outage of mine, partially caused by me draining my queues to update to .64, and which finished after we the well went dry, when I did get units I got CPU topped off first and then GPU several hours later. Usually it's the other way around but I believe I know why it was in this case. I believe this happeded because 1) some units can only be done on CPU since the GPU apps have problems; 2) GPU units are in a very high demand due to the current trend of everyone using GPUs to crunch and 3) the reserve of units are gone and 30/s doesn't cut it. This leads to a situation where there may be CPU only units available while GPU is always in extreme short supply. As it stands now, I need an GPU unit only every 40-50 minutes on average based on the mix of long and short units, I am usually capped off at my 100. However if your needs are considerably higher, as in a few minutes, your GPU is likely to run low/out while CPU can still be filled by CPU only units. "Life is just nature's way of keeping meat fresh." - The Doctor |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
Yeah well, I can always switch to Beta if I see ice forming on my CPU and GPU :-) Isn't it meant to be almost summer there in the North? Grant Darwin NT |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13731 Credit: 208,696,464 RAC: 304 |
Yeah well, I can always switch to Beta if I see ice forming on my CPU and GPU :-) Anything below 25°c is very cold. Grant Darwin NT |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.