Panic Mode On (52) Server problems?

Message boards : Number crunching : Panic Mode On (52) Server problems?
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 . . . 10 · Next

AuthorMessage
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1140135 - Posted: 13 Aug 2011, 18:16:18 UTC

Both of my crunchers have no problems at all holding a 10-day cache. Of course they're both slow by everyone's standards, but when either of them need just a few seconds of work, 1-5 requests and they get a new task, or multiple tasks.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1140135 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1140139 - Posted: 13 Aug 2011, 18:33:50 UTC

I am hoping that the AP work in the splitter queue will dry up for a bit.
Then maybe the MB work will flow a bit better and fill some caches. I've only been monitoring my top rig, and it's cache is kinda floating up and down as downloads can complete and work is issued.

It has only been in recent history that I can recall the AP work being split faster than the MB work. It used to be that all of the MB work was split and sent, leaving only AP splitting being done. Quite the opposite at the moment.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1140139 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34258
Credit: 79,922,639
RAC: 80
Germany
Message 1140191 - Posted: 13 Aug 2011, 21:19:57 UTC

AP work isn´t split any faster Mark.
It gets downloaded slower for some reason.
I could manage to catch 1200 MB in less than 2 hours last night.

Its very weird downloaded 1 AP today with 650 KB/s.
The next one with 6 kb again.

Maybe a ghost WU.



With each crime and every kindness we birth our future.
ID: 1140191 · Report as offensive
Profile Dimly Lit Lightbulb 😀
Volunteer tester
Avatar

Send message
Joined: 30 Aug 08
Posts: 15399
Credit: 7,423,413
RAC: 1
United Kingdom
Message 1140210 - Posted: 13 Aug 2011, 22:18:32 UTC - in response to Message 1140135.  

Both of my crunchers have no problems at all holding a 10-day cache. Of course they're both slow by everyone's standards, but when either of them need just a few seconds of work, 1-5 requests and they get a new task, or multiple tasks.

Hehehe, even slower here and no problems :).
ID: 1140210 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1140223 - Posted: 13 Aug 2011, 22:46:10 UTC - in response to Message 1140025.  

The faster machines having smaller caches would reduce the server load considerably


No, they will ask for the same amount of work, no matter how big the cache is.

No they won't.
Depending on the speed of a machine a 2 day cache may require 2,000 Work Units, so a 5 day cache will require 5,0000 Work Units. 10,000 Work Units for a 10 day cache, etc.
Once a cache is full, then the only requests will be to top up the cache & will be made at the rate the machine processes work. But it has to build up the cache in the first place before point is reached. So while building up the cache the demand for work is above & beyond what would be required once the cache has filled up.

My machines are a good example of that.
When the caches aren't full, they will try to get work every 5 minutes. When the caches are full, it's 1 or 2 reqeusts every 30 min to 2 hours (assuming each request for work is successfull). That's a big difference in demand.
Grant
Darwin NT
ID: 1140223 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1140226 - Posted: 13 Aug 2011, 22:52:31 UTC - in response to Message 1140191.  

Its very weird downloaded 1 AP today with 650 KB/s.
The next one with 6 kb again.

Maybe a ghost WU.

I think there are still problems with one of the download servers.
For several weeks now downloads have been very variable- some will timeout as soon as they start, or they start- but then it just sit there & nothing is downloaded & it might take 5 minutes or more for it to time out.
Disabling & re-enabling network access & a few retries & then you'll get a bunch all downloading- maybe at 5kB/s within a day of the outage but often at 15kB/s or more a couple of days or more after the outage.
And the then next attempt to download will time out straight away again or not start to download anything at all...
Grant
Darwin NT
ID: 1140226 · Report as offensive
SupeRNovA
Volunteer tester
Avatar

Send message
Joined: 25 Oct 04
Posts: 131
Credit: 12,741,814
RAC: 0
Bulgaria
Message 1140266 - Posted: 14 Aug 2011, 1:32:54 UTC - in response to Message 1140089.  

Someone else might like to comment, but your cruncher with the two GPU has had more than 10 errors in the last 24 hours, compared to others with similar setups that are giving less than one error per day.

Anyway, the blockage appears to have been cleared as there are quite a number of WU (mostly AP for your GPUs).

yep that is why because the nvidia drivers stop working because of high OC of the CPU it was for 1 min before i fix it.Just make 3.7Ghz on Q6600 B3 1.325 VID :D keep pushing to 4Ghz :D
just now i'm writing have download 100 astropulse units
ID: 1140266 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14653
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1140358 - Posted: 14 Aug 2011, 9:43:46 UTC - in response to Message 1140139.  

I am hoping that the AP work in the splitter queue will dry up for a bit.
Then maybe the MB work will flow a bit better and fill some caches. I've only been monitoring my top rig, and it's cache is kinda floating up and down as downloads can complete and work is issued.

It has only been in recent history that I can recall the AP work being split faster than the MB work. It used to be that all of the MB work was split and sent, leaving only AP splitting being done. Quite the opposite at the moment.

It looks like your wish has been granted ;-)

In fact, I think it's been the case for a while that the AP splitters, working flat out, can turn a 'tape' into WUs a lot faster than the MB splitters can. But they don't always work flat out. There are safeguards in place to avoid them over-filling the WU data storage when, for example, the ready-to-send queue gets big enough.

Now that AP crunching on ATI has reached the mainstream, and more people are beta-testing the NVidia AP app, WUs are collected from that queue more quickly, and there is less need to inhibit the splitters.

Personally, I'd prefer to see fewer AP splitters in play. That would spread out the deployment of AP tasks from new tapes, rather than the current 'boom and bust', with 17 full tapes of AP work already clogging the download pipe but MB struggling to play catch-up.
ID: 1140358 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1140367 - Posted: 14 Aug 2011, 10:35:50 UTC - in response to Message 1140358.  
Last modified: 14 Aug 2011, 10:38:45 UTC

Thanks Richard....
You understood what I was getting at.
About 1000 AP left in ready to send.
We'll see what shakes if another dataset does not load and start AP splitting back up again.
If the MB work does not have to fight the AP work for bandwidth, maybe downloads will start doing a bit better on the MB work.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1140367 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34258
Credit: 79,922,639
RAC: 80
Germany
Message 1140383 - Posted: 14 Aug 2011, 13:07:52 UTC

We AP crunchers had to wait up to 4 weeks to get a few.

Had no problems myself to fill my MB cache up to 14 days.



With each crime and every kindness we birth our future.
ID: 1140383 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 1140621 - Posted: 15 Aug 2011, 0:54:12 UTC

I still have 72 of them on my other machine, most of them came down in one bunch.

ID: 1140621 · Report as offensive
Profile Paul D Harris
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1122
Credit: 33,600,005
RAC: 0
United States
Message 1140684 - Posted: 15 Aug 2011, 3:54:57 UTC

I got another 160 ap video card wu's but they are low on my cache since I am not doing them right now it took about 2 days to get them I also got several hundred other gpu and cpu wu's also all in the 2 days over 1000 in all wu's.
ID: 1140684 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34846
Credit: 261,360,520
RAC: 489
Australia
Message 1140700 - Posted: 15 Aug 2011, 5:09:08 UTC - in response to Message 1140684.  

Well you guys and girls have had 12 days of me only feeding 2 of my 3 PC's but...

my X4 630 has now been handed down to my youngest son and has been replaced by an i5 2500K.

Plus its in a feeding frenzy so, if you can't get any work then blame me. :D

Cheers.
ID: 1140700 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1140701 - Posted: 15 Aug 2011, 5:24:39 UTC - in response to Message 1140700.  


Looks like we're about to run out of MB work again- the Ready to Send buffer has been shrinking steadily for several hours now & it would appear the splitter problem has resurfaced- they're just not cranking up the production to meet demand.
Grant
Darwin NT
ID: 1140701 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1140734 - Posted: 15 Aug 2011, 9:17:26 UTC - in response to Message 1140701.  


Untill the splitters get their act together, that's it for any more MB work apart from the odd WU here & there.
Grant
Darwin NT
ID: 1140734 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14653
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1140738 - Posted: 15 Aug 2011, 9:29:06 UTC - in response to Message 1140734.  

Untill the splitters get their act together, that's it for any more MB work apart from the odd WU here & there.

A creation rate of 18/sec can't all be resends, can it? So, some splitter action, and some requests will get something - just not all of them, and not enough to satisfy demand. We may just limp through until the lab opens ;-)
ID: 1140738 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13746
Credit: 208,696,464
RAC: 304
Australia
Message 1140740 - Posted: 15 Aug 2011, 9:33:54 UTC - in response to Message 1140738.  

Untill the splitters get their act together, that's it for any more MB work apart from the odd WU here & there.

A creation rate of 18/sec can't all be resends,

Nope, they're still splitting, but for whatever reason they're just not cranking it up as required.
Grant
Darwin NT
ID: 1140740 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34258
Credit: 79,922,639
RAC: 80
Germany
Message 1140741 - Posted: 15 Aug 2011, 9:37:48 UTC


Ready to send went down from 226000 10 hours ago to zero.
Results out in the field increased from 5,4 to 5,8 million.
So it seems some are catching up nicely.



With each crime and every kindness we birth our future.
ID: 1140741 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1140850 - Posted: 15 Aug 2011, 18:24:33 UTC

Hopefully a good sign...
MB splitting rate now shows almost 40/second and the Cricket has sprung back to life.

Meow meow meow.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1140850 · Report as offensive
Profile KWSN Ekky Ekky Ekky
Avatar

Send message
Joined: 25 May 99
Posts: 944
Credit: 52,956,491
RAC: 67
United Kingdom
Message 1141171 - Posted: 16 Aug 2011, 11:17:51 UTC - in response to Message 1141149.  
Last modified: 16 Aug 2011, 11:18:18 UTC

WARNING WARNING!! The system works great now, let's take it down quick and destroy it :-)


I think you got what you asked for - nothing getting reported at present! Presumably they also found ET and don't want any more results?
Actually I see cricket's "bits out" have dropped again, so presumably that is what has happened.

ID: 1141171 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 . . . 10 · Next

Message boards : Number crunching : Panic Mode On (52) Server problems?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.