The 400 and 50 WU limits are way too small

Message boards : Number crunching : The 400 and 50 WU limits are way too small
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · Next

AuthorMessage
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1197743 - Posted: 20 Feb 2012, 13:58:24 UTC - in response to Message 1197731.  
Last modified: 20 Feb 2012, 14:12:31 UTC

I just wish the GPU limit was a little higher than 400, though... Shorties are a killer. :(


What i do is Bump the 'Maintain enough tasks to keep busy for at least --- days' setting up high enough so shorties are processed first, (or 'connect about every' setting in older Boinc Managers)
with some suspending of CPU shorties, so too many don't run at once, i then restore the previous settings once the GPU shorties are complete, that has enabled me to build quite a few days cache up,
it also helps if you can do Astropulse on CPU and GPUs, each Wu will fill your cache up just a bit more than a MB Wu does.

(It also helps that i have a ATI GPU fitted, this doubles my GPU limit, so i get more Wu's for my Nvidia GPU, and a few AP Wu's for the ATI GPU, which then goes and does work elsewhere most of the time)

Claggy
ID: 1197743 · Report as offensive
Profile red-ray
Avatar

Send message
Joined: 24 Jun 99
Posts: 308
Credit: 9,029,848
RAC: 0
United Kingdom
Message 1197793 - Posted: 20 Feb 2012, 15:45:23 UTC - in response to Message 1197743.  

Wow, I just got a CPU WU, that makes a total of 3 in 24 hours!

20/02/2012 15:30:21 | SETI@home | Reporting 1 completed tasks, requesting new tasks for CPU
20/02/2012 15:30:25 | SETI@home | Scheduler request completed: got 1 new tasks

but I noticed the system is not even asking for GPU WUs. After some digging I found this is down to an "NVIDIA GPU work fetch deferral interval" of 00:40:00. I guess this is great for the servers but means once a client has run dry the chance of new WUs is even less. Is this another 6.12 "feature" I wonder. Pressing [Update] seems to make it ask so I feel a .BAT file is needed.
ID: 1197793 · Report as offensive
Profile red-ray
Avatar

Send message
Joined: 24 Jun 99
Posts: 308
Credit: 9,029,848
RAC: 0
United Kingdom
Message 1197825 - Posted: 20 Feb 2012, 16:53:23 UTC - in response to Message 1197793.  
Last modified: 20 Feb 2012, 16:53:55 UTC

Lookes like my .BAT file did the trick!

20/02/2012 16:49:14 | SETI@home | Sending scheduler request: Requested by user.
20/02/2012 16:49:14 | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
20/02/2012 16:49:35 | SETI@home | Scheduler request completed: got 33 new tasks
ID: 1197825 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1197831 - Posted: 20 Feb 2012, 17:03:43 UTC
Last modified: 20 Feb 2012, 17:04:51 UTC

Eric and I have spoken about this.
If he gets the AP validation algorithms corrected and successfully tested on Beta, the limits may be raised. Not before then. DA's attempt at 'fixing' certain other 'problems' skewed things way off kilter.......so far as to not allow the usual workfetch to work properly. Boinc properties of DCF and WTF have not worked properly since he screwed with them.

It has taken much more time to recover from this than expected. But, it shall happen.


It's particularly the WTF factor that I am waiting to see fixed......LOL.

Best wishes and kitty kisses, DA.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1197831 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1197925 - Posted: 20 Feb 2012, 18:53:13 UTC

"The 400 and 50 WU limits are way too small" for some computers, way too big for most. We need BOINC to provide a time-based limits option, restricting each host to about 2 days of work "in progress" would be appropriate for this project currently.
                                                                  Joe


ID: 1197925 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1197933 - Posted: 20 Feb 2012, 19:08:09 UTC - in response to Message 1197727.  

The idea for a super host has been around for some time: http://boinc.berkeley.edu/trac/wiki/SuperHost.
Most of the time people want to do this for a machine to proxy all of their other hosts through.

I like the limits for the faster turn around time, but dislike them the few times my machines have run out of work. Then going over their their backup project. Which is always when I am trying to do some long term benchmarking to check out configuration changes.


i am too new to have heard this superhost discussion. thx for the info.

take your situation , Hal, you have .... 32 PC ? (unless you didnt merged them ^^). all your 32 PCs are asking 32 times every 5 mins for work.
i dont need to type in here 32x times "Gimme work!". instead of that you would have only 1 asking work, and would get more constancy at each time you ask it.
i hope you dont need to go all the floor and check 32 PC in 32 different room ^^

imagine 1,000 persons like you => 32,000 pc asking for work instead of just 1,000 servers. thats stress.

Is it right when we ask for work, we are downloading them from a Server that has only a buffer of 1000WU only at the time ?
even if on the server page it s written 500,000 WU ready to sent, if that 1000WU buffer server is empty, you get : "sorry, the project has no work available!" ?

I am well aware of how my machines act. That is why I have removed some of them the past few years. So I wouldn't be pounding the servers even more than I am now. I take advantage of those fancy network tools people have made to manage everything. Until we got some new network equipment I did have to get up and walk to the other side of the room to check on a few of the boxes, but now I can do it all without even rolling my chair 2 feet.

It is nice to be able to see all of my machines at once. So I can see how the limits are effecting their queue. My faster ones end up with just a few days work. While the slower boxes hit their queue limit before the task limit.

The least I heard the feeder held 100 tasks and refilled once a minute.

SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1197933 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1197937 - Posted: 20 Feb 2012, 19:11:42 UTC - in response to Message 1197925.  

"The 400 and 50 WU limits are way too small" for some computers, way too big for most. We need BOINC to provide a time-based limits option, restricting each host to about 2 days of work "in progress" would be appropriate for this project currently.
                                                                  Joe


Provided that the project servers - which is where such a limit would have to be applied - have an accurate estimate of the runtime of tasks cached. At the moment, the servers ignore DCF entirely when calculating their own idea of how long newly-allocated work will run during work allocation, and the well-rehearsed APR capping problem means that hosts are running some extreme DCFs, far from the 1.0000 taken for granted by CreditNew. (I've got a 0.0691)
ID: 1197937 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1197941 - Posted: 20 Feb 2012, 19:16:42 UTC - in response to Message 1197933.  

The least I heard the feeder held 100 tasks and refilled once a minute.

I thought it was 100 tasks too, but it has to be faster than once per minute. That would limit the project to 6,000 tasks per hour - we know the regular throughput is far, far higher than that. The feeder refresh interval will be every second or two.
ID: 1197941 · Report as offensive
Profile SciManStev Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Jun 99
Posts: 6652
Credit: 121,090,076
RAC: 0
United States
Message 1198058 - Posted: 21 Feb 2012, 0:47:59 UTC - in response to Message 1197925.  

"The 400 and 50 WU limits are way too small" for some computers, way too big for most. We need BOINC to provide a time-based limits option, restricting each host to about 2 days of work "in progress" would be appropriate for this project currently.
                                                                  Joe



That was well spoken. In my situation I would like to see 2 days of CPU work, and two days of GPU work if that were possible, which still amounts to 2 days of work. That would be a huge improvement over current conditions. The next problem is just to get back to the limits we currently have. That alone can be no easy task, especially durinig a shortie storm.

Steve
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website
ID: 1198058 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1198068 - Posted: 21 Feb 2012, 1:19:14 UTC - in response to Message 1197941.  

The least I heard the feeder held 100 tasks and refilled once a minute.

I thought it was 100 tasks too, but it has to be faster than once per minute. That would limit the project to 6,000 tasks per hour - we know the regular throughput is far, far higher than that. The feeder refresh interval will be every second or two.

Perhaps I was just remembering "minute" instead of "second". As they say the memory is the 2nd thing to go, but no one can remember the first.

Perhaps a variable "Max allocated tasks" like the "Max tasks per day" thing would be the best solution.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1198068 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 1198075 - Posted: 21 Feb 2012, 2:10:58 UTC - in response to Message 1197941.  

The least I heard the feeder held 100 tasks and refilled once a minute.

I thought it was 100 tasks too, but it has to be faster than once per minute. That would limit the project to 6,000 tasks per hour - we know the regular throughput is far, far higher than that. The feeder refresh interval will be every second or two.

As of December 2010 the -sleep_interval command line value for the Feeder was 3 seconds. But that sleep time is only used if nothing was done during a pass through the array. The default array size is 100, and the default query to the BOINC database to enumerate the next tasks is sized 200. I think the 100 matches observed effects here, but the query size could be larger than default.

The "nothing was done" could be because no requests were successful since the last Feeder pass or because there were no tasks ready to send. My guess is the usual pattern is the Feeder makes two or three passes through the array, doing something on the first, very little or nothing on the second, etc. That's assuming the database enumeration happens quickly each time, though.
                                                                  Joe
ID: 1198075 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1198081 - Posted: 21 Feb 2012, 2:30:07 UTC
Last modified: 21 Feb 2012, 2:37:07 UTC

I am wondering if the GPU limits could be raised, like, right NOW.
What I understand is that the current holdup is fixing the AP validator.

As no stock app exists for AP on GPUs.....and whack me if I am wrong...why can't the GPU limits be raised right now without much risk of damage? Leave the CPU limits where they are at if you must. It's the GPUs on the power hosts that are what is starved when anything goes slightly south with the servers.

EDIT... I just emailed Eric.....we'll see what he thinks.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1198081 · Report as offensive
Lionel

Send message
Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Australia
Message 1198101 - Posted: 21 Feb 2012, 3:25:13 UTC - in response to Message 1198081.  


it is rare for me to get up to the limits at any time ... in fact it is hard to get work at any time (wiggo keeps sucking the pipe dry) ... they need to be lifted otherwise we are all going to be knocking on the door all the time ...






ID: 1198101 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1198105 - Posted: 21 Feb 2012, 3:33:13 UTC - in response to Message 1198101.  


it is rare for me to get up to the limits at any time ... in fact it is hard to get work at any time (wiggo keeps sucking the pipe dry) ... they need to be lifted otherwise we are all going to be knocking on the door all the time ...


For now, I just hope the kitties can gather enough kibble overnight to run through tomorrow's outage.

And hope they can get the deleter process back in order. I dunno where they are getting the disc space right now.


"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1198105 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1198128 - Posted: 21 Feb 2012, 4:35:03 UTC - in response to Message 1198127.  

me i m getting
2/20/2012 11:09:49 PM SETI@home Message from server: This computer has reached a limit on tasks in progress

I am nowhere near that, but the kitties have been gathering some kibbles into their bowls!!
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1198128 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1198135 - Posted: 21 Feb 2012, 5:04:08 UTC - in response to Message 1198127.  

it is rare for me to get up to the limits at any time ... in fact it is hard to get work at any time (wiggo keeps sucking the pipe dry) ... they need to be lifted otherwise we are all going to be knocking on the door all the time ...

It really isn't my fault if I'm closer to the point where those work units land in this country. ;)

me i m getting
2/20/2012 11:09:49 PM SETI@home Message from server: This computer has reached a limit on tasks in progress

Yes I'm getting very sick of seeing that message all the time. :(

Cheers.
ID: 1198135 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1198137 - Posted: 21 Feb 2012, 5:08:05 UTC - in response to Message 1198135.  

it is rare for me to get up to the limits at any time ... in fact it is hard to get work at any time (wiggo keeps sucking the pipe dry) ... they need to be lifted otherwise we are all going to be knocking on the door all the time ...

It really isn't my fault if I'm closer to the point where those work units land in this country. ;)

me i m getting
2/20/2012 11:09:49 PM SETI@home Message from server: This computer has reached a limit on tasks in progress

Yes I'm getting very sick of seeing that message all the time. :(

Cheers.

Yeah, I wish we could get things sorted and get back to Boinc calling the shots.
So depressing when I can't have the rigs weather a day or two outage.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1198137 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1198159 - Posted: 21 Feb 2012, 9:43:42 UTC - in response to Message 1198137.  

Yeah, I wish we could get things sorted and get back to Boinc calling the shots.
So depressing when I can't have the rigs weather a day or two outage.

I hope Eric can get that assimilator sorted out for when the new download server arrives - after all, we'll want massive download congestion so we can see what it can do under real load (and give them a chance to fine tune the software).
ID: 1198159 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1198161 - Posted: 21 Feb 2012, 9:49:39 UTC - in response to Message 1198159.  

I hope Eric can get that assimilator sorted out for when the new download server arrives - after all, we'll want massive download congestion so we can see what it can do under real load (and give them a chance to fine tune the software).

Well at least the number to be assimilated has dropped but I wonder why the AP creation rate is so low.

Cheers.
ID: 1198161 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1198164 - Posted: 21 Feb 2012, 10:15:38 UTC - in response to Message 1198161.  

I hope Eric can get that assimilator sorted out for when the new download server arrives - after all, we'll want massive download congestion so we can see what it can do under real load (and give them a chance to fine tune the software).

Well at least the number to be assimilated has dropped but I wonder why the AP creation rate is so low.

Cheers.

Ooops, sorry - it's a validator that Eric is working on, of course. 'Not enough coffee' error.
ID: 1198164 · Report as offensive
Previous · 1 · 2 · 3 · 4 · Next

Message boards : Number crunching : The 400 and 50 WU limits are way too small


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.