Message boards :
Number crunching :
Possible Workunit Distribution "Solution"
Message board moderation
Author | Message |
---|---|
Christopher Hauber Send message Joined: 10 Feb 01 Posts: 196 Credit: 71,611 RAC: 0 |
Perhaps in these early days of up and downs with workunit supply, and in the future when there is just plain more power than work, some kind of system should be employed to help better distribute workunits when they are prepared. Every time UCB queues up a bunch of workunits, they are gone before some people are really able to get much from it because of the caching of workunits on PCs and so-called farms. When I first started thinking about this a little bit ago, I thought in terms of maybe one unit every 10 minutes, but realized that wasn't very practical for some modem folks. So perhaps creating a maximum download of say 4 or 5 workunits every ten minutes would be a good compromise between dialup and broadband. All that would need to happen is for the server to send a maximum of 4 or 5 to a single computer, and then require BOINC to ask for more later. The 10 minute rule part of that already exists as minimum time between RPCs so the only change would be the amount sent at one time. I think that might slow down the depletion of workunit supplies enough that more people could benefit from the supplies. It would still allow people to build up their caches, but perhaps add some "fairness" to it. It wouldn't prevent supplies from being exhausted, but it would slow it down a little. Chris |
Mike Send message Joined: 17 Feb 01 Posts: 34253 Credit: 79,922,639 RAC: 80 |
Hi I dont think its an good idea, because this will increase the server load again. The WUs load are instead 50 per day / host. When the storm of download is over it will work nearly normal i think. The splitter are creating 1000 WUs all 10 minutes as Rom said this should be enough. greetz Mike |
ColdRain Send message Joined: 28 Feb 02 Posts: 14 Credit: 1,092,701 RAC: 0 |
It's not enough. At all. 1000 WU's every 10 minutes is 144 kWU per day. That's ONLY 1 day of work for about 20000 ppl. The current number of active users on S@H-1 is more then 450000. Imagine what would happen if S@H-2 had to cope with let's say only 10% of them ... |
Pascal, K G Send message Joined: 3 Apr 99 Posts: 2343 Credit: 150,491 RAC: 0 |
Fair? this is not a game we are volunteers, if there is no work oh well.... Finite Work Supply Bah!!! I am in it for the money. Message from the Head ;o) of the Horse Head Nebula Branch of Seti@Home, Check is in the mail |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
> Fair? this is not a game we are volunteers, if there is no work oh well.... > Exactly! I've devoted exactly ONE of my systems to BOINC. The others are still running Classic. When/if my BOINC system runs out of SETI WUs, I've got it setup so it works on a non-BOINC/non-classic distributed project (Smallpox research). When the WUs became available yesterday, my system downloaded 1.5 to 3.0 days of work into my cache. If nothing's available when it's gone...well, that's life... deal with it. |
Christopher Hauber Send message Joined: 10 Feb 01 Posts: 196 Credit: 71,611 RAC: 0 |
Well I'm not sure that it would really increase server load since it wouldn't really be doing anything it doesn't do now. It already prevents RPCs less than 10 minutes apart. It would simply be the addition of an "if 5 sent, stop" type thing. If the client still wants more, it can ask again in 10 minutes. When the splitter goes down temporarily or something and workunits become unavailable, then are suddenly available again, there are a lot of clients that suddenly suck up the available stockpile making unavailable for lots of people trying to fill their cache. Perhaps "fair" wasn't the proper way to word it. There would be more benefit to it than that I believe. Not only would it give more computers a chance to get work (which wouldn't eliminate complaining, but may reduce it), you have more workunits being crunched simultaneously, thus making better use of the resources. I also believe this would be useful when the time comes that not enough data is recorded to keep up. When that happens, when a new data tape is popped in, there will be a sudden supply of workunits that get sucked up by a few hungry computers. Like I said before, the more computers you have working, the faster you get the work done and make much better use of your resources. And hopefully reduce the amount of ill-will from the users. At least then they feel like they have a better chance of even being able to help. You all should know how I stand with regards to BOINC and how things have been going so far. I've always been supportive of how things have been going and understanding of things. I've always tried to use sound reasoning and use common sense when posting. You are right Pascal, if there is no work, then oh well. That has never been a complaint of mine. This is for when work becomes available. It's is just an idea for possibly making better allocation of the resources available to the project while possibly alleviating some of the discontent from users -- and with little to no increase in server load. I realize nothing will satisfy some people and people will always complain. I realize that as we volunteer our computers, fairness isn't really a priority. I've dont always EXPECT to be able to obtain work, nor for everyone to obtain work. But I do believe it should be easier for large numbers of computers to get small amounts of work than for smaller numbers computers to get large amounts of work. If you still don't think it's a good idea, then fine. But since I didn't really have much time to post earlier when it first occured to me and so much disapproval, I thought maybe I should clarify and explain my reasoning on this. It may NOT be a good idea, but as far as I can tell right now, I think it makes sense. Chris |
[B^S] Zain Upton Send message Joined: 16 Feb 01 Posts: 132 Credit: 43,763 RAC: 0 |
Chris, Well I dont know about others, but I think your posts are ALWAYS well thought out, informative, and generally right on the money. I wouldnt be listening to the whiners that will always appear in global projects like this. My hat is off to you Chris *tip* --- <p align="center"> </p>Click on the counter to visit our team site including Extended Team Stats with daily totals, Member signatures and more! |
Toby Send message Joined: 26 Oct 00 Posts: 1005 Credit: 6,366,949 RAC: 0 |
I'm not sure but I think something like this might already be in place. When I was first able to download work units after the latest outage, I only got 3 at first. Then a little while later I got 3 or 4 more. Eventually my queue was filled up to my 3 day queue depth but it didn't all come in at once. Not sure if that was just a fluke or if it is built into the system. Anyone else notice one way or the other? ------------------------------------------- - A member of The Knights Who Say NI! Possibly the best stats site in the universe: http://boinc-kwsn.no-ip.info |
[B^S] Zain Upton Send message Joined: 16 Feb 01 Posts: 132 Credit: 43,763 RAC: 0 |
Reports from our team members (SETI Synergy) are highly variable. Some members did just as you did, got a few work units, then a few more, and have had some to crunch ever since. Other members still have not recieved one work unit... and are getting frustrated even more hearing of these people who are crunching seemingly randomly. But thats how it goes. You have to trust the dev have a master plan. <p align="center"> </p>Click on the counter to visit our team site including Extended Team Stats with daily totals, Member signatures and more! |
Keck_Komputers Send message Joined: 4 Jul 99 Posts: 1575 Credit: 4,152,111 RAC: 1 |
I don't know how this would effect the back-end. I would expect that when there are multiple projects running this type of idea would be helpfull, because then when the client is doing its backoff from one project it would likely get work from another one. In the current situation with only a few active projects I don't think it would be much help. I would expect this idea to help ease user frustration some since you would be likely to get a workunit in addition to the backoff message. The way I would work it would be dynamic. If there are few workunits or the server is heavily loaded then give only a few workunits at a time and request a 1 to 3 hour backoff. The exact number of workunits and the length of the backoff would vary based on how heavy the load is and how many workunits are available. John Keck BOINCing since 2002/12/08 |
Scott Brown Send message Joined: 5 Sep 00 Posts: 110 Credit: 59,739 RAC: 0 |
Chris' idea is also sound scientifically. It is in the interest of the project to have a wide variety of machines processing units to increase the validity of the results. Of course, we are overlooking an obvious solution. Instead of only three hosts per work unit, why not increase the number of host to four or five. While we certainly don't want to crunch the same data over and over again as is happening with classic now, verification by 4 or 5 hosts is not absurd and would help alleviate work unit supply/demand. Scott |
Jaaku Send message Joined: 29 Oct 02 Posts: 494 Credit: 346,224 RAC: 0 |
Sure 5 sounds good, but so does 15 what is the change of getting the same WU and plus you could compare them and delete the stray results. But any way if SETI was real intrested they would go to more dishes and collect there data even if they dont do it them selves but just copy what it looks at! ^ Click Here ^ ^ Click Here ^ ^ Click Here ^ |
KWSN - MajorKong Send message Joined: 5 Jan 00 Posts: 2892 Credit: 1,499,890 RAC: 0 |
> Of course, we are overlooking an obvious solution. Instead of only three > hosts per work unit, why not increase the number of host to four or five. > While we certainly don't want to crunch the same data over and over again as > is happening with classic now, verification by 4 or 5 hosts is not absurd and > would help alleviate work unit supply/demand. > > Scott > > That is the EXACT reason that Berkeley has set the number to 3. They don't want BOINC/S@H to be doing busy-work like S@H-Classic is. The BOINC framework is designed to be multi-project, and it is expected that one runs multiple projects to cover 'down time'. If one is ONLY interested in running S@H, then one MUST accept being idle during times of 'no work available'. ------------ KWSN-MajorKong KWSN Forum Admin (retired) http://www.kwsnforum.com BOINC Beta tester |
Pascal, K G Send message Joined: 3 Apr 99 Posts: 2343 Credit: 150,491 RAC: 0 |
> > Fair? this is not a game we are volunteers, if there is no work oh > well.... > > > Exactly! I've devoted exactly ONE of my systems to BOINC. The others are > still running Classic. When/if my BOINC system runs out of SETI WUs, I've got > it setup so it works on a non-BOINC/non-classic distributed project (Smallpox > research). > > When the WUs became available yesterday, my system downloaded 1.5 to 3.0 days > of work into my cache. If nothing's available when it's gone...well, that's > life... deal with it. > > I like you Randy, you understand the system...Seti@Home is not here to keep us busy they are here to find ET..... If I find ET, I hope I do not get stuck with the long distance charges, ;o) Bah!!! I am in it for the money. Message from the Head ;o) of the Horse Head Nebula Branch of Seti@Home, Check is in the mail |
Scott Brown Send message Joined: 5 Sep 00 Posts: 110 Credit: 59,739 RAC: 0 |
When I suggested verification by more than 3 hosts per work unit, it was not to advocate "busy-work". But that does beg the question of what is "busy-work". I am afraid that I don't agree that going to 4 hosts (or even 5) per unit is fundamentally different from the arbitrarily set 3 hosts/unit. My point was that by simply raising this bar by a minimal number, some of the work demand could be resolved. Perhaps 4 hosts might even provide a more efficient manner of crediting for users (not that I am very concerned with receiving credit myself, but it is clearly a major motivator for many). For example, maintain the 3 hosts verification requirement, but send the unit to 4 hosts. The first 3 to return the unit determine the credit received--the fourth hosts merely receives this credit. In cases where one hosts does not return/returns a corrupted unit, the fourth host's results count in determining the credit and such credit is then allocated more quickly. Thus, some portion of work demand is resolved; results are returned more quickly to Berkeley (i.e., without reissuing a unit, etc.); and users concerned with credit are happier since their credits will be posted more quickly. Scott |
Ingleside Send message Joined: 4 Feb 03 Posts: 1546 Credit: 15,832,022 RAC: 13 |
AFAIK they was planning on using 2 for verification & distributing 3, but due to all the cry-babies in beta complaining about always being paired off by a debug-free *nix-box or someone with outdated benchmark-numbers, forgetting all beta-stats should be zeroed, they increased the required to verification to 3. As for upping the distribution to example 5, it's not certain this will help at all since the limiting step currently is slow database-access, and adding 999 results from 333 wu is maybe not slower than adding 1000 results from 200 wu... But something that can be done is either use a debug-version of seti, or change some of the wu-limits so more science is done... Both will lead to slower crunching, and therefore less wu needs to be distributed. |
Rom Walton (BOINC) Send message Joined: 28 Apr 00 Posts: 579 Credit: 130,733 RAC: 0 |
Ingleside is correct, the database currentlly needs a 3Mbit write pipe to the disks and we are not achiving that with the current disk subsystem. The new disk subsystem is supposed to have a 10Mbit write pipe capability. Just increasing the number of results means you pay for the writes at the time somebody attemts to return the results as each result/workunit has to be updated to notifiy the transitioner that it needs to be scheduled for validation or more result creation. ----- Rom BOINC Development Team, U.C. Berkeley |
Byron Leigh Hatch @ team Carl Sagan Send message Joined: 5 Jul 99 Posts: 4548 Credit: 35,667,570 RAC: 4 |
|
DaMaCon Send message Joined: 23 May 03 Posts: 5 Credit: 189,415 RAC: 0 |
Neophyte question here (in fact, two) 1) I have received and processed a number of WUs, but under my Project tab I see 0 in Total and Average Credit. I've recently converted from the old seti to boinc, and have followed some of the discussion about the troubles of the new system. Should I expect to see some credit, or do I have to do something else? 2) The rather neat sigs show interesting stats - are there docs that reveal how? Thanks. <a> [/url] |
Scott Brown Send message Joined: 5 Sep 00 Posts: 110 Credit: 59,739 RAC: 0 |
> Ingleside is correct, the database currentlly needs a 3Mbit write pipe to the > disks and we are not achiving that with the current disk subsystem. > > The new disk subsystem is supposed to have a 10Mbit write pipe capability. > > Just increasing the number of results means you pay for the writes at the time > somebody attemts to return the results as each result/workunit has to be > updated to notifiy the transitioner that it needs to be scheduled for > validation or more result creation. > > ----- Rom > BOINC Development Team, U.C. Berkeley > > Rom and Ingleside, Point well taken. Thanks for the info! Scott |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.