Message boards :
Number crunching :
Still shorter deadlines, 4.34 - 55.12 days.
Message board moderation
Author | Message |
---|---|
Ingleside Send message Joined: 4 Feb 03 Posts: 1546 Credit: 15,832,022 RAC: 13 |
The estimates the splitter is using was adjusted 2 days ago for "mid-range", meaning for angle-ranges 0.225485776 < AR < 1.127428904 This update should make the estimates closer to reality, but it still seems to be on the high-end. Still, the deadlines also made tighter, so here is the updated table. The times is "normalized" to a computer that uses 1h under "normal" SETI@Home, to make it easier for others to get a rough idea of how fast/slow their computers will be... Crunch-times and Deadlines in Seti_Enhanced is heavily influenced by angle-range, so if your computer uses 1h with the "standard" v4.18-seti-application the theoretical cpu-times and deadlines are: AR - cpu-time (h) - Deadline (days) 0.112742888 - 2.87 - 19.10 (same for smaller angle-range) 0.12 - 3.04 - 20.25 0.15 - 3.39 - 22.57 0.20 - 3.97 - 26.43 0.225 - 4.26 - 28.36 0.226 - 8.26 - 54.96 0.25 - 7.28 - 48.45 0.30 - 5.80 - 38.58 0.35 - 4.78 - 31.82 0.40 - 4.05 - 26.92 0.41 - 3.92 - 26.11 0.42 - 3.81 - 25.33 0.43 - 3.70 - 24.60 0.44 - 3.59 - 23.90 0.45 - 3.49 - 23.24 0.46 - 3.40 - 22.61 0.47 - 3.31 - 22.01 0.48 - 3.22 - 21.44 0.49 - 3.14 - 20.89 0.50 - 3.06 - 20.37 0.55 - 2.72 - 18.08 0.60 - 2.44 - 16.22 0.65 - 2.21 - 14.67 0.70 - 2.01 - 13.38 0.75 - 1.84 - 12.27 0.80 - 1.70 - 11.32 0.85 - 1.58 - 10.49 0.90 - 1.47 - 9.77 0.95 - 1.37 - 9.13 1.00 - 1.29 - 8.56 1.05 - 1.21 - 8.06 1.10 - 1.14 - 7.60 1.127 - 1.10 - 7.37 1.128 - 0.65 - 4.34 (same for larger angle-range) Please note, this info is based on the current estimates the Splitter is using, so your real cpu-times will very likely be different. Also, even if estimates is correct for one Angle Range, it doesn't mean isn't 2x away for another angle-range. So, don't be surprised if majority of work is 2x-3x, instead of the 4x this table indicates for angle-range 0.4. Oh, and yes, it's no typing-error that Seti_Enhanced is markedly faster on high angle-range than "normal" SETI@Home is. :) As for anyone saying it's unreasonable to compare against a computer using only 1h in "normal" SETI@Home, please remember that instead of hour you can read the table as "multiply with...". So, if example your computer uses 4h with "normal" seti-application, and you get an AngleRange = 0.41, just take 4h * 3.92 = 15.68h. :) |
W-K 666 Send message Joined: 18 May 99 Posts: 19087 Credit: 40,757,560 RAC: 67 |
Bump |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
Making this one sticky for as long as the transition to SE takes. |
Metod, S56RKO Send message Joined: 27 Sep 02 Posts: 309 Credit: 113,221,277 RAC: 9 |
IMHO there's one thing that should be kept in mind while tinkering with deadlines: how realistic are they for majority of users. Consider a dial-up user who's comfortable connecting to the net once every say 5 days. Connect every setting that matches this would be 5 days, right? We're used to get WUs with 2-week deadlines, so no problem here. With seti_enhanced, deadlines wildly vary, some as low as 4 days. Which is a no-go for the user described above. Does scheduller consider setting of connect every? If not, there are (at least) two possibilities:
Metod ... |
Odysseus Send message Joined: 26 Jul 99 Posts: 1808 Credit: 6,701,347 RAC: 6 |
Consider a dial-up user who's comfortable connecting to the net once every say 5 days. Connect every setting that matches this would be 5 days, right? We're used to get WUs with 2-week deadlines, so no problem here. From what I've seen others write around here, yes: if the host doesn't look like it will reconnect before the deadline, it won't get the work. (I'm not sure whether it's the scheduler or the BOINC client that decides.) |
ai5000 Send message Joined: 1 Jan 01 Posts: 57 Credit: 2,805,412 RAC: 0 |
Would it be possible for Boinc to crunch the work unit with the earliest deadline first? I thought it was already suppose to do this, but on my systems Boinc is just crunching units in the order received regardless of when the deadline is. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13746 Credit: 208,696,464 RAC: 304 |
Would it be possible for Boinc to crunch the work unit with the earliest deadline first? It will, if there is a chance of the earlier deadline Units not being done before the deadline. I thought it was already suppose to do this, but on my systems Boinc is just crunching units in the order received regardless of when the deadline is. If there is no chance of the deadline being missed, it will crunch them in the order downloaded. If partway through a unit it notices that the deadline is getting close for another Work Unit, it will preempt the one it's crunching- put it on hold, cruch the Work Unit with the earlier deadline & then continue on witth the pre-empted Work Unit. Grant Darwin NT |
KWSN THE Holy Hand Grenade! Send message Joined: 20 Dec 05 Posts: 3187 Credit: 57,163,290 RAC: 0 |
Would it be possible for Boinc to crunch the work unit with the earliest deadline first? I thought it was already suppose to do this, but on my systems Boinc is just crunching units in the order received regardless of when the deadline is. I've noticed that there's some sort of "near deadline" type scheduling going on... If you see a message on the order of "computer is overcommitted" the BOINC manager will suspend whatever you're working on and start crunching the short deadline WU. (on my AMD ~2600+ [I overclock] the "overcommitted" deadline seems to be about 9 days - I keep a 3.75 day queue, but connect nightly) . Hello, from Albany, CA!... |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
Would it be possible for Boinc to crunch the work unit with the earliest deadline first? I thought it was already suppose to do this, but on my systems Boinc is just crunching units in the order received regardless of when the deadline is. CPU scheduler: Earlier than 4.35 - no attempt to meet deadlines. 4.35 to 5.3.? - Version 1, 1a of meeting deadlines. 5.3.?, 5.4.x - Version 1b of meeting deadlines. Version 1: The host will enter EDF if: Running an EDF simulation indicates that a result will later than 80% of the time from now to the report deadline. A result is due within 24 hours. A result is due within 2 * the connect every X days. Version 1a: Replace the EDF simulation with a RR simulation. Replace 80% with 90%. Version 1b: The host will enter EDF if: A RR simulation indicates that a result will be later than 90% of the computation deadline. The computation deadline is: report deadline - (Connect every X + Switch applications every Y + 24 hours). Version 2 is comming. BOINC WIKI |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13746 Credit: 208,696,464 RAC: 304 |
CPU scheduler: What's Version 2? Grant Darwin NT |
Jack Gulley Send message Joined: 4 Mar 03 Posts: 423 Credit: 526,566 RAC: 0 |
I've noticed that there's some sort of "near deadline" type scheduling going on... If you see a message on the order of "computer is overcommitted". I basically understand how all of that works, but I was quite surprised to see messages on two of my machines (one last night and the other today {Monday}) about the "computer is overcommitted", when in could not have been. Both had been running with a DAYS value of less than 0.5 running both Crunch3r 4.11 and 5.12 applications, with Trux's calibration enabled running a mix. I had selectively processed and returned the few 4.18 WU's, removed the Crunch3r 4.11 application and its 4.11/4.18 entries in the app_info.xml file and the calibration option, then rebooted the systems. Then changed the preferences for their venue to 0.9 DAYS and updated the system, waited the obligatory 10 minutes, trying to keep the cat off the keyboard, and updated again to download and check the additional Enhanced WU's. The "overcommitted" message pops up during this download before the last two download. The only clue, is that two or three more Enhanced WU's downloaded than I expected and they were given estimates only about 2/3 as long as Enhanced WU's have been taking on those AMD64 systems. |
W-K 666 Send message Joined: 18 May 99 Posts: 19087 Credit: 40,757,560 RAC: 67 |
I've noticed that there's some sort of "near deadline" type scheduling going on... If you see a message on the order of "computer is overcommitted". Jack, We are going to see a lot more EDF mode happening. Especially on slower computers, computers that are only on for linited times and those who have large caches of work. A scenario is: a slowish computer, on 8 hours a day, downloads and starts crunching a task (thats the right word now) with a 55 day deadline, after about 10 days the host decides it needs more work to fill the cache and downloads a unit with deadline of 5 days. Without the scheduler the host would continue crunching the original task which requires 10 more days to finish, this would go past the deadline for the second task, therefore waste of time and effort for second task. So JM7's scheduler preempts the first unit, puts the host in EDF and crunches the second task. The second task completes in 3 days and the host then goes back to the first task, all tasks completed and returned by deadlines. Ergo, EDF is not to be worried about, its a design feature, and we should all shout "Hurray for JM7". Andy |
archae86 Send message Joined: 31 Aug 99 Posts: 909 Credit: 1,582,816 RAC: 0 |
Then changed the preferences for their venue to 0.9 DAYS and updated the system, waited the obligatory 10 minutes, trying to keep the cat off the keyboard, and updated again to download and check the additional Enhanced WU's. The "overcommitted" message pops up during this download before the last two download. The only clue, is that two or three more Enhanced WU's downloaded than I expected and they were given estimates only about 2/3 as long as Enhanced WU's have been taking on those AMD64 systems.You and I both run Trux tx36 as our client. I've noticed some oddities in its prefetch behavior which may have something to do with your situation. It appears that it evaluates work on hand fairly, considering resource share, when deciding whether to fetch more work. So when a machine is clicking a long in one-at-a-time mode, it asks for a few seconds of new work just after the work on hand dips below replenishment level, gets one new download, and all is well. But when the work on hand is well below replenishment level (for example, when communcations or the servers have been down for a few hours, or one has just bumped up the days parameter as in your case) so a burst of new work is quite rightly requested, a proportionately huge overfetch routinely occurs. One component leading to overfetch appears to be that for the calculation of how many seconds of work to request, resource share is not taken into account. Another appears to be that the server end applies a big multiplier (possibly the inverse of the current duration correction factor) in translating the requested seconds into actual new downloads. This optimism in computing your capacity when giving you work seems not, however, to carry over into computations of your capacity to get it done, both for the overcommitted declaration you've just observed, and for a refusal to download new work on the grounds one could not get it done in time. Lastly, the deadlines vary with angle range on enhanced Work Units. Perhaps you got at least one with a very short deadline--making it much easier to be diagnosed as overcommitted. I doubt much of this behavior is specific to tx36, as I suspect the relevant code is a faithful copy of the standard client of that period. However running crunch3r version 411 with tx36 set to calibrate does generate low values for the result duration correction factor (I'm at 0.158245 on a Gallatin which runs crunch3r 4.11 on one of its two Hyperthread "cpus"). So if my guess that DCF is involved is right, in that sense tx36 may accentuate the problem. |
W-K 666 Send message Joined: 18 May 99 Posts: 19087 Credit: 40,757,560 RAC: 67 |
archae86 You are correct in you asumption that the resource share is not included in the calculation for new work, You should base your cache size on the number of projects that you crunch for and ensure that number * days of cache do not exceed shortest deadline - 1 day. If you do not want to go into EDF freequently. To abide by those rules to minimise EDF operation users would probably have to reduce their cache to approx 1 day, with the deadlines for enhanced. But EDF is not a error it is a design feature to ensure all work as far as possible is returned on time. Read my scenario in previous post. Andy |
archae86 Send message Joined: 31 Aug 99 Posts: 909 Credit: 1,582,816 RAC: 0 |
archae86 Actually, it is included in some parts of the calculation, and not in others. This is an error. But EDF is not a error it is a design feature to ensure all work as far as possible is returned on time.I was responding to Jack's question with information rather than a lecture. |
Mike Send message Joined: 17 Feb 01 Posts: 34258 Credit: 79,922,639 RAC: 80 |
Hi I dont like the EDF too Crystallize since it starts, but i life with that. I allways manage that in projects tab suspend and resume. regards from germany Mike With each crime and every kindness we birth our future. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13746 Credit: 208,696,464 RAC: 304 |
I dont like the EDF too Crystallize since it starts, but i life with that. I've found if i just leave it alone it all works out. Everything gets returned without missing a deadline. Grant Darwin NT |
Martin A. Boegelund Send message Joined: 4 Jul 00 Posts: 292 Credit: 387,485 RAC: 1 |
I've started a thread called "Report deadlines - are you guys crazy?" about what's bugging me with those short deadlines. My rant should have been posted in this thread of course. Now you've got the link. My main objection is that now I have to turn on my computer only to crunch, whereas I normally live a happy life as a background cruncher, crunching only when my PC is on for other purposes. Turning on my PC only to crunch is in disharmony with the energy considerations message on the SETI frontpage these days. Seti@home started out as an initiative to put idle computers to work, not letting those empty CPU cycles go to waste thus wasting energy. I feel that you have abandoned this original goal by forcing small-time crunchers to turn on PC's only for the purpose of crunching. Please consider pushing the deadlines farther into the future. "Are you suggesting coconuts migrate?" |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
I'd like to think that I'm a whole lot smarter than my computer, and I have tremendous difficulty in predicting the future. Why is it remarkable that a computer would also have difficulty predicting the future? My only real complaint is that terms like "overcommitted" seem to mean a whole lot more to people than they should. Overcommitted means that BOINC might have a little bit of trouble with deadlines so it's a good idea to not add more work. The worst-case scenario has BOINC doing some work in the wrong order, and EDF correcting for that later. Better safe than sorry. |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
Hi Mike, What happens if you don't manage it? -- Ned |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.