Message boards :
Number crunching :
Observation of CreditNew Impact (2)
Message board moderation
Author | Message |
---|---|
Lionel Send message Joined: 25 Mar 00 Posts: 680 Credit: 563,640,304 RAC: 597 |
Note: Data for last 9 days added with additional commentary. I’ve had a look at the data around v7 and v6 WUs as well as AP WUs and below is a quick observational analysis of my data and the impact of v7 in relation to granted credits. Under v6, I was roughly averaging 100 credits per Work Unit (WU). Under v7, it seems that the average is sitting around 75-80 credits per WU. In looking at run-times and taking the outliers out, cpu run time was around 600-660 seconds (10-11 minutes) per WU for v6, and appears to be around 800-1100 seconds (13+ to 18+ minutes) for a v7 WU. CPU time seems to have gone up by a factor of 2-3 from 50-60 seconds for v6 to 90-180 seconds for v7. So doing a quick Back of the Envelope (10.5/15.83=0.66) shows that from a WU processing/throughput capability, I can expect to do roughly 66% of the volume of WUs that I did before (for example, if I was doing 400 WUs per day under v6, I can now expect to do around 264 WUs per day under v7). Looking at the impact on credit gives 0.66*0.775 = 0.514 or 51.4%. In essence I can expect that daily credit for v7 will drop to circa 51% of what I was getting under v6. I am aware of the comments around “that the system needs time to settle down†and that “it thinks all the WUs coming back at the moment are easy, hence the low credit†however, if the system continues to perform as is, then I can expect to see no change from current trajectory. To test the assumption, I have looked at credit per day pre v7 and post v7 implementation (the data is contained below). The average daily credit prior to migration was 221,878. Following migration on 1st June, the average daily credit is showing as 102,376 which is circa 46.1% of the previous daily average under v6. Up to and including 22 June, I saw no real change in daily total even though was comment that the credit system had been tweaked. As part of my thinking, I decided to “benchmark†v7 credits against Astropulse credits. I started this on 23 June with the migration of a single box. Over the next day or two, I noticed what appeared to be an increase in daily totals, so decided to migrate the other two boxes to AP only to see what the full impact would be. The assumption that I was running with was that pre v7, v6 and AP should have been fairly well benchmarked against each other and that the granting of credits would be in approximate equilibrium based on a daily basis. If this was the case, then I would expect to see a rise in daily totals from circa 102k to circa 220k and for the daily total to remain around 220k. From 23 June onwards, totals increased on a daily basis towards the peak of 222k on 29 June. Unfortunately there has been a lack of AP WUs starting around 30 June and so daily totals have declined over the last few days. My observation in here (albeit based on a small amount of data) is that v6 and AP seemed to be fairly well benchmarked against each other. The issue is with v7. V7 is not well benchmarked against v6 WUs, nor is it well benchmarked against AP WUs. In fact, one can almost double their existing daily run rate through only processing AP WUs. Daily run rates: 2013.05.16 – 244,130 2013.05.17 – 220,168 2013.05.18 – 231,098 2013.05.19 – 226,353 2013.05.20 – 224,723 2013.05.21 – 210,477 2013.05.22 - 0 2013.05.23 – 431,485 2013.05.24 – 229,312 2013.05.25 – 228,767 2013.05.26 – 239,021 2013.05.27 – 231,271 2013.05.28 – 231,050 2013.05.29 – 0 2013.05.30 – 392,635 2013.05.31 – 209,556 Migration of all boxes to v7 on “day 1†2013.06.01 – 123,072 2013.06.02 – 94,061 2013.06.03 – 102,333 2013.06.04 – 99,896 2013.06.05 - 65,653 2013.06.06 - 112,209 2013.06.07 - 102,538 2013.06.08 - 110,760 2013.06.09 - 89,757 2013.06.10 - 96,018 2013.06.11 - 111,653 2013.06.12 - 90,091 2013.06.13 - 119,848 2013.06.14 - 99,884 2013.06.15 - 104,561 2013.06.16 - 110,566 2013.06.17 - 110,603 2013.06.18 - 102,856 2013.06.19 - 85,268 2013.06.20 - 140,694 2013.06.21 - 70,247 2013.06.22 - 109,698 Migration of all boxes away from v7 to AP only (over period of ~4 days) 2013.06.23 – 126,873 2013.06.24 – 141,847 2013.06.25 – 169,625 2013.06.26 – 169,517 2013.06.27 – 183,693 2013.06.28 – 206,019 2013.06.29 – 222,126 2013.06.30 – 211,468 (Lack of AP wus from here) 2013.07.01 – 182,394 Comment on Recognition: Whilst some are focusing just on credit, the issue is not about credit as such. It’s about recognition. There are many distributed computing projects to which people contribute resources. The manner and means in which those projects recognise individual contribution is through a system that is based on and allocates credits. Some projects choose to recognise a person’s contribution more than other projects, thus they grant a higher credit rate per contribution for that project. In short, credits are effectively an indication of a person’s contribution to a project. In the case of "the New Credit System" implemented by Seti, recognition of personal contribution has been reduced. At present, the indication is that recognition for effort is effectively half that of what it was prior to the new recognition system being employed. |
James Sotherden Send message Joined: 16 May 99 Posts: 10436 Credit: 110,373,059 RAC: 54 |
And other projects have told DR.A. to stuff his newcredit. Now if he said you will run newcredit or you cant run Boinc, Then you will see parity. I run Seti@Home beacause I believe in it. [/quote] Old James |
Michael W.F. Miles Send message Joined: 24 Mar 07 Posts: 268 Credit: 34,410,870 RAC: 0 |
From what I am seeing running v7 MB cpu tasks are just not worth it. The time it takes a gpu to run AP is one hour on average for a 700 credit the same time for a MB tasks may get you 70 big difference Why so little credit for a MB task. It sure looks like AP is king again for credits Michael Miles |
Lionel Send message Joined: 25 Mar 00 Posts: 680 Credit: 563,640,304 RAC: 597 |
James, Michael, I agree. The new credit system is an absolute stuff up. I also agree that running v7 on a cpu is now virtually not worth it. v7 has taken this over the tipping point. You are far better running AP on a cpu and getting at least twice the credit per core per hour. Also, when AP wus come back you are far better to only processing AP wus, killing off all the others till they are no APs then flipping back to enhanced and v7 to fill the lack of work void. But you know what staggers me is the deafening silence coming from Anderson, Korpela and co. You would think that given the angst around over v7 credits that they would communicate more with the volunteers and briefing us on what they are doing to fix the problem, rather than just appear to be sitting on their hands and putting their heads in the sand. |
Blurf Send message Joined: 2 Sep 06 Posts: 8962 Credit: 12,678,685 RAC: 0 |
But you know what staggers me is the deafening silence coming from Anderson, Korpela and co. You would think that given the angst around over v7 credits that they would communicate more with the volunteers and briefing us on what they are doing to fix the problem, rather than just appear to be sitting on their hands and putting their heads in the sand. Lionel--- 1) Dr Anderson very rarely--if ever--posts on the forums. Don't expect to see posts from here. 2) Dr. Korpela and his staff do not engage in "sitting on their hands"--they're extremely short both budget & staff-wise so they can either take time to post on the forums with every little thing they do, or they can work on keeping the servers. Please cut them some slack. Matt does post updates when he can. |
betreger Send message Joined: 29 Jun 99 Posts: 11408 Credit: 29,581,041 RAC: 66 |
Well stated. |
Thomas Send message Joined: 9 Dec 11 Posts: 1499 Credit: 1,345,576 RAC: 0 |
Thanks very much for the analysis of changes in your RAC Lionel ! Very interesting to see a big cruncher like you. It's very instructive especially for me because I can compare your stalls in KC compared to my team. However, I totally agree with Blurf : don't forget that the staff and budget of the project are minimal and that the project is still working in the emergency. You're here for over 13 years ! You should know... We do what we can with what we have. FYI, here's the RAC of my team before and after v7 2013.05.01 >> 505 404 2013.05.02 >> 525 118 2013.05.03 >> 510 465 2013.05.04 >>543 912 2013.05.05 >> 489 550 2013.05.06 >> 510 695 2013.05.07 >> 0 2013.05.08 >> 962 063 >> Maintenance 2013.05.09 >> 565 545 2013.05.10 >> 552 424 2013.05.11 >> 502 682 2013.05.12 >> 556 887 2013.05.13 >> 567 531 2013.05.14 >> 517 180 2013.05.15 >> 599 158 2013.05.16 >> 553 868 2013.05.17 >> 575 511 2013.05.18 >> 602 858 2013.05.19 >> 560 806 2013.05.20 >> 578 527 2013.05.21 >> 0 2013.05.22 >> 1 225 668 >> Maintenance 2013.05.23 >> 637 222 2013.05.24 >> 599 316 2013.05.25 >> 613 521 2013.05.26 >> 593 240 2013.05.27 >> 587 166 2013.05.28 >> 0 2013.05.29 >> 1 080 200 >> Maintenance 2013.05.30 >> 559 477 2013.05.31 >> 356 910 Average >> 550 KC 2013.06.01 >> 320 867 >> Migration V7 2013.06.02 >> 442 288 2013.06.03 >> 287 334 2013.06.04 >> 181 137 2013.06.05 >> 308 772 2013.06.06 >> 268 995 2013.06.07 >> 289 123 2013.06.08 >> 268 049 2013.06.09 >> 259 008 2013.06.10 >> 300 288 2013.06.11 >> 264 932 2013.06.12 >> 271 672 2013.06.13 >> 256 083 2013.06.14 >> 273 654 2013.06.15 >> 263 289 2013.06.16 >> 276 272 2013.06.17 >> 259 107 2013.06.18 >> 214 800 2013.06.19 >> 311 603 2013.06.20 >> 389 124 2013.06.21 >> 246 658 2013.06.22 >> 284 413 2013.06.23 >> 283 260 2013.06.24 >> 305 561 2013.06.25 >> 276 445 2013.06.26 >> 260 408 2013.06.27 >> 310 254 2013.06.28 >> 296 944 2013.06.29 >> 325 508 2013.06.30 >> 355 612 2013.07.01 >> 311 340 Average >> 289 KC Before/After -47% I also found that many members who use LUNATICS have not yet made the switch from 0.40 to 0.41 They are not supplied with WU's until they are not switching This certainly has an influence on the amount of WU's treaties and therefore on the global RAC |
Thomas Send message Joined: 9 Dec 11 Posts: 1499 Credit: 1,345,576 RAC: 0 |
PS : however, I think it's a mistake not to accept all types of WU's. If everyone did the same, we'll never get to stabilize the situation. And we are here primarily to advance the science. Our RAC is secondary. |
bill Send message Joined: 16 Jun 99 Posts: 861 Credit: 29,352,955 RAC: 0 |
PS : however, I think it's a mistake not to accept all types of WU's. Yep. Plus looking at the project from the lab's side, nothing is broken. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13835 Credit: 208,696,464 RAC: 304 |
RAC still falling, although it might (finally) be starting to bottom out. Grant Darwin NT |
Raistmer Send message Joined: 16 Jun 01 Posts: 6325 Credit: 106,370,077 RAC: 121 |
Quite strange conclusion considering elative speeds of stock CPU (and even opt CPU) AP and GPU AP. I would recommend directly opposite config: to crunch MB on CPU and AP on GPU. At least for ATi. For NV hosts better choice would be MB-only setup of CPU and CUDA opt apps. And avoid CPU AP for both configs. EDIT: and this recommendation has nothing with definitely broken credits awarding. It bases on relative apps performance per se. SETI apps news We're not gonna fight them. We're gonna transcend them. |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
I see normal AR tasks now providing the ~100 credits per tasks instead of the 50-80 I was seeing initially. While many of my MB CPU only machines have a RAC that is still falling I imagine this will correct itself in a few more weeks. Granted the credit system used is still rather broken. Perhaps the BOINC team could get a math major to intern for them or something. SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
Ingleside Send message Joined: 4 Feb 03 Posts: 1546 Credit: 15,832,022 RAC: 13 |
In the case of "the New Credit System" implemented by Seti, recognition of personal contribution has been reduced. At present, the indication is that recognition for effort is effectively half that of what it was prior to the new recognition system being employed. I've no idea how much faster optimized v7 is compared to stock v7 and neither how much faster optimized v6 was compared to stock v6. But to get some numbers, let's say optimized v6 was 2x faster, and optimized v7 is 1.1x faster. Where's basically 3 ways to give credit: 1: Give same credit/wu even wu is different. 2: Give same credit/second for stock application. 3: Give same credit/second for optimized application. Let's also compare two identical computers, except one runs stock and the other optimized. Under v6, the stock did 100 wu/day and got 10000 credit/day while optimized did 200 wu/day and got 20000 credit/day. Let's also say, with stock v7 each wu takes 2x longer to run, meaning with v7 stock does 50 wu/day and optimized 55 wu/day. This means: If #1: stock gets 5000 credit/day, optimized gets 5500 credit/day. If #2: stock gets 10000 credit/day, optimized gets 11000 credit/day. If #3: stock gets 18181 credit/day, optimized gets 20000 credit/day. On top of this, let's say the two computers ran v6 for 1000 days, meaning the stock had 10 M credits and optimized 20 M credits and let's say the optimized didn't upgrade to v7 but stopped crunching. This means: If #1: stock needs 2000 days to catch-up. If #2: stock needs 1000 days to catch-up. If #3: stock needs 550 days to catch-up. Atleast to me, #2 is the system what most correctly shows optimized v6 was really 2x faster than stock meaning needs double the time to catch-up with a computer stopped crunching. If #1, stock gets 1/2 the credit as before while optimized gets roughly 1/4 the credit as before. If #3, stock gets roughly 82% more credit as before while optimized has no change. With #3 even someone ran optimized for 2.7 years if they stop running they'll be overtaken by someone running stock the whole time in only 1.4 years and atleast to me this is not a good "thank you for being 2x faster". But, what about creditNew? As has been mentioned, this gives roughly 10% lower credit for stock application than should, meaning: If #1: stock gets 4500 credit/day, optimized 4950 credit/day. If #2: stock gets 9000 credit/day, optimized gets 9900 credit/day. If #3: stock gets 16364 credit/day, optimized gets 18000 credit/day. This gives the additional effect if optimized computer doesn't run v7: If #1: stock needs 2222 days to catch-up. If #2: stock needs 1111 days to catch-up. If #3: stock needs 611 days to catch-up. "I make so many mistakes. But then just think of all the mistakes I don't make, although I might." |
Tom* Send message Joined: 12 Aug 11 Posts: 127 Credit: 20,769,223 RAC: 9 |
Before everyone talks about V6 stock V6 optimized apps V7 stock and V7 optimized apps, we need to get a baseline vis a vis stock vs optimized apps both GPU and CPU Everything following is only related to MB apps My understanding (please correct me if I am wrong:-)) on day 1 The stock CPU apps for MBV7 are the optimized apps used for V6 plus AVX for even faster Stock MB application and VLAR's were moved to NVIDIA GPU's so we had a triple whammy. 1. V7 apps do more science so take longer across the board CPU and GPU. 2. V7 CPU apps stock are now the old optimized apps for V6 so ALL CPU based systems are now running optimized CPU apps CreditNew now has a New baseline of NO CPU apps (too soon to calculate needs weeks to stabilize on new application) but GPU's are all returning many results against each other and not the CPU's 3. VLAR's which only ran on CPU's (for the most part) in V6 and take three times longer on GPU's in V7 are now really confusing CreditNew Summary - CPU apps in V7 are now twice as fast as in V6 and not processing as many VLAR's. Since the CPU app is used as the baseline in CREDITNEW?? it appears all the GPU's have slowed down plus they are taking longer on each job. Questions - 1. Since VLAR's have moved from NVIDIA to CPU's have we seen any improvement in credits for GPU tasks? 2. Are the Optimized Applications for both GPU and CPU in V6 pretty much the same as the stock apps for V7? At least until we get another breakthru in processing speed of Newer optimized applications?? 3. Is it true to say that the GPU's are getting less credit per hour of work on SETI and is it true to say CPU applications are getting more credit per hour of work? Bottom line Objectives of V7 SETI Get rid of Shorties - accomplished for the most part so far at least Increase optimizations across the board - accomplished Increase Science by having each workunit start the NITPECKER AutoCorrelation I am sure there are more these are just my observations |
ML1 Send message Joined: 25 Nov 01 Posts: 20957 Credit: 7,508,002 RAC: 20 |
Before everyone talks about V6 stock V6 optimized apps V7 stock and V7 optimized So what and how do we calibrate to something tangible that works across all hardware?... Cobblestones come unstuck when banged across different projects because they do not directly measure the resources utilised... As an alternative, one idea could be to count/estimate transistor or bit transitions usefully utilised. That would give useful counts for even non-compute projects. However, there would need to be another count of some kind to reward for such as availability and storage. And then, for any scheme, how/who for the implementation...? Happy fast crunchin', Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Josef W. Segur Send message Joined: 30 Oct 99 Posts: 4504 Credit: 1,414,761 RAC: 0 |
... The stock Sah v7 CPU app primarily differs from the stock Sah v6 CPU app by the addition of code to do the Autocorrelation search. There were also some AVX functions added and it is linked to a newer version of FFTW which also supports AVX. Both stock CPU apps do have considerable optimization, but that is done in a manner allowing the same application to run well on a wide variety of systems. The relative speed of the stock apps is certainly pertinent to a discussion of CreditNew, because CPU efficiency is higher than GPU efficiency as Dr. Anderson conceives the term. Two quotes from the CreditNew wiki page are pertinent: GPUs typically have a higher (10-100X) peak FLOPS than CPUs. However, application efficiency is typically lower (very roughly, 10% for GPUs, 50% for CPUs). If an application has both CPU and GPU versions, the version normalization mechanism figures out which version is most efficient and uses that to reduce the credit granted to less-efficient versions. The v6 and v7 CPU apps are about the same speed when doing v6 tasks, with a small advantage to the v7 version on hosts with AVX capability. The added time of Autocorrelations on CPU is somewhat less than a 10% increase averaged over all angle ranges. On GPU the impact of Autocorrelations is much greater, which translates into a serious loss of efficiency as Dr. Anderson conceives it. Joe |
jason_gee Send message Joined: 24 Nov 06 Posts: 7489 Credit: 91,093,184 RAC: 0 |
... For my two cents in that context, I'm pleased to see Dr A's theories matching reality, since algorithmically both the CPU & GPU autocorelations are order O(nlogn) , however my baseline/reference 'get it working' GPU autcorrelation implementation uses the 4nfft approach, so becomes 4x(nlogn). It's then pretty easy to see how 10% becomes 40%. A tasty Type 2 DCT kernel with attention to max bandwidth and cache locality should improve that handily, once all the dust has settled and other fires extinguished. "Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions. |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
LOL... Now, could you express that in kibbles so the kitties could understand what you just said? Grrnffx32(nonoggin) becomes grrrf(kibblenoggin) with DCT kibbles in the bowl. Meow? "Time is simply the mechanism that keeps everything from happening all at once." |
jason_gee Send message Joined: 24 Nov 06 Posts: 7489 Credit: 91,093,184 RAC: 0 |
Now, could you express that in kibbles so the kitties could understand what you just said? how about 'Quick & dirty gets the job done, but perfection[purrrrfection ?] takes a little more time & effort' ? "Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions. |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
Now, could you express that in kibbles so the kitties could understand what you just said? OK, that'll work for now. We'll wait for the next installment. Meowpurrrrrrrring along 'till then. Thanks, Jason. "Time is simply the mechanism that keeps everything from happening all at once." |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.