trux

Have you released this version?

Tks
Roberto

---

No, sorry, not yet. Still making some changes and planning to test it first alone for couple of days, then with the help of some team members. Only when it proves to work as intended, and when the Enhanced application is not released in the mean time, I'll release it then publicly. Currently, I do not expect releasing it this week.

---

trux
BOINC software
Freediving Team
Czech Republic

I see one possible problem, and that is the use of the "Results duration_correction_factor".

I have have had problems with this value being way off and not representing the correct value for a system. (This effects how many results will be downloaded for a given days setting.) It takes a lot of WU's to get this value to correct on its on, and there does not seem to be quick way to correct it when it is way off for some reason.

But if it is used in the process to "adjust" credits by "adjusting time", then it will result in this value being corrected even more slowly when it is very far off, if it gets corrected in the proper direction at all.

There is also a risk that this method could, depending exactly how the calculations are done both in the client and the server, cause this value to slowly drift away from where it should be. This effect would take a long time to show up and not be noticeable in a short quick test. (ie. you are using a "correction factor" to adjust values that effect the adjustment of the "correction factor".)

In the case of the duration_correction_factor already being way off, this method could force someone to run in the Calibration mode for a very long time, as the Results duration_correction_factor is corrected back into the correct range. Sort of hard to force people to do this, as during this process they could be claiming less and less credit per WU, and the end result of having a corrected duration_correction_factor is that they end up claiming less credit per WU. (But a valid claim.)

The calibration is much more complicated than just simply applying the duration_correction_factor. And, of course, there is mechanism to avoid excessive claimed credit, impossible reported final_dpu_time, or credits lower than with uncalibrated client. Preliminary tests shown good and persistent results claiming around 32 cobblestines for full-lenght units and proportionally lower for short or aborted noisy WU's. Of course, tests on more machines are necessary. And do not be afraid - if I see it does not work as intended, I simply do not release the client.

---

trux
BOINC software
Freediving Team
Czech Republic

You misunderstood, Paul. As I wrote, the Flop counting will be useless in the moment a developer comes with a clever trick that modifies the algorithm in the way, it uses a different number of operations. I mean something similar to what happened when Hans Dorn with Harals Naparstd introduced the caching, or even something more radical. I know that caching is already built in the Enhanced version, but it does not mean new improvemnt cannot be found. Once a new more efficient algorithm is developped, there is no reason it should claim less credit than the older software for the same units.

Ah, but it should claim less credit! When we are counting cobblestones, we are counting how much effort the computer used to get the result. And we are granting credit based on amount of effort the computer put into it. The most fair way to grant that credit (based on effort) is by counting the number of FLOPS the CPU used to get the result... So if the computer takes less effort to get the same results, it should claim less credit.

I'll give you a super-duper simplistic example.

Suppose we never discovered that to calculate an area of a circle is pi * r^2.

Instead we have been using people to calculate this area by fitting as many rectangles into the circle that will fit and summing all those areas up! (integration).

To get an accurate result of the area of the circle, would require many additions and multiplications. Then, we discovered pi*r^2!

Now suppose we paid people to calculate the area of a circle. Do we pay the guys who calculate the area of the circle (pi*r^2) the same amount as the original set of people?

Nope. We pay that guy alot less per circle, but overall he will get paid much more as he can do that much more circles in a day.

Instead, we should get everyone else to calculate areas of circles the same better way. Now that would be progress. Your other idea, I would call inflation.

Jimmy

---

Ah, but it should claim less credit! When we are counting cobblestones, we are counting how much effort the computer used to get the result. And we are granting credit based on amount of effort the computer put into it.

Another super duper simplistic example would be the delivery of a letter.

Choice 1 is a postal service which puts your letter in a bag with thousands of others, puts that bag in an aircraft with thousands of other bags, and delivers it in a couple of days for a few cents.

Choice 2 is to give your letter and instructions for delivery to the neighbourhood idiot who may (or may not), after a few months of wandering, deliver the letter to the intended recipient after many miles of walking and many inquiries as to how to find the intended recipient.

Obviously, the idiot put more effort into it and so is entitled to more credit than the postal service. Right? Even at minimum wage for say 3 months, this may amount to something like $3000. Therefore, we pay the idiot $3000 and the postal service 25 cents to accomplish the same goal? Ridiculous!

It is the accomplishment of a result which should be rewarded equally, not the amount of effort (wasted or otherwise) which is put into it.

---

Oh I'm going to love this. Using crunch3rs optimised seti ap my 3.2GHz P4 HT can complete approx 60 wu's a day. I could then use the new Trux altered BOINC client to get 32c/wu. So I'd get upwards of 1900c/day, whereas prior to all optimisations I'd be lucky to hit 400c/day. Gotta love it! Currently I rarely get 32c/wu and probably average about 20c/wu = 1200c/day (please note this machine is currently focusing mainly on CPDN where this is not an issue, with a little SETI on the side).

Even though I am all for it and am using them, I believe all optimisations should be released by the projects.

SETI@home should be releasing specific optimised SETI applications for all platform types (these could be supplied by 3rd parties and validated by the projects prior to release therefore reducing the load on the projects).

Berkerly should be releasing all BOINC releases.

It is just fairer overall this way. If I, as a new volunteer, downloaded BOINC, attached to SETI and got the standard app, then crunched a wu in good faith, claimed my 32c and regularly only received an average of 20c then I'd be seriously cheesed off, especially when I'd be getting under 10c for a proportion of my completed work. In fact this is a point I made back in the beta days when the variability in granted credit between linux and win machines was causing much angst. Bring on the FLOP and IOP counting I say!

But in the mean time go ahead and release your change trux. I look forward to getting 32c per SETI wu :) How would this work for say Rosetta or Einstein?

Live long and crunch.

---

Paul
(S@H1 8888)
And proud of it!

You misunderstood, Paul. As I wrote, the Flop counting will be useless in the moment a developer comes with a clever trick that modifies the algorithm in the way, it uses a different number of operations. I mean something similar to what happened when Hans Dorn with Harals Naparstd introduced the caching, or even something more radical. I know that caching is already built in the Enhanced version, but it does not mean new improvemnt cannot be found. Once a new more efficient algorithm is developped, there is no reason it should claim less credit than the older software for the same units.

Ah, but it should claim less credit! When we are counting cobblestones, we are counting how much effort the computer used to get the result. And we are granting credit based on amount of effort the computer put into it. The most fair way to grant that credit (based on effort) is by counting the number of FLOPS the CPU used to get the result... So if the computer takes less effort to get the same results, it should claim less credit...

Jimmy

The _intent_ of BOINC has always been to grant credit based on the amount of work done, not the efficiency. The method of calibrating the computer's capability with benchmarks and then using CPU time to calculate credits has too much variation to do that reliably for S@H, though average granted credit is within a factor of 2 of what it should be.

The so-called "flops counting" cannot fulfill that intent unless it gives the same value on various hardware. So far, it does.

                                                       Joe

The calibration is much more complicated than just simply applying the duration_correction_factor. And, of course, there is mechanism to avoid excessive claimed credit, impossible reported final_dpu_time, or credits lower than with uncalibrated client. Preliminary tests shown good and persistent results claiming around 32 cobblestines for full-lenght units and proportionally lower for short or aborted noisy WU's. Of course, tests on more machines are necessary. And do not be afraid - if I see it does not work as intended, I simply do not release the client.

You do realise the figure of 32 credits/unit that everybody refers too was taken from the reference unit. The reference unit as far as I know takes about 33% longer to process than a 'normal average' unit, and therefore the average claimed credits/unit should be about 24.

---

The time for calculating a reference unit is very close to the time of any full-length unit. The average is lowered by shorter or noisy units, and as I already explained, it is being taken in view in the calibrated client, and calculated proportionally. If you are still suspicious, I invite you to scrutinize the source code.

The biggest advantage of the calibrating, besides fair S@H credits, is also the fact that it in no way messes up other projects as otherwise artificial bloating of benchmarks does.

Btw, the new client version is now finished and currently tested by several users. You can see it in action if you browse my computers - most of my Windows machines run now the last calibrating client. For example here:

http://setiathome.berkeley.edu/results.php?hostid=1566775&offset=500

You may need to browse couple of pages - it changes permanently. Also please keep on mind that the client needs couple of hours or days to be really efficient.

Make sure to check also the unit details - (the leftmost column in the result table). In the field stderr out, you can see the details of the calibration, including uncalibrated (real) values. For example here:

http://setiathome.berkeley.edu/result.php?resultid=190049745

The client now uses a combination of modifying both reported flops and final_time, but always so that it does not violate any given conditions. Also, the external manipulation is made more difficult than in the standard client by adding some checksums.

Beside it, I added some other new features:

- priority project(s) - as long as there is work for the priority project(s), the other registered backup projects will never started. They are only used when there is no more work available for the primary project.

- network masks in remote_hosts.cfg - the possibility to assign entire blocks of IP addresses

- possibility of resetting of all long time and short time debts

I'll post more details, the URL, and the source code, as soon as it is better beta tested

---

trux
BOINC software
Freediving Team
Czech Republic

The time for calculating a reference unit is very close to the time of any full-length unit. The average is lowered by shorter or noisy units, and as I already explained, it is being taken in view in the calibrated client, and calculated proportionally...

Sounds very good.

How are you making the calibration? Do you run a reference WU and then take the CPU time for that?

Beside it, I added some other new features:

Good extras.

Can these be added into the official sources?

Good work,

Regards,
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

How are you making the calibration? Do you run a reference WU and then take the CPU time for that?

Yes, this was my initial approach, but finally I found out this hassle and time wasting is unnecessary. Besides it it would be necessary to do it periodically. So far it looks it can calibrate equally well with the help of the available data, and historical records. If you wish to know the exact details, I invite you to look at my source code - it will be published together with the client.

Can these be added into the official sources?

Yes, if the official team is interested, I have nothing against it. Maybe one of the volunteeer developers frequenting this board can take care of it (if they find it worth of it)

---

trux
BOINC software
Freediving Team
Czech Republic

sched/handle_request.C line 602-617.

Thanks, Ingleside, this is a valuable information. It indeed looks like I may have to change the method and use benchmarks instead.

Note that you could return the modified benchmarks as <fpops_per_cpu_sec> and <iops_per_cpu_sec> for each project rather than changing the overall BOINC benchmark fields. Those values are meant to be set from an application which does its own benchmarks, in effect your calibration procedure is calculating the same.

                                                    Joe

Note that you could return the modified benchmarks as <fpops_per_cpu_sec> and <iops_per_cpu_sec> for each project rather than changing the overall BOINC benchmark fields. Those values are meant to be set from an application which does its own benchmarks, in effect your calibration procedure is calculating the same.

As I wrote, I do both. And I have to do both, because changing alone the benchmark does not give sufficient flexibility, and would work without finetuning the final_time only if you always managed to return the result immediately after completing.

EDIT: the overall benchmark remains untached, of course, but I explained that already earlier

---

trux
BOINC software
Freediving Team
Czech Republic

Beside it, I added some other new features:

- priority project(s) - as long as there is work for the priority project(s), the other registered backup projects will never started. They are only used when there is no more work available for the primary project.

Coolness!

Looks good.

Wait eagerly for it to be released.

---

Foamy is "Lord and Master".
(Oh, + some Classic WUs too.)

Btw, the new client version is now finished and currently tested by several users. You can see it in action if you browse my computers - most of my Windows machines run now the last calibrating client. For example here:

http://setiathome.berkeley.edu/results.php?hostid=1566775&offset=500

You may need to browse couple of pages - it changes permanently. Also please keep on mind that the client needs couple of hours or days to be really efficient.

looks good, do you have some examples of finished results for other projects?

By the look of your results it seems to be working fine but when will it be availiable to us.

I for one am looking forward to claiming a fair amount of credit for my work as since I started using optimised apps I have tried several optimised clients and can not find one that claims more than 10 credits per WU.

---

Are you working on this with any of the linux optimizers? Considering that the benchmarks and associated credit claims tend to be lower on computers running linux than than those running windows I think it would be helpful in reducing the difference in credit claims and the variation in credit granted for work completed.

---

Nothing travels faster than the speed of light with the possible exception of bad news, which obeys its own special laws.
Douglas Adams (1952 - 2001)

looks good, do you have some examples of finished results for other projects?

I run only Eistein@home and LHC on some computers, but there are not enough results (if any) yet there, and I did not activate the calibration at those projects yet. There are couple of our team users testing it, but I do not yet have any feedback regarding other projects than S@H.

I may need to invite some more beta testers. If some of you are interested, please register in our team discussion board. I'll offer the client for testing to registered users probably later today. I do not want yet publishing it here, I prefer to keep the number of beta testers limited and under control to avoid problems with recalling the client in case of serious glitches.

---

trux
BOINC software
Freediving Team
Czech Republic