Think what mine feels like after weeks of walking through and around the code - some bits work "as advertised", others don't, and some are all but impenetrable in their reliance of so many other bits of code.

One thing is certain - the dependence of credit granted on the nature of the cruncher (combination of application and processor) is all but impossible to stabilise unless one has a constant supply of absolutely non-variant tasks. Any change in the nature of the task will cause instability in the result, some instabilities will be "ignorable", and others will be "horrible", and the same is true of changes to the cruncher.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Back to work tomorrow & I can see myself not getting any sleep tonight because i'll have this stuck, spinning around in my head, with no hope of understanding let alone resolution...

---

Grant
Darwin NT

to save a little bit of your head pinning - there is nothing that the user can do do prevent the credit scoring system throwing wobbles :-(

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

I think CreditNew is something like quantum entanglement.
A working theory, but not really fully understood.
MeowLOL.

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Nice one Mark - I'd not thought of that analogy :-)

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Well, when staring in the face of adversity it sometimes helps to try to maintain a sense of humor about it all.

Meow.

---

"Freedom is just Chaos, with better lighting." Alan Dean Foster

Back to work tomorrow & I can see myself not getting any sleep tonight because i'll have this stuck, spinning around in my head, with no hope of understanding let alone resolution...

:)

It was one of the things I wanted to get to in this thread but thankfully Richard already explained it nicely... for your own sanity you may want to start crunching stock for a few weeks :) CN doesn't seem half as crazy on stock. For now, the healthiest approach (for everybody) would be to think about it this way:

We* are essentially beta-testing apps on main (the anonymous apps) and getting "Seti main" credit for them. I mean it's not like Anonymous Platform apps are relegated to Seti Beta right? It's kinda cool that we can use other "stuff" here on Main...

So we kinda have our cake and are eating too :) And to add insult to injury we complain when CN gets its panties in a twist trying to figure out if those anon apps are cheating or not and does whacky things with the credit ;)

All of the above was one of the things I wanted us to discuss in a bit more detail in this thread.

Some others include:
- Why it's always been and always will be... flops (aka Cobblestones aka "Credit")
- Why we don't really need any kind of GPU estimates anyway (on SETI)
- Why GPUs really ARE inefficient. It's not why you think! :D
- Why credit has nothing to do with time (sort of)
- Why we can't have much higher fpops even if we wanted to
- maybe a couple more as I've got everything jumbled in my head... I wasn't planning on doing a bullet point list :)

*I say "we" even though I'm not crunching anon atm, I will be soon (and have in the past)

We* are essentially beta-testing apps on main (the anonymous apps) and getting "Seti main" credit for them.

True and not-true.
True - some people are developing their own applications, and (sadly) the rules over at Beta coupled with the small number of tasks makes main the only realistic place to test (in their eyes). This is a very limited number of users.
Not true - the majority of anonymous platform users are running well tested, stable and accurate applications that would "pass for main" if they went through the Beta process.
Not true - if a user decides to run multiple tasks on a single GPU and stock applications they will be displayed as "anonymous" applications.
And for the your reference the word is "optimised" not "anonymous" - although in one sense the latter is correct, in that you can't see what actual application has been used without "opening the lid" on the task results file.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

So we kinda have our cake and are eating too :) And to add insult to injury we complain when CN gets its panties in a twist trying to figure out if those anon apps are cheating or not and does whacky things with the credit ;)

Not true - The instability in CreditNew is not only down to the application, it is dominated by the hardware. CreditNew was designed back in the days when nobody knew how far GPUs would develop, the distributed computing world was dominated by CPUs which have a limited range of performance compared to the performance range of GPUs. From my own collection of computers the CPU performance range is a about 2.5:1 and the GPU performance range is easily 10:1, and may even be 20:1. The bits of the software in the CreditNew calculation that are meant to "calibrate" for different processors are just about stable with a short term range of about 3:1, greater than that they get increasingly unstable. And worse, this instability not only affect the GPUs (bad enough), but also impacts on CPUs (really bad).

Please define what you mean by "cheating" as different people have different meanings when associated with CreditNew.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

We* are essentially beta-testing apps on main (the anonymous apps) and getting "Seti main" credit for them. I mean it's not like Anonymous Platform apps are relegated to Seti Beta right? It's kinda cool that we can use other "stuff" here on Main...

So we kinda have our cake and are eating too :) And to add insult to injury we complain when CN gets its panties in a twist trying to figure out if those anon apps are cheating or not and does whacky things with the credit...

I don't see any mention of 'Beta Testing' on the Anonymous platform page. This is what it says;

...This addresses the needs of most BOINC participants, but it's inadequate if:

your computers have platforms not supported by BOINC or by the project;
for security reasons, you want to only run executables you have compiled yourself;
you want to optimize applications for particular architectures.

To handle these cases, BOINC offers a mechanism called anonymous platform. This lets you build applications yourself, or obtain them from a third party, rather than getting them from the project server. This can be used only for projects that make their source code available. As an example, see the instructions for SETI@home.

If the BOINC client is not available for your platform, you must build it yourself.

To use the anonymous platform mechanism for a particular project:

Get the project's application source code and compile it, or download an executable from a third party.
Run the BOINC client and attach to the project. This will create a 'project directory' (whose name is the project URL) in the BOINC directory. Exit the client. Put your executables in the project directory (Mac OS X users: see note below).
Create a file app_info.xml in the project directory. This file describes the applications you have compiled or downloaded. Its format is described below.
Run the BOINC client again. When it requests work from the scheduling server, it will report its platform as 'anonymous' and send a list of the applications you have supplied. The server will sends tasks for those applications.
https://boinc.berkeley.edu/wiki/Anonymous_platform

In fact, the Only reason to go through the Beta process is if you plan on having the App placed on the SETI Server. If you don't intend on having the App(s) on the SETI Server then going through the Beta Process is definitely Not required to run your Apps as Anonymous platform. It's right there, in black and white. I for one am happy with the current situation. I have nicely running Apps on my machines that work just as well if not better than the ones on the SETI Server.

Now, if there is some problem with Credit allotment on the Anonymous Platform mechanism that is in place, perhaps the people who built the credit system should fix it. In my view, the Credit problem has nothing to do with the Anonymous Platform mechanism, but instead a project wide dysfunction. Most likely due to the system being App based rather than Device based and not meant to handle such a large difference in Devices.

TBar - thanks for the quote from the Anonymous apps page and the additional comments - I was eating my breakfast, sorting out phone calls (two unsolicited and two work before 6:30am)
To use your words "CreditNew is a project wide dysfunction" - I only wish I'd thought of that phrase!

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

anon apps ...///... and does whacky things with the credit

and

CN doesn't seem half as crazy on stock

An interesting view point, but not really borne out in practice. While it is true that the credit granted may appear to be more stable when only running the stock applications. The fact is that the instability in the credit granting system affects everyone, its just that those running the various optimised applications tend to spend more time watching their systems than those running stock applications so notice any changes.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

While considering CreidtNew - how many people reading this thread know that there are FOUR different ways of calculating it? I must admit I didn't until recently. All these are available to every project at setup, and the project chooses the one that is "most appropriate" to their needs/desires/ability....

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

The 'stock' (server application) case is easier to grasp. Each WU has a fixed size, calculated by the splitter from the AR. That's expressed in fpops, and will be the same every time for WUs of the same AR, such as the blc vlars. If you study <rsc_fpops_est>, you'll know exactly how big tasks of each AR have been assessed to be - that's a real figure. In the meantime, the server is also keeping track of how fast your machine has been processing tasks recently - the APR - and passes back that number to your client every time new work is allocated. It's the fixed task size, and the varying speed, which your client uses to estimate how long each task is going to take.

First big point of failure, especially with AstroPulse where initial CPU code was unoptimised at all and optimization was done on algorithm level. That just ruin the idea of pre-calculated flops.

Any errors in estimation flops vs AR (for MB), flops vs blanking % (for AP, but I don't remember if someone bothered to account changes in FLOPS for blanking though in reality blanking can change task computational weigh at least two fold easely) will add to declination from reality.

CreditScrew just what it is, based on too many non-valid in real life assumptions.
The most hated (for me) part in it: it directly discourages any app optimization. I think it's counterproductive. Think Petry would agree...

---

SETI apps news
We're not gonna fight them. We're gonna transcend them.

Maybe the problem is the definition of a Cobblestone? Maybe we need to redefine it; or not so much redefine it, as define it in a different way?

BOINC's unit of credit, the Cobblestone (named after Jeff Cobb of SETI@home), is 1/200 day of CPU time on a reference computer that does 1,000 MFLOPS based on the Whetstone benchmark
a 1 GigaFLOP machine, running full time, produces 200 units of credit in 1 day

Keep in mind that a Credit, the period over which it is measured, the measurement of FLOPS (Floating point Operations Per Second), are all arbitrary. They're all based on a value someone has chosen, and been developed from there.
General relativity & philosophy aside, time is the one non-arbitrary factor. And it's the bit that is making it impossible for me to wrap my head around the definition of a Cobblestone (ie- Credit) and relate it to the FLOPs required to process a given WU.


For those of us that have been around a while, our first experience with Credit was the initial Credit system.
The big problem with this system was the large range of Credit granted for a particular WU, depending on your wingman. What was granted depended on your & their hardware & the application being used (even more than it does now).

The second system (the one many of us remember with great fondness) actually counted the number of FLOPs done while processing the WU, instead of using the benchmark of the hardware doing the processing. There was a scaling factor, so that the Credit for a given job was on par with the less ridiculously high or low claims of the previous system. And this system resulted in WUs of a particular Angle Range (AR) getting the same amount of Credit, irrespective of the hardware or application being used & the wingmen involved.

So regardless of what the original official definition of a Cobblestone is, we came to consider a WU of a particular Angle Range to be worth a particular amount of Credit. And the one thing in common with each of the systems, was the method for deciding how much work had been done was the number of FLOPs required (be it a estimated or actual counted value). And over time with the current Credit system the amount of Credit awarded for a given AR has declined, significantly. So if the original & second Credit systems were reasonably close to awarding Credit inline with the official definition of a Cobblestone, we're no longer doing that.


Like the original definition of the Cobblestone, why don't we now arbitrarily say the Credit awarded for work done using the first 2 systems was in accordance with the definition of a Cobblestone? That being the case, then the Credit awarded for a WU of a given AR (and therefore estimated computation FLOPs) now should be the same.
We still use the scaling/normalisation that Joe did many years ago to compensate for longer/shorter than estimated runtimes using the reference application & hardware. If you use an optimised application that gives improved runtimes on some WUs, then you get the benefit of more Credit by doing more work. If it runs slower on other types of WUs, then you miss out on those- you can't keep your cake & eat it too. You have to take the bad with the good, but overall you will be in front of the curve. This being the case for CPU & well as GPU applications.

With this system there is no need for normalisation between hosts & versions. There is no issue with claimed Credit- you get paid according to the work you have processed. Unrealistic values for performance (APR- Average Processing Rate, Peak FLOPS) etc have no impact on Credit granted. No need for penalising systems for perceived inefficiency. The more efficient the system, the more Credit they will get, the less efficient, the less Credit they will get; no need to manipulate the value of Credit awarded.
The present system for dealing with Cherry Picking either isn't implemented, or doesn't work. A mechanism separate from the Credit one would be a better choice.

As things are, each Project is able to determine the Credit that is paid out, with this revised system projects that choose to pay reasonable levels of Credit, based on work done (FLOPS), will make it possible for stats between projects to be comparable again. We won't have to deal with getting more or less Credit depending on who our wingman is & what hardware & application they are running. No need to fiddle the Credit awarded depending how efficient or not the Credit mechanism thinks certain hardware might be. Unrealistic performance values have no impact on Credit awarded.

---

Grant
Darwin NT

First big point of failure, especially with AstroPulse where initial CPU code was unoptimised at all and optimization was done on algorithm level. That just ruin the idea of pre-calculated flops.

Thanks Raistmer. This goes back into "the dark ages", and clearly demonstrates how things have changed over the years - as I recall the improvement was run times were halved and the credit system really had a turn for the worse.
Do you have any feeling for the impact of the various "sse", "avx" type technologies on runtime? I ask because as far as I understand some of these are in stock apps and some aren't.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

The most hated (for me) part in it: it directly discourages any app optimization. I think it's counterproductive. Think Petry would agree...

There are many people who would agree with you

Looking at Gran't last post, I think he is suggesting moving from a "client biased" credit system to a "task based" one - as much of the work required to implement (FLOPs estimate) this is already done it would be a very simple implementation. But would be "politically undesirable" for certain folks....

Doing so would make it a very useful tool for comparing the performance of applications and hardware - something that Raistmer has identified as being next to impossible to do reliably today.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

First big point of failure, especially with AstroPulse where initial CPU code was unoptimised at all and optimization was done on algorithm level. That just ruin the idea of pre-calculated flops.

Thanks Raistmer. This goes back into "the dark ages", and clearly demonstrates how things have changed over the years - as I recall the improvement was run times were halved and the credit system really had a turn for the worse.
Do you have any feeling for the impact of the various "sse", "avx" type technologies on runtime? I ask because as far as I understand some of these are in stock apps and some aren't.

The SSE, AVX etc ('SIMD', as a generic term) optimisations do make a significant difference (improvement) in runtimes - up to and above three-fold, was Jason's assessment. They are the last remaining valid reason for running Lunatics: all the GPU apps have been adopted as stock apps, so they aren't just 'similar', they're identical.

Originally, people also used Lunatics because app_info.xml was the only way to set up multiple tasks per GPU. Now, everything you need (fractional counts, command line, etc.) can be applied through app_config.xml instead.

The stock CPU position is interesting. Credit New (I'm told) fundamentally assumes that the stock CPU app runs at benchmark speed - FALSE. Even the base stock code contains optimisations, which are specifically and deliberately excluded from the Whetstone definition. In addition, many of the CPU ops are run in the FFTW library (external code), and that does nowadays include SIMD optimisations up to and including AVX if available.

So the normalisation is screwed to pot, as we knew anyway.

Certainly CreditNew bases its performance assumptions on a basic CPU instruction set, ignoring all the SIMD technologies. To get round this it takes wild guesses, and probably gets them wrong, in much the same way it does with GPUs. However the error for CPUs will not be as large as it is for GPUs.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Hi Raistmer, I'm so glad you dropped by :)
I asked you this 5 years ago but things were crazy back then (plus I know you don't really care about credit) so I didn't want to bug you too much and backed off. But now that things have calmed down may I ask again:

The most hated (for me) part in it: it directly discourages any app optimization. I think it's counterproductive. Think Petry would agree...

- If less math per task is needed (i.e. fewer floating point operations) then that's fine. Did anyone "clean-up" the math for v7? Because if they did and less math is required, it won't matter what anyone declares a task to be worth. Credit will be lower. As it should.

- But if "real" flops are the same or even increased, and you are introducing faster apps and you are seeing less credit for exactly the same kind of work... the question is why?

Because the only way I can imagine for that to happen (doesn't mean I'm right of course) is if we are supposedly running at "peak efficiency". And obviously in the real world this is impossible.