Hmmm, OK sine waves are not in themselves random, but one may sample the sine wave at a random interval and it can the appear to be a random number generator. When considering the way the credit system work it does sample at a random interval, thus the result of the request for data may appear to be a constrained random number.
Great, two people both "right" and "wrong" at the same time.
But the credit generator appears to have a couple of potential wave form generators running at once, and they will beat with each other and increase the apparent "randomness" of the observed.
And I forgot that there is an underlying process that is working to normalise everything. That is going to be next to impossible to resolve without some serious heavy lifting of the code - part of this is a bit of the so called "anti-cheat" mechanism, which tries to smoother step changes. This is rather insidious because it can't handle changes in hardware or improvements in the application very well, and nine times out of ten does so by very crudely increasing the apparent processing rate of the system, but it savagely overshoots, so reduces the "FLOPS count" for both the current task and all subsequent ones for the computer concerned.

Yes Richard, from the description of Joe's graph by Alex it does sound like Joe's "sine wave" is triggered by a PID overshoot (which may be an undershoot depending on the the trigger stimulus).

Just removing, or nulling out, the anti-cheat mechanism would not be a quick task, and may produce some unwanted artifacts because like most things to do with the credit calculation process it has tags and fragments in all sorts of places.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

But here's the problem. You cannot call a sawtooth/sinewave a "random number generator". A sinewave is not chaotic. I think I pointed out that it looks like CN is trying to award an overall average* and Jason kinda agreed... But after that, the whole thing was completely forgotten.

*Example: Say a task is worth 50.
CN was awarding 30, 70, 30, 70, 45, 55, 60, 40, 30, 70 etc.

That would be a result of the mix of work being processed. When it's the one type of Work, the Credit per WU falls (or rises again) to a general overall level (with the usual variations around that point). When there is a mix of work, the Credit allocated for a given WU will tend to rise or fall depending on the mix of work (mostly GBT, mostly Arecibo or 50/50).

---

Grant
Darwin NT

A good example of Credit New's issues.
Arecibo WU, non VLAR. Estimated computation size 20,763 GFOPS. Processing time, 2hrs 15min.
GBT WU. Estimated computation size 20,631 GFLOPS. Processing time, 1hr, 20min.

Judging by the processing times, I'm going to assume you're talking about CPU processing - and for the sake of it, let's assume the same CPU (and application) in each case.

Yes & yes.
Both WUs done on the same system CPU.

After several days of mostly GBT work, my CPU APR (Average Processing Rate) for that system has gone from around 22 GFLOPS to just over 37 GFLOPS.

---

Grant
Darwin NT

You do have two threads for this here, if you perhaps did not know.

If not thinking about CreditScrew as such here, but only about numbers, also that random numbers, like also the generation of such, should be about Number theory,
like also Polynomials, for much of the same, where a result might be impossible to determine in advance, like that of the larger RSA numbers.

Therefore, two sine curves in parallell, and you might get such a complete mess, where it also could be Order for that single thing only, like that of intertwined, or entangled,
next for still that mess.

Perhaps another excuse here when next also telling about a couple of fabulous stories.

Hi,

I find it odd that my TITAN V is constantly awarded below 100 credits when it does an Arecobo vlar. It does them in 75 seconds and
that my 1080 and 1080Ti get 110-140 credits of the same type work when they use 125 - 170 seconds.

Is it possible that screwit new thinks that 75 seconds is too little time, a fake or something?

Petri

---

To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones

I seem to remember someone saying, the faster you do work, the less you get paid. The thought is, eventually we will do work so fast, that the credit will be negative.

---

The weather has been amazing here so I've been mostly using up my free time for my chili pepper seedlings and trying in vain to exhaust my inexhaustible little daughter :)

I guess cobblestones will have to wait :)

I see Petri has handed out homework too ;)
I thought CN basically ignored GPUs and went on CPU info to award credit for Seti.
Doesn't seem to be the case here :(

Perti's machine is now showing a "more normal" 40-70ish range for all his GPUs. The previous high values certainly "look strange" at first, but are probably values obtained against uncalibrated devices or very slow returners (for some reason a lot of those return just after an outage or a public holiday)

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

I thought CN basically ignored GPUs and went on CPU info to award credit for Seti.

Well, sot of, but if only it were that simple....
There is a scaling factor in the calculation that is supposed to cope with GPUs, but it is VERY crude, and doesn't cope with very fast GPUs properly. Also the logic that triggers its use is also not very robust and will make errors frequently, again when the GPU concerned is a very fast one (as Petri's are).
What one has to remember is that much of both the logic and the maths was laid down at about the time when GPUs were just coming onto the market, and nobody had any knowledge of just how fast they would get - they are a LONG way ahead of the curve when compared with CPUs. Back then it was assumed that GPUs would be about 10 times faster than a CPU, but today we are seeing many GPUs more than 100 times faster then CPUs - and some may be even further ahead.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

I thought CN basically ignored GPUs and went on CPU info to award credit for Seti.

Well, sot of, but if only it were that simple....
There is a scaling factor in the calculation that is supposed to cope with GPUs, but it is VERY crude, and doesn't cope with very fast GPUs properly. Also the logic that triggers its use is also not very robust and will make errors frequently, again when the GPU concerned is a very fast one (as Petri's are).
What one has to remember is that much of both the logic and the maths was laid down at about the time when GPUs were just coming onto the market, and nobody had any knowledge of just how fast they would get - they are a LONG way ahead of the curve when compared with CPUs. Back then it was assumed that GPUs would be about 10 times faster than a CPU, but today we are seeing many GPUs more than 100 times faster then CPUs - and some may be even further ahead.

Thank you for your comment and for the information content too.
I'd say 100x is the new normal.

-- -- there are places here on our planet where the Sun does not go down <3 They are the very same places where the Sun does not come up when there is the deepest of the winter

---

To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones

How is the wu.fpops_est value for each WU determined? (The estimated computation size).
As this is used in the Normalisation part of Credit New, it has a significant impact on granted Credit.

Where I posted a FLOPS & runtime comparison earlier, I compared 2 different WUs of similar FLOPS estimates on the CPU.
Today I made the mistake at looking at WUs across 2 machines & different hardware (it'd be easier if i did this when we had all Arecibo or GBT, but I had a batch of similar WUs that have consistent runtimes to use).

I had a batch of 17my18 Arecibo WUs on both systems, work spread over both GPU & GPU.

On my 8700k/ GTX 750Ti system
CPU WU estimated FLOPS 18,411
GPU WU estimated FLOPS 41,473

on my 2600k/ GTX 1070 system
CPU WU estimated FLOPS 20,533
GPU WU estimated FLOPS 14,509

None of the compared WUs are the same WU, however their processing times were all within seconds of similar WUs crunched by that device on that system. So the work required to process each WU was (within a few seconds) effectively the same as the other WUs.
So why does each one have a such a vastly different Estimated Computation Size? A low of 14,500 to a high of 41,500 for WUs that all would take the same time (give or take a few seconds, even on CPU) to process on a given device?

The more powerful the GPU, the less FLOPs required to process. 14,509 v 41,473
The more powerful the CPU, the less FLOPs required to process. 18,411 v 20,533
The difference between CPU estimates is negligible. The difference between GPUs is extreme. All for WUs that would actually have the same runtimes on a given device. So imagine what the effect of the Normalisation & anti-cheating parts of Credit New would be for WUs with the same runtime, but such vastly different estimates for processing required. (as for the example I posted earlier where the estimates were the same, but the runtimes significantly different).

---

Grant
Darwin NT

The estimate is based on the user's CPU performance, then (poorly) scaled for GPUs. In the case of a GPU-only user the initial estimate is based on an idealised CPU, and scaled by an assumed fudge factor.

In theory, for a given cruncher, the system "learns" what GPU factors to apply. But in practice this bit can re-set itself, or go completely wild if anything comes along to disturb the semblance of equilibrium that has been achieved. For example if equilibrium was achieved on VLARS and suddenly a batch of normal units came along the fudge factor will probably crash through the floor (about 90% of the time), but it may go sky high.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

The estimate is based on the user's CPU performance, then (poorly) scaled for GPUs. In the case of a GPU-only user the initial estimate is based on an idealised CPU, and scaled by an assumed fudge factor.

In theory, for a given cruncher, the system "learns" what GPU factors to apply. But in practice this bit can re-set itself, or go completely wild if anything comes along to disturb the semblance of equilibrium that has been achieved. For example if equilibrium was achieved on VLARS and suddenly a batch of normal units came along the fudge factor will probably crash through the floor (about 90% of the time), but it may go sky high.

Are we talking about the same thing here?
The number of FLOPS it takes to process a particular WU would be dependent on it's Angle Range. A Very Low Angle Range, lots of processing (FLOPS) required. A Very High Angle Range, barely any processing (FLOPS) required (in comparison). What processes it doesn't matter. The AR for a WU won't change depending on what is processing it, so the Estimated FLOPS for that WU won't change depending on what is processing it either.

What processes it makes no difference to the work that needs to be done. Faster hardware and/or a more efficient application will do the work faster, but the actual work to be is unchanged. The AR & hence the FLOPS for a given WU should be the same, regardless of what device is processing it.

---

Grant
Darwin NT

The AR is a fixed property of the work unit, however the FPOPS is dependent on the cruncher (sort of) as well as the AR - this is one of the worst features of CreidtNew.
Each individual cruncher has its own scaling factors for GPUs these are almost all total guess work, and the algorithm to "correct" them is very unstable, and will tend to initially grossly under estimate the performance, then try to compensate, and overshoot, do that a few times (assuming the incoming work units remain "of the same type") before more or less settling down. Change the type of incoming work unit (in the worst case even the part of the range within a given type) and we go back to wild osculations. Rinse, repeat......

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Note that there are two separate and distinct mechanisms in use, depending on whether you have declared your own applications - 'Anonymous Platform', app_info.xml, Lunatics or whatever - or whether you allow the server to send and manage the applications you use - the vast majority of project users, though possibly a minority of regular posters on these boards, work this way.

The 'stock' (server application) case is easier to grasp. Each WU has a fixed size, calculated by the splitter from the AR. That's expressed in fpops, and will be the same every time for WUs of the same AR, such as the blc vlars. If you study <rsc_fpops_est>, you'll know exactly how big tasks of each AR have been assessed to be - that's a real figure. In the meantime, the server is also keeping track of how fast your machine has been processing tasks recently - the APR - and passes back that number to your client every time new work is allocated. It's the fixed task size, and the varying speed, which your client uses to estimate how long each task is going to take.

In the anonymous platform case, which most people reading this thread will be using, the roles are reversed. You machine will be assumed to have a fixed speed: you can declare that yourself in app_info.xml (don't bother), or an arbitrary number will be used. The server still keeps track of APR, but doesn't pass it directly back to the client as an estimated speed. Instead, it tweaks the estimated size - the fpops - of each task as it's allocated to you: tasks allocated this morning may have different fpops from tasks allocated last night. fpops is no longer a real figure - it'll vary from machine to machine, and from time to time, even for resends of the same WU.

Anyone who genuinely wants to understand CreditNew and study how it works in the real world should probably keep at least one machine running in stock application mode. One obvious difference: because task sizes are fixed, and APR varies, any drift in APR will change the runtime estimation of every task in your cache. If you run a batch of slow tasks, APR will fall, estimated runtime will rise, and your machine will fetch work less often because it'll think that the cache is fuller. Anonymous platform machines will fetch work more steadily, because the runtime estimate is fixed (by the task size tweak) when the work is first allocated, and doesn't change thereafter.

(Reference: RuntimeEstimation)

The AR is a fixed property of the work unit, however the FPOPS is dependent on the cruncher (sort of) as well as the AR - this is one of the worst features of CreidtNew.

Seriously?
It's insane IMHO. Absolute madness if it's true.
Iteration is all well & good for solving equations & playing with Fractals, but to use it in this way...

Even though I don't agree with the hardware/ application normalisation, particularly the (perceived) efficiency penalisation, and the anti-cheating implementations, I sort of (in a very vague way) understand them. But to change the estimate for the work to be done, depending on the type of device doing the processing is just insane.
How on Earth can it be possible to correctly determine how much work was actually done, whether or not a person is cheating, or how efficient or not a device is, if the estimate for the work to be done isn't a set value for a given WU??? And if that value varies, depending on the result of the processing done using the previous (variable) value?????
Well &^%$ me.


If that is the case, the best thing that could be done would be to make the wu.fpops_est value for each WU set by (i'm guessing) the splitters or Scheduler and don't change it.
It has a particular AR, so it should take this number of Floating Point Operations to process it. That doesn't vary depending on the device processing it, so that value shouldn't vary.


Richard posted a link to work Josef Segur did previously relating to estimated and actual crunching times compared to AR and the work done to normalize them.

Ideally on the reference CPU with the reference application it would take x seconds to process a WU of a particular AR (estimated FLOPs). As Joe showed, In real life the processing time can increase or decrease significantly at certain ARs. So some sort of normalisation is necessary.
So that- a WU that has twice the estimated FLOPs should take twice as long to process. A WU with half the estimated FLOPs, half the time to process. For that particular reference CPU & application that takes x seconds to process, you should get y Credits. Twice as long to process, 2*y. Half as long to process, 1/2*y. So it doesn't matter how long it takes to process a given WU, you will still get the same amount of Credit per hour regardless of whether your are crunching all shorties, or all VLARs on that reference CPU with that reference application. (I can't see getting this exactly right being possible, but from what I remember from the previous Credit system, it can be damn close).
Of course processing work on a different CPU, or using a different application, or both, or on a GPU, with different applications, will result in differing runtimes, but the Credit paid for a given WU will be the same, regardless of the hardware or software used to process it.

---

Grant
Darwin NT

The AR is a fixed property of the work unit, however the FPOPS is dependent on the cruncher (sort of) as well as the AR - this is one of the worst features of CreidtNew.

Seriously?
It's insane IMHO. Absolute madness if it's true.

Remember that all the work - keeping track of fpops, speed, and credit - is done on the server, and the server holds - and uses - the absolute fixed fpops values for the WU, calculated from Joe's curve.

The 'tweaked' fpops value sent to your client is used for one purpose only - to show you a plausible runtime estimate. It plays no other part in the system.

In the anonymous platform case,....
The server still keeps track of APR, but doesn't pass it directly back to the client as an estimated speed. Instead, it tweaks the estimated size - the fpops - of each task as it's allocated to you:

Ok.

And looking at the reference Richard linked to,

However, the only way to change the client's runtime estimate is by adjusting the wu.rsc_fpops_est that we send to the client.

And the problem is that wu.rsc_fpops_est is used in determining, well, pretty much everything when it comes to granting Credit.
My head hurts,

---

Grant
Darwin NT

Remember that all the work - keeping track of fpops, speed, and credit - is done on the server, and the server holds - and uses - the absolute fixed fpops values for the WU, calculated from Joe's curve.

The 'tweaked' fpops value sent to your client is used for one purpose only - to show you a plausible runtime estimate. It plays no other part in the system.

Sorry, the way I read it is that the wu.fpops_est is used in the granting of credit.
That it is how much work is expected to be done to process a given WU.
How long a WU takes to process (runtime), and how long it is expected to take to process (wu.fpops_est), and how powerful the device processing the WU is (peak FLOPS) all contribute to the final determination of the Credit granted through the efficiency & normalization routines.



EDIT- I just saw your next post after posting this post-

rsc_fpops_est_real is used on the server for every important calculation.
rsc_fpops_est_tweaked is used on the client (only in the non-stock case), and doesn't matter.

So when the finished WU is returned, the rsc_fpops_est_tweaked isn't sent back. It's ignored completely?

OK.
That makes more sense than having Credit New being iterative.

---

Grant
Darwin NT

My head hurts,

Welcome to the club!

rsc_fpops_est_real is used on the server for every important calculation.

rsc_fpops_est_tweaked is used on the client (only in the non-stock case), and doesn't matter.