Average Credit Decreasing?

Message boards : Number crunching : Average Credit Decreasing?
Message board moderation

 To post messages, you must log in. "Oldest first Newest first Highest rated posts first

Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 33 · Next

AuthorMessage
Lionel

Joined: 25 Mar 00
Posts: 680
Credit: 563,640,304
RAC: 597
Message 1786588 - Posted: 11 May 2016, 5:33:15 UTC - in response to Message 1786508.

expect credit to go south again soon ...
ID: 1786588 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1786614 - Posted: 11 May 2016, 7:03:51 UTC - in response to Message 1786508.

CN*cobblestone_scale = CREDIT

But if CN is arbitrary then no matter what you multiply it with, the result will also be arbitrary.
...

True. It *appears* and is Ã¡rbitrary (actually chaotic), because of the faults.
expanding the CN part:
peak_flops*elapsedseconds*pfc_scale*cobblestone_scale

- 1st term peak_flops: (Sticking point for later)
- 2nd term elapsedseconds, measured, is the important one, because the whole purpose is to automatically adjust the second term to predict how long your next tasks will be.
- 3rd term, pfc_scale, and its inverse 1/pfc_scale used for prediction, are misnamed perhaps unintentionally, will come back to it (prediction is important for scheduling)
- cobblestone_scale. Only a unitlesss conversion from one scale to another (measuring stick), could easily be banana_scale.

so the only term here that is truly arbitrary/chaotic, is the scaling term, that is supposed to compensate for peak_flops being all over the map, and natural variation in elapsedseconds

It could just as easily be:
CN = Credit

And "Credit" could be anything. Anything EXCEPT Cobblestones ;)

Exactly, you can award Virtual-Tacos if you want, however we still need to be able to estimate how long tasks will take (obviously broken in various situations)

Back to 'Ã­s it fixable?'. I contend yes. Here is why:
You can look at your task history and predict how long future tasks will take --> Humans are good at that, it's called 'Ã©stimate localisation'

pfc_scale, because normalisation to a reference application is a step, should have been named 'relative_efficiency'. With only 1 application, the credit claim boils down to

claim*cobblestone_scale

We saw that on Albert@home tests, precisely as you say, the same amount as flopcounting (There are natural instabilities in elapsed_seconds, OK, but not that much, noise can be smoothed with damping if you know what you're doing, humans also do that with their knees while riding a skateboard.) Let's put the damping/stability aside as very fixable.

So accepting (with a stretch) that projects using creditNew understand their application well enough to put a reasonable #operations on their tasks to start with (There are formal techniques and it's not actually mutable), Where's the problem ?

The main (scaling) problem is with how that peak_flops is yet another misnamed number, misused.
For CPU it's Boinc_Whetstone (well disguised)
For GPU it's marketing GFlops

Now comes the tricky bit. Normalisation to the reference application, when there is only one application, shows 'Ã§orrect' credit and 'reasonable' estimates (ignoring that noise/damping issue), as our experiments on Albert indicated.

on Windows/Linux-Intel/Mac-Intel Boinc_Whetstone is flat out WRONG, too low when you have SSE through AVX engaged. Now we have different hosts with different features being normalised to an increasing number of results claiming a fraction of Actual

The estimates come too short, are unstable depending on what hosts are returning work, bounds periodically get exceeded (indicating breakages in teh system), and arguably least important you get a fraction of the credit intended.

The fix ? Well the server side knows your CPU features, the project hopefully knows for the reference application(s) what instruction sets they enabled.

We then know a rule of thumb to determine how much boinc Whetstone underclaims for a given instruction set ( Sisoft_SIMD_Whetstone/Boinc_Whetsone )

At least for 'reasonable' enough correction (as Humans do, and all we need for prediction/estimation purposes) you should find on your hosts, Sisoft SIMD singleThreaded Whetstone is approximately 2.5 to 5x the Boinc Whetstone figure.

Funny part is, on the larger scale (many results/hosts), the 2x slop in that will even cancel out by the fact that some hosts would be on the high side, while others on the low.

The only reasons it won't self-correct without compensation, are that first FPU-only applications and hosts don't represent enough numbers of results (if any anymore) compared to the massive #of underclaiming SSE-AVX results.

Address that discrepancy ('Ã“ptimising' Boinc Whetstone won't cut it), and add some damping to stabilise, and it will dial in as intended (i.e. estimates).

Homework:
A) Run Sisoft highest SIMD single threaded benchmark, and divide that by Boinc Whetstone on your host, Multiply your RAC by that, that's cobblestone_scale award +/-noise. should be ~2.5-5x or so (Mine's 3.3x)

B) work out all the factors that affect reported elapsed_time in your results, and how quickly responding estimates need to adapt for practical purposes/change, while not bouncing around all over the shop giving meaningless noise. (not quite as subjective as it sounds)

C) if you have a smartphone, open up Googlemaps and marvel (lol) at how relativistic estimate localisation through sensor fusion and statistically based controllers is done right (with GPS and GLONASS).
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1786614 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1786618 - Posted: 11 May 2016, 7:32:18 UTC - in response to Message 1786588.

expect credit to go south again soon ...

Lol, gave me an idea for a principle:

Credit is inversely proportional to the wonkiness of Boinc's estimates.
(which get more wonky every time an old host dies or new more efficient one comes online)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1786618 ·
Richard Haselgrove
Volunteer tester

Joined: 4 Jul 99
Posts: 14474
Credit: 200,643,578
RAC: 874
Message 1786627 - Posted: 11 May 2016, 8:42:48 UTC - in response to Message 1786614.

True. It *appears* and is Ã¡rbitrary (actually chaotic), because of the faults.

Hold that thought. "Chaotic" is a good scientific (mathematical) word. Just don't call it random in David's hearing (he's touchy about that).
ID: 1786627 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1786628 - Posted: 11 May 2016, 8:44:24 UTC - in response to Message 1786627.

True. It *appears* and is Ã¡rbitrary (actually chaotic), because of the faults.

Hold that thought. "Chaotic" is a good scientific (mathematical) word. Just don't call it random in David's hearing (he's touchy about that).

Agreed. Chaotic is not the same as Random, It is NOT random.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1786628 ·
Richard Haselgrove
Volunteer tester

Joined: 4 Jul 99
Posts: 14474
Credit: 200,643,578
RAC: 874
Message 1786636 - Posted: 11 May 2016, 9:06:53 UTC - in response to Message 1786628.

True. It *appears* and is Ã¡rbitrary (actually chaotic), because of the faults.

Hold that thought. "Chaotic" is a good scientific (mathematical) word. Just don't call it random in David's hearing (he's touchy about that).

Agreed. Chaotic is not the same as Random, It is NOT random.

Om. Chant after me: "Chaos is not random".
ID: 1786636 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1786646 - Posted: 11 May 2016, 9:41:55 UTC - in response to Message 1786636.
Last modified: 11 May 2016, 9:42:49 UTC

True. It *appears* and is Ã¡rbitrary (actually chaotic), because of the faults.

Hold that thought. "Chaotic" is a good scientific (mathematical) word. Just don't call it random in David's hearing (he's touchy about that).

Agreed. Chaotic is not the same as Random, It is NOT random.

Om. Chant after me: "Chaos is not random".

A wise puppet, (Agro Vation, my Avatar), once said: "Beauty is only skin deep, but ugly goes right through to the bone."
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1786646 ·
William
Volunteer tester

Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1786647 - Posted: 11 May 2016, 9:45:37 UTC - in response to Message 1786627.

True. It *appears* and is Ã¡rbitrary (actually chaotic), because of the faults.

Hold that thought. "Chaotic" is a good scientific (mathematical) word. Just don't call it random in David's hearing (he's touchy about that).

'Three is Chaos'

No it's not random, but the code amounts to fairly unpredictable credit awards (though as for any chaotic system, there are patterns [*]) and to the layman unpredictable is pretty much the same as random. We say chaotic, when we know you could calculate it.
'random' is a word we use when there don't seem to be any rules. Very few things are truly random. Blame physics for that ;)

[*] and if you were in possession of every single variable used in CN you could indeed calculate how much credit you'll be getting. CN IS a calculation after all.
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1786647 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1786649 - Posted: 11 May 2016, 9:53:25 UTC - in response to Message 1786647.
Last modified: 11 May 2016, 10:04:46 UTC

Well that's just the difference between transferring the (awkward) theory, from formulae to practice.

Properly engineered solutions are a marriage of form and function, the synthetic (theory) and the aesthetic (simplicity, proportion and beauty), crafting an optimal synthetic-aesthetic, i.e. 'State of the Art', recognised since the renaissance.

[Edit:] leading to the aerospace design rule of thumb --> if it looks wrong, it probably is wrong.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1786649 ·
shizaru
Volunteer tester

Joined: 14 Jun 04
Posts: 1130
Credit: 1,967,904
RAC: 0
Message 1786949 - Posted: 12 May 2016, 8:52:45 UTC

BTW can anybody put some real numbers into the Creditnew equation? I'd like to play around these days but loathe to work with just letters and words. Feels like 10x the effort (at least for the way my own brain is wired).

Anyway, can anybody create a template? Might even be a good idea to have links on the CN Wiki page directing to real world examples of a Seti task and other Boinc projects tasks... dunno.

William? Ageless? Jason? Anybody?

ID: 1786949 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1786951 - Posted: 12 May 2016, 8:55:41 UTC - in response to Message 1786949.
Last modified: 12 May 2016, 8:57:09 UTC

Got a spreadsheet somewhere on Googledocs for analysing Albert data [Edit: may have been seti-beta CPU, will see], Will dig it out on the weekend.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1786951 ·
William
Volunteer tester

Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1786958 - Posted: 12 May 2016, 9:11:26 UTC - in response to Message 1786951.
Last modified: 12 May 2016, 9:18:22 UTC

Got a spreadsheet somewhere on Googledocs for analysing Albert data [Edit: may have been seti-beta CPU, will see], Will dig it out on the weekend.

Data from Eve, now sadly demised. Beta iirc, only place that keeps results almost indefinitely and is best if you need to compare stuff.

I'll rummage.

edit: on second thought, you may have a spreadsheet, I've got the original data... somewhere. dang.
edit 2: I did find the spreadsheet though. I can't find the original graph on the cloud. Must be somewhere though...
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1786958 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1786964 - Posted: 12 May 2016, 9:28:57 UTC - in response to Message 1786958.

Oh I definitely have the spreadsheet, just going to have to wait until the weekend. Yeah, Eve's Seti-Beta credits for shorties-only, versus Correct credit driven with controllers, versus /3 to compare spikiness/instabilities.

Will have to find it, but IIRC the cool observation was some really obvious self similar (non-identical repeating) patterns, which are evidence of undamped chaotic behaviour (Seems so much easier to explain after time)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1786964 ·
Ulrich Metzner
Volunteer tester

Joined: 3 Jul 02
Posts: 1256
Credit: 13,565,513
RAC: 13
Message 1787123 - Posted: 12 May 2016, 22:38:27 UTC

Ok, i'm done with this futile discussions...
I'm off to milkyway, see you again, if this mess is sorted out...
Aloha, Uli

ID: 1787123 ·
ML1
Volunteer moderator
Volunteer tester

Joined: 25 Nov 01
Posts: 16011
Credit: 7,508,002
RAC: 20
Message 1787137 - Posted: 12 May 2016, 23:36:13 UTC - in response to Message 1786951.
Last modified: 12 May 2016, 23:39:37 UTC

Got a spreadsheet somewhere...

The source problem is that we are not measuring reality, or even a consistent consistently measurable abstraction of anything tangibly real. The "Cobblestone" was a useful but imperfect measure for compute-intensive projects. However, since then, we have other considerations and also even vastly differing measures of compute rates between old x86-FPU, SIMD, GPU, and others...

... Try following some of the ideas of NIST calibration to get real?...

(Note, this is a very old discussion...)

Myself, I favor NIST-style calibration to award for transistor-transitions. That should work well for for both compute intensive and network intensive tasks until we move into quantum computing...

Happy crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1787137 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1787146 - Posted: 13 May 2016, 0:17:59 UTC - in response to Message 1787137.

Let's be clear, for estimates that drive the whole thing, Don't need perfect precision here. +/-10% prediction from measured runtime would be excellent compared to the wackiness now in play.

There's a difference between having a pet hamster and petting it, and petting it so much you strangle it to death.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1787146 ·
Brent Norman
Volunteer tester

Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Message 1787149 - Posted: 13 May 2016, 0:43:55 UTC

I'm still taking bets as to the date we get our RAC down to half of what it should be.
ID: 1787149 ·
jason_gee
Volunteer developer
Volunteer tester

Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Message 1787152 - Posted: 13 May 2016, 0:47:47 UTC - in response to Message 1787149.
Last modified: 13 May 2016, 0:51:01 UTC

I'm still taking bets as to the date we get our RAC down to half of what it should be.

It's been around 1/3.3 +/- since creditNew introduction, in two main downward jumps. (Initial, then stock CPU AVX optimisation)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1787152 ·
uglybiker
Volunteer tester

Joined: 6 Dec 02
Posts: 32
Credit: 11,417,951
RAC: 42
Message 1787170 - Posted: 13 May 2016, 1:53:55 UTC

I just look at my credit and start singin' like the Boss.

I'm goin' down, down, down, down
I'm goin' down, down, down, down.....
ID: 1787170 ·
Brent Norman
Volunteer tester

Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Message 1787176 - Posted: 13 May 2016, 2:01:20 UTC - in response to Message 1787170.

Follow that song with Tom Petty's song Free Fallin'
ID: 1787176 ·
Previous · 1 . . . 12 · 13 · 14 · 15 · 16 · 17 · 18 . . . 33 · Next

Message boards : Number crunching : Average Credit Decreasing?

©2022 University of California

SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.