Fourth Result Wasted

Message boards : Number crunching : Fourth Result Wasted
Message board moderation

To post messages, you must log in.

AuthorMessage
john_morriss
Avatar

Send message
Joined: 5 Nov 99
Posts: 72
Credit: 1,969,221
RAC: 48
Canada
Message 229665 - Posted: 11 Jan 2006, 21:01:05 UTC

If I look at the WU results for my computer and see that I'm often the fourth result returned (but within deadline), does that mean that my result on those WUs is not part of the scientific result? Note: This is a doing science question, not an earning credits question!
ID: 229665 · Report as offensive
Profile Pooh Bear 27
Volunteer tester
Avatar

Send message
Joined: 14 Jul 03
Posts: 3224
Credit: 4,603,826
RAC: 0
United States
Message 229672 - Posted: 11 Jan 2006, 21:11:49 UTC

It looks like you are one of those who keep a large cache. So, you are going to naturally be one of the later ones to send results back. I keep my cache low and usually am one of the first couple that send it back.

I do not think it a waste that they send out 4, and do validating at 3. If someone's machine goes down, rogue, or any other number of issues, then only 3 of the 4 results will be returned. It was determined that by sending 4 and only needing 3 a good percentage of the WUs were validated after the first batch. When only 3 were sent, a lot of resending was done.

So, for the science, they are being safe.



My movie https://vimeo.com/manage/videos/502242
ID: 229672 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 229674 - Posted: 11 Jan 2006, 21:13:34 UTC

Think of it this way. The 4th result is an insurance policy.


BOINC WIKI
ID: 229674 · Report as offensive
john_morriss
Avatar

Send message
Joined: 5 Nov 99
Posts: 72
Credit: 1,969,221
RAC: 48
Canada
Message 229686 - Posted: 11 Jan 2006, 21:28:13 UTC - in response to Message 229674.  

Think of it this way. The 4th result is an insurance policy.


Good points, except I hate paying insurance, let alone being insurance!

I'm trying to feed a 30 WU/day habit through a slow dial-up feed, so the large queue was, I thought, a wise precaution.

And if I shorten my queue and speed up my turn-around time, it would only be to turn whoever would have been #3 into road-kill. And my low-ball Credit Claim will tend to move the Credit Granted to the next lower value, for everyone. So then they will... Hmm... The interactions here are more complex than I thought...
ID: 229686 · Report as offensive
Profile Scary Capitalist
Avatar

Send message
Joined: 21 May 01
Posts: 7404
Credit: 97,085
RAC: 0
United States
Message 229713 - Posted: 11 Jan 2006, 22:01:56 UTC

Think of it this way. If you weren't around to do that 4th work unit it would have been sent to someone else to do. That ties up another machine. Regardless of whether your unit is one of the first 3 you are still getting that unit out of the way so another machine can move on to something else. In a sense, logically, you can ignore whether it is the 3rd or 4th to be completed. They've all got to get sent out to somebody so you are doing as much of that process as anyone else.
Founder of BOINC team Objectivists. Oh the humanity! Rational people crunching data!
I did NOT authorize this belly writing!

ID: 229713 · Report as offensive
john_morriss
Avatar

Send message
Joined: 5 Nov 99
Posts: 72
Credit: 1,969,221
RAC: 48
Canada
Message 229750 - Posted: 11 Jan 2006, 22:29:02 UTC - in response to Message 229713.  

Think of it this way. If you weren't around to do that 4th work unit it would have been sent to someone else to do. That ties up another machine. Regardless of whether your unit is one of the first 3 you are still getting that unit out of the way so another machine can move on to something else. In a sense, logically, you can ignore whether it is the 3rd or 4th to be completed. They've all got to get sent out to somebody so you are doing as much of that process as anyone else.



Logic be damned... :)


ID: 229750 · Report as offensive
Scarecrow

Send message
Joined: 15 Jul 00
Posts: 4520
Credit: 486,601
RAC: 0
United States
Message 229768 - Posted: 11 Jan 2006, 22:53:06 UTC - in response to Message 229665.  

Note: This is a doing science question, not an earning credits question!


As you can see, there's a lot of reasons for the 4th result, and no matter what, it's a necessary evil. But no matter how you explain it, in the situation where the 4th result comes in and the first 3 have credits granted and the canonical result is in, all that 4th result is good for is credits for the host. If a person didn't care about the credits he could abort that workunit prematurely and as far as I can tell, it wouldn't make one bit of difference, as far as 'the science' is concerned, if good ol' #4 ever showed up at all.



ID: 229768 · Report as offensive
Jack Gulley

Send message
Joined: 4 Mar 03
Posts: 423
Credit: 526,566
RAC: 0
United States
Message 229778 - Posted: 11 Jan 2006, 23:12:37 UTC - in response to Message 229768.  

Note: This is a doing science question, not an earning credits question!

If a person didn't care about the credits he could abort that workunit prematurely and as far as I can tell, it wouldn't make one bit of difference, as far as 'the science' is concerned, if good ol' #4 ever showed up at all.

The bad thing is this causes the WU and its results to hang around in the data base taking up space and bloating it (making it a little slower) for two weeks until the aborted result is over due.

And of course aborting a WU before it has started crunching has little or no effect on how many Credits you earn and wastes little of your processing time.
ID: 229778 · Report as offensive
Profile David@home
Volunteer tester
Avatar

Send message
Joined: 16 Jan 03
Posts: 755
Credit: 5,040,916
RAC: 28
United Kingdom
Message 229816 - Posted: 11 Jan 2006, 23:49:57 UTC
Last modified: 11 Jan 2006, 23:50:41 UTC

A good question from the science angle. I gave up on Einstein@home for this very reason. In Einstein@home (but not SETI@home) the WU is actually different computations on the same data file. The data file (large file) is sent out once and then when new work is requested Einstein@home WUs are new calculations based on this data file. This is repeated until all calculations are done and a new data file is sent.

The issue I had with Einstein is that the WUs associated with a data file are all done by the same four PCs. So if you are the slowest PC of the four and return the result last (fourth) you are stuck in a downward cycle as you will not be sent the next WU until you return a result. So you start the next WU later than the faster PCs as they were sent the WU earlier. I noticed that in Einstein I was not being sent a WU until several days after it was initially sent out to the fastest PC so I could never be anything other than the fourth PC to return. I got the credit OK but I did wonder whether the science result was being used as quorom had already been met.

I guess from SETI@home angle the big question is if you are the fourth person to return a WU, and that WU happened to be a WU that contained a signal that was interesting would you be listed as one of the finders when the next list of interesting candidates is produced? If so then all is fine.

ID: 229816 · Report as offensive
Profile Landroval

Send message
Joined: 7 Oct 01
Posts: 188
Credit: 2,098,881
RAC: 1
United States
Message 229820 - Posted: 11 Jan 2006, 23:56:22 UTC - in response to Message 229816.  

The issue I had with Einstein is that the WUs associated with a data file are all done by the same four PCs. So if you are the slowest PC of the four and return the result last (fourth) you are stuck in a downward cycle as you will not be sent the next WU until you return a result.

Einstein seems to have stopped sending out the 4th unit unless it's needed. My outstanding Einstein units only have a 4th unit if there was a problem--downloading error, client error on computation, etc. On the pending ones, it looks like only 3 were sent out and validation is still done on 3.

Happy crunching,
Brian

If you think education is expensive, try ignorance.
ID: 229820 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 229834 - Posted: 12 Jan 2006, 0:11:06 UTC

When it was decided to send an initial 4th result there were between 20 and 25% errors/no returns/late returns/invalid. Thus one in 4 results was incorrect in some manner. I have heard that the test will be done again once enhanced is moved over and it will be decided at that point whether the 4th is still needed.


BOINC WIKI
ID: 229834 · Report as offensive
Dave Mickey

Send message
Joined: 19 Oct 99
Posts: 178
Credit: 11,122,965
RAC: 0
United States
Message 229899 - Posted: 12 Jan 2006, 2:00:11 UTC

I think there's another perspective that applies as well,
in terms of deciding if the 4th result is wasted.

If I understand the process, when the 3 results achieve
validation, then one of those (which one, not sure) is blessed
with the mantle of being "The Canonical Result". I think
this means that the analysis data of that machine's output
is deemed to be "The" actual, sole, result for that
computation, and is the only thing that is moved to the
science DB (or wherever SETI truth is stored).

Results 2 and 3 are just insurance against the canonical
result being wrong. (Once again, not necessarily the
second and third received, but just generally, 2 of the 3).

This leads me to the conclusion that, in terms of science,
results 2 and 3 are indeed discarded as well as the 4th,
so I'm just as happy returning the first, second, third, or
fourth result. Whatever!

But maybe I'm wrong, too!

Dave
ID: 229899 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 229910 - Posted: 12 Jan 2006, 2:14:19 UTC - in response to Message 229899.  

I think there's another perspective that applies as well,
in terms of deciding if the 4th result is wasted.

If I understand the process, when the 3 results achieve
validation, then one of those (which one, not sure) is blessed
with the mantle of being "The Canonical Result". I think
this means that the analysis data of that machine's output
is deemed to be "The" actual, sole, result for that
computation, and is the only thing that is moved to the
science DB (or wherever SETI truth is stored).

Results 2 and 3 are just insurance against the canonical
result being wrong. (Once again, not necessarily the
second and third received, but just generally, 2 of the 3).

This leads me to the conclusion that, in terms of science,
results 2 and 3 are indeed discarded as well as the 4th,
so I'm just as happy returning the first, second, third, or
fourth result. Whatever!

But maybe I'm wrong, too!

Dave

Yes, one of the results becomes the cannonical result and is moved to the science DB.

Apparently the validator is somewhat complex. If any 2 are "strongly" similar, then the third does not have to validate in order to choose a cannonical result and grant credit. If three are "weakly" similar, then one of those is chosen as the cannonical result. If it takes the 4th one, it is already in the pipeline and should be there soon.

Having the 4th one in the pipeline helps if there are a large number of failures because it tends to keep the DB smaller. As a secondary effect, it cuts down on the whining.


BOINC WIKI
ID: 229910 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 230130 - Posted: 12 Jan 2006, 12:35:00 UTC - in response to Message 229910.  


Apparently the validator is somewhat complex. If any 2 are "strongly" similar, then the third does not have to validate in order to choose a cannonical result and grant credit. If three are "weakly" similar, then one of those is chosen as the cannonical result. If it takes the 4th one, it is already in the pipeline and should be there soon.


You currently needs atleast 3 "success"-results before validation is tried.

Validator checks till finds a pair of results to be "Strongly similar", and chooses one of the pair as "Canonical result". Wu is now Validated, and also triggers Assimilator.
If can't find a pair to be "Strongly similar", it's "no consensus yet" and wait for another "success"-result.

After wu validated, any remaining results is checked if "weakly similar" to the Canonical result. Appart for deciding crediting, it doesn't matter if none of these remaining results passes validation.


Testing for "weakly similar" is only done after wu has been validated by finding a pair of results to be "Strongly similar".
ID: 230130 · Report as offensive
Profile mikey
Volunteer tester
Avatar

Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 230293 - Posted: 12 Jan 2006, 22:00:38 UTC - in response to Message 229778.  

The bad thing is this causes the WU and its results to hang around in the data base taking up space and bloating it (making it a little slower) for two weeks until the aborted result is over due.
And of course aborting a WU before it has started crunching has little or no effect on how many Credits you earn and wastes little of your processing time.

This could be easily solved by deleting the 2 extra results and then auto deleitng the 4th when it comes in and just grant credit to the 4th. But apparently the database size is not currently that much of an issue. A small script could be written that says if unit is validated and credits are issued save result A delete results B and C, issue Y credits, and when result D comes in and is valid, issue Y credits and then delete result.

ID: 230293 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20283
Credit: 7,508,002
RAC: 20
United Kingdom
Message 230320 - Posted: 12 Jan 2006, 23:14:00 UTC - in response to Message 230293.  

This could be easily solved by deleting the 2 extra results and then auto deleitng the 4th when it comes in and just grant credit to the 4th...

Not so fast!

The results need to be compared to test that the results are valid. This gets a bit more complicated for when a number of results are "weakly similar" and then another result comes in inbetween that then makes them all acceptable... And there's other permutations there. Some clever logic would be needed at least and for little gain except increased complexity.

The wrecking cheaters would soon find the exploit if credit was granted blindly for any 3rd or 4th result...

Happy crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 230320 · Report as offensive
Profile Pooh Bear 27
Volunteer tester
Avatar

Send message
Joined: 14 Jul 03
Posts: 3224
Credit: 4,603,826
RAC: 0
United States
Message 230404 - Posted: 13 Jan 2006, 2:23:16 UTC

Remember this is for the science, not the credits. The credits are just a result of the science. They are doing things to make sure the science is valid. They do not want to play with their formulas.



My movie https://vimeo.com/manage/videos/502242
ID: 230404 · Report as offensive
Profile David@home
Volunteer tester
Avatar

Send message
Joined: 16 Jan 03
Posts: 755
Credit: 5,040,916
RAC: 28
United Kingdom
Message 231075 - Posted: 14 Jan 2006, 14:53:55 UTC - in response to Message 229910.  

[Yes, one of the results becomes the cannonical result and is moved to the science DB.



Thanks John, makes sense in terms of keeping the size of the science database under control. Is there also a record kept of the ids of the other 2 or 3 who crunched the same WU? Just that if it turns out to be an interesting WU for reobservation I would hope that all 3 or 4 users would be credited with crunching it and not just the owner of the cannonical result. I assume this is so but a reassurance would be cool.


ID: 231075 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 231312 - Posted: 15 Jan 2006, 1:50:37 UTC - in response to Message 231075.  

[Yes, one of the results becomes the cannonical result and is moved to the science DB.



Thanks John, makes sense in terms of keeping the size of the science database under control. Is there also a record kept of the ids of the other 2 or 3 who crunched the same WU? Just that if it turns out to be an interesting WU for reobservation I would hope that all 3 or 4 users would be credited with crunching it and not just the owner of the cannonical result. I assume this is so but a reassurance would be cool.


Sorry, I don't have an answer for this one.


BOINC WIKI
ID: 231312 · Report as offensive

Message boards : Number crunching : Fourth Result Wasted


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.