Just a SETI Classic observation

Message boards : Number crunching : Just a SETI Classic observation
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 157
Credit: 10,720,947
RAC: 362
United Kingdom
Message 12673 - Posted: 27 Jul 2004, 10:34:50 UTC
Last modified: 27 Jul 2004, 10:46:13 UTC

Re: SETI Classic

The current (as at: Sat Jul 24 21:30:09 2004) stats indicate that there have been:

Results received: 1,479,376,371

(taken from here)

and yet according to this:

Data collection
Last updated: Sat Jul 24 23:11:08 2004

(taken from here)

-quote
"So far, SETI@home has generated 209,750,143 workunits from the data received from Arecibo."
-end quote


Pardon me for my bad maths, but does that mean that "on average", every returned WU has been crunched around 7 times. (ie 1,479,376,371 divided by 209,750,143).

If this is so, then there's a hell of a lot of redundancy....!

Likewise, if you look here then it would seem that there's been many days during the last 5 years when no worthwhile data has been available to convert into WU's.

Does all this make the whole SETI project seem fairly pointless (in comparison to say, 24/7 operation from whatever telescopes are available - whatever happened about using Parkes?), plus we'd have far more WU's being generated and a greater chance of finding something - after all, isn't that why we're all doing this - to find an ET signal or two?


Most of the original premise was to help find ET by using spare CPU cycles on many internet connected PC's and because the science was too great to be accomplished by the scientists alone that using extra help from PC's being donated to the project would greatly help the project.

But on interrogating the above data, it would seem that SETI Classic is now simply "spinning it's wheels" and going nowhere...!

(PS: Am aware that BOINC might get around this - although I've already received quite a few _4 and _5 WU's - implying these have already been sent out at least 5 or 6 times already...!)

Reasoned views/comments please.


ID: 12673 · Report as offensive
STE\/E
Volunteer tester

Send message
Joined: 29 Mar 03
Posts: 1137
Credit: 5,334,063
RAC: 0
United States
Message 12679 - Posted: 27 Jul 2004, 11:11:45 UTC

(PS: Am aware that BOINC might get around this - although I've already received quite a few _4 and _5 WU's - implying these have already been sent out at least 5 or 6 times already...!)
==========

I got 1 WU last night with a _7 on the end of it...Each work unit will be sent out 3 times originally, then if 3 results are not returned by the deadline for that work unit it will be sent out again. I don't know if it will be sent out 3 more times or just be sent out 1 more time until somebody returns a result for that WU.
ID: 12679 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 157
Credit: 10,720,947
RAC: 362
United Kingdom
Message 12684 - Posted: 27 Jul 2004, 11:22:01 UTC - in response to Message 12679.  


>
> I got 1 WU last night with a _7 on the end of it...Each work unit will be sent
> out 3 times originally, then if 3 results are not returned by the deadline for
> that work unit it will be sent out again. I don't know if it will be sent out
> 3 more times or just be sent out 1 more time until somebody returns a result
> for that WU.
>

I s'pose the problem the dev's have got is that:
- they want to have a quick turnaround time in order to provide the credits (and to keep crunchers happy that "somethings happening").

- they cannot afford a 28 day or longer "deadline" timescale, as crunchers don't want to wait a month for credit - SETI Classic gave the credit immediately upon completion.

- the credit system requires that 3 units are returned. Personally I think that that doesn't help anybody, except the project, as they are only trying to reduce the risk of cheating. But, so far as I can tell, you cannot cheat BOINC and it's computations.


So, we have the "send 3 units and then wait" approach. Which is fine as long as crunchers continue to get WU's and don't go off in a huff because there isn't any work to do, leaving un-finished/finished work sitting in a cache somewhere.

Tim
ID: 12684 · Report as offensive
STE\/E
Volunteer tester

Send message
Joined: 29 Mar 03
Posts: 1137
Credit: 5,334,063
RAC: 0
United States
Message 12687 - Posted: 27 Jul 2004, 11:31:36 UTC
Last modified: 27 Jul 2004, 11:32:53 UTC

Well I figure a WU with a _7 on the end has been around for awhile, maybe as long 2 months already, if you figure a 2 week wait time before it's sent out again ... :/
ID: 12687 · Report as offensive
Bart Barenbrug

Send message
Joined: 7 Jul 04
Posts: 52
Credit: 337,401
RAC: 0
Netherlands
Message 12694 - Posted: 27 Jul 2004, 11:55:54 UTC - in response to Message 12684.  

> But, so far as I can tell, you cannot cheat BOINC and it's computations.

Why not? If someone really wants to: the source is available, so you can make a client that reports higher benchmarks scores or longer computation time etc. But that only counts towards claimed credit, and the median operation performed by the validator server-side will ensure that if the credit for one of the results for a unit has been "artifically" increased, the actual granted credit will be one of the others. So a cheater won't get rewarded. Unless very many people cheat and there are more than one "artifically increased" claimed credit scores for one result, but should that happen, more sanity checks on claimed credits can be put in the server-side software. The point is to prevent what so often happens: that a few bad apples ruin it for the rest. The median operation gaves reasonable protection against that.
ID: 12694 · Report as offensive
Profile UBT - Timbo
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 157
Credit: 10,720,947
RAC: 362
United Kingdom
Message 12696 - Posted: 27 Jul 2004, 12:04:00 UTC - in response to Message 12687.  
Last modified: 27 Jul 2004, 12:05:13 UTC

> Well I figure a WU with a _7 on the end has been around for awhile, maybe as
> long 2 months already, if you figure a 2 week wait time before it's sent out
> again ... :/
>

Of course, it could have been sent out once, not returned. 2 weeks later sent out and not returned again. 2 weeks later, etc, etc, etc,.

Am not sure how many people have joined up with BOINC so far since it went "active", but can be sure that many will be on dial-up and so there could be an almighty big "splurge" of returned WU's as time goes on, and as more and more WU's are made available - so encouraging signs for the future, if the data splitters can keep up with the demand.

But the sad truth is that there are still not enough WU's for the number of people actively working on SETI Classic and now SETI BOINC.

And I'd hate for such a good project to suffer due to it being over-subscribed.

Maybe the BOINC concept, of sharing CPU cycles across multiple projects, is a really good thing - with more projects coming online (Einstein@home sounds very interesting, for instance), so the pull down of SETI WU's will be more evenly distributed and hence there will be enough WU's to go around to everyone participating.

But don't hold your breath...!
ID: 12696 · Report as offensive
Profile Christopher Hauber
Avatar

Send message
Joined: 10 Feb 01
Posts: 196
Credit: 71,611
RAC: 0
United States
Message 12698 - Posted: 27 Jul 2004, 12:07:51 UTC - in response to Message 12687.  

I don't think it necessarily has to wait 2 weeks to be sent out again. Bad returned results will cause it to be sent sooner won't they? I don't know about the "unrecoverable" errors that occur sometimes when it can't access the slots properly (or other reasons I suppose), but I should hope that errors from workunits that are deleted from your computer without being processed for some reason are reported to Berkeley for redistribution. Even so I am sure that there will be workunits here and there that just stick around.

I was also under the impression that returned results were also checked for similarity, not just claimed credit. In order to cheat the system then, the same bogus results would have to be generated by three different computers. I suppose it is possible for that to happen, but it reduces the odds. But really it is stupid to do that anyway. The only reason I care about the numbers is to track my progress personally. Sending in bogus results won't even do that for you. And you don't get any kind of prize for the highest credit. All that is is bad science and those people who want to cheat should just go elsewhere.

Chris

> Well I figure a WU with a _7 on the end has been around for awhile, maybe as
> long 2 months already, if you figure a 2 week wait time before it's sent out
> again ... :/
>
ID: 12698 · Report as offensive
Profile Thierry Van Driessche
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3083
Credit: 150,096
RAC: 0
Belgium
Message 12699 - Posted: 27 Jul 2004, 12:08:53 UTC - in response to Message 12694.  
Last modified: 27 Jul 2004, 12:09:06 UTC

> The median operation gaves reasonable protection against that.

The granting of credits was a long discussion. In the Beta Test Phase, the granted credit was the lowest value of only 2 claimed credits.

Now it is the median one of 3 claimed credits.

I believe this is now a more fairly approach.

ID: 12699 · Report as offensive
Profile Christopher Hauber
Avatar

Send message
Joined: 10 Feb 01
Posts: 196
Credit: 71,611
RAC: 0
United States
Message 12702 - Posted: 27 Jul 2004, 12:14:38 UTC - in response to Message 12696.  

The Dev Team has said there will not always be enough work for everyone to continuously process. Computers are getting faster, more sky is being processed, and there will be a time when data just can't be recorded fast enough. That doesn't mean the project suffers or is over subscribed though. It just means they have a little extra power at their disposal. Every single one of us have more power at our disposal than we use or we wouldn't be able to process SETI. I don't see it as being any different. Their "computer" just meets their demand with some room to spare when needed.

And sharing that spare power with other projects could prove very useful as well like you said.

Chris

> But the sad truth is that there are still not enough WU's for the number of
> people actively working on SETI Classic and now SETI BOINC.
>
> And I'd hate for such a good project to suffer due to it being
> over-subscribed.
>
> Maybe the BOINC concept, of sharing CPU cycles across multiple projects, is a
> really good thing - with more projects coming online (Einstein@home sounds
> very interesting, for instance), so the pull down of SETI WU's will be more
> evenly distributed and hence there will be enough WU's to go around to
> everyone participating.
>

ID: 12702 · Report as offensive
Daniel Schaalma
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 297
Credit: 16,953,703
RAC: 0
United States
Message 12705 - Posted: 27 Jul 2004, 12:22:42 UTC

I think that a WU with a _7 at the end simply means it is part 7 of a very long work unit that has been split into many portions. I could be wrong though. This would be a good question for Rom...

Also, from what I read about BOINC, it will provide the opportunity to crunch data from OTHER radiotelescopes. Seti Classic is, by necessity, locked into Aricebo because the software was written specifically for it's scan time. Because BOINC sends out the proper client for each WU, it will be able to utilize other sites besides Aricebo. BOINC is SO much more flexible than Seti Classic.

Daniel.
ID: 12705 · Report as offensive
Profile Thierry Van Driessche
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3083
Credit: 150,096
RAC: 0
Belgium
Message 12706 - Posted: 27 Jul 2004, 12:26:43 UTC - in response to Message 12705.  
Last modified: 27 Jul 2004, 12:28:16 UTC

> I think that a WU with a _7 at the end simply means it is part 7 of a very
> long work unit that has been split into many portions. I could be wrong
> though. This would be a good question for Rom...

What I understood is one WU is send out first with _0, _1 and _2 the very first time. If then, results are not coming back at time or results are wrong for one reason or another, the numbers then becomes _3, _4 and so on.
ID: 12706 · Report as offensive
Daniel Schaalma
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 297
Credit: 16,953,703
RAC: 0
United States
Message 12708 - Posted: 27 Jul 2004, 12:30:49 UTC

Ok, thanks Thierry. I stand corrected
ID: 12708 · Report as offensive
STE\/E
Volunteer tester

Send message
Joined: 29 Mar 03
Posts: 1137
Credit: 5,334,063
RAC: 0
United States
Message 12709 - Posted: 27 Jul 2004, 12:33:45 UTC

I don't think it necessarily has to wait 2 weeks to be sent out again.
==========

Thats why I said maybe as long as, because I realize that there could be other reasons for it being sent out that many times ...
JOIN TEAM

ID: 12709 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 12728 - Posted: 27 Jul 2004, 13:48:55 UTC - in response to Message 12709.  

> I don't think it necessarily has to wait 2 weeks to be sent out again.
> ==========
>
> Thats why I said maybe as long as, because I realize that there could be other
> reasons for it being sent out that many times ...

It will wait two weeks to be sent out if it was never returned. This happens if the machine has trouble and the WU is deleted, or if a reset or detach occurs. It may or may not happen if there is a fatal error processing the WU. A new copy of the WU will be sent immediately if the verifier fails to validate the results as being the same, or if a result is returned with a processing error.

There has been some discussion of addin the ability to detach after all work is completed and reported. But this has not been implemented, and I am not certain that it is on the schedule.

ID: 12728 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 12753 - Posted: 27 Jul 2004, 14:31:50 UTC - in response to Message 12694.  

> > But, so far as I can tell, you cannot cheat BOINC and it's computations.
>
> Why not? If someone really wants to: the source is available, so you can make
> a client that reports higher benchmarks scores or longer computation time etc.
> But that only counts towards claimed credit, and the median operation
> performed by the validator server-side will ensure that if the credit for one
> of the results for a unit has been "artifically" increased, the actual granted
> credit will be one of the others. So a cheater won't get rewarded. Unless very
> many people cheat and there are more than one "artifically increased" claimed
> credit scores for one result, but should that happen, more sanity checks on
> claimed credits can be put in the server-side software. The point is to
> prevent what so often happens: that a few bad apples ruin it for the rest. The
> median operation gaves reasonable protection against that.

I am not sure I understand youir answer here. You start off by saying that you could cheat and end by describing some of the mechanisms that will prevent that.

Christoper had it correct when he indicated that the results in the quorum of results will be compared to ensure that they are "the same" (all of the results are within predefined limits of each other) before the quorum is "closed" and credit is granted. So, even if there is a group of cooperating cheaters they have no way to make artificial quorums in that there is tracibility of the WU issued so that cheaters cannot clone results.

So, even if they tinker with the base code it will not matter, the super quick results returned will be rejected because they do not "compare" as equivelent to the results returned by non-cheaters. Those results will be dropped and the quorum fillled by a reissued WU.

Long term, and I don't know if this is an included feature yet or not, is to "track" the percentages of computers that are returning bad results above a threashold to restrict use of that machine by constraining the numbers of WU issued. This additional test would not only eliminate cheaters but over-clockers that are returning bad results due to instabilities in the computers.
ID: 12753 · Report as offensive
Bart Barenbrug

Send message
Joined: 7 Jul 04
Posts: 52
Credit: 337,401
RAC: 0
Netherlands
Message 12781 - Posted: 27 Jul 2004, 15:39:39 UTC - in response to Message 12753.  

> I am not sure I understand youir answer here. You start off by saying that
> you could cheat and end by describing some of the mechanisms that will prevent
> that.

You're right: I wasn't clear on that. Overall, indeed cheating does not pay. It would be easy to make a client that claims more than it should, but the server will not grant it.
ID: 12781 · Report as offensive
Profile Christopher Hauber
Avatar

Send message
Joined: 10 Feb 01
Posts: 196
Credit: 71,611
RAC: 0
United States
Message 12787 - Posted: 27 Jul 2004, 16:02:51 UTC - in response to Message 12781.  

That said, I'd kind of like to confirm something then. When credit is granted, does it grant the mean or the median value?

Chris


> You're right: I wasn't clear on that. Overall, indeed cheating does not pay.
> It would be easy to make a client that claims more than it should, but
> the server will not grant it.
>
ID: 12787 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 12803 - Posted: 27 Jul 2004, 16:48:27 UTC - in response to Message 12787.  

> That said, I'd kind of like to confirm something then. When credit is granted,
> does it grant the mean or the median value?
>

Currently in Seti, then 3 results has passed validation the credit-granting is run. Of the validated results, the highest & lowest claimed is discarded and the remaining validated is averaged and granted to all that passes validation. In most cases this will therefore be the middle, or median. ;)

Any results returned later, but before their deadline runs out, and that passes validation, will automatically be assigned the same as the others. No re-calculation of credit is done.

ID: 12803 · Report as offensive

Message boards : Number crunching : Just a SETI Classic observation


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.