Validation inconclusive

Message boards : Number crunching : Validation inconclusive
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 891261 - Posted: 4 May 2009, 19:13:40 UTC - in response to Message 891192.  

There's shouldn't be anything too interesting about that, considering its the CPU that records the results and returns them. An overheating or malfunctioning CPU can produce bad results even if using the same stock app as someone else.
ID: 891261 · Report as offensive
Profile elbea64

Send message
Joined: 16 Aug 99
Posts: 114
Credit: 6,352,198
RAC: 0
Germany
Message 891289 - Posted: 4 May 2009, 20:03:25 UTC - in response to Message 891261.  

Yet there are only inconclusives with stock app involved and now there's one with two stock apps which makes it's rather likely that it's the stock app producing inconclusives and i find that is interesting.

If you think of, that most people that will see inconclusives are those more interested and therefore very likely run opt apps makes it even more interesting. It's very likely that a lot of stock app users never will see a inconclusive because of less interest than opt app users so it's likely that there are a lot of inconclusives between stock apps that will never show up here.

so statistics is very interesting here

And besides of temporary malfunctioning hardware, there could be programming errors, errors in specific hard-/software combinations or hardware design errors that come out with inconclusive validations and for them it's very interesting if it is the stock app or one of the opt apps
ID: 891289 · Report as offensive
Profile ccappel
Avatar

Send message
Joined: 27 Jan 00
Posts: 362
Credit: 1,516,412
RAC: 0
United States
Message 891360 - Posted: 4 May 2009, 23:59:39 UTC - in response to Message 891261.  

There's shouldn't be anything too interesting about that, considering its the CPU that records the results and returns them. An overheating or malfunctioning CPU can produce bad results even if using the same stock app as someone else.

I can understand that, but 3 separate inconclusives from 3 of my hosts with 3 different wingmen, 2 of which reported the exact same credit (see my OP). I find it unlikely to be malfunctioning hosts.
"Life is a tragedy for those who feel, and a comedy for those who think."

"I never get into an argument that I cannot win."
ID: 891360 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 891492 - Posted: 5 May 2009, 5:10:56 UTC

..but don't you think if they were software errors that they would have cropped up earlier? I find it hard to believe that, given the length of time that AP v5 has been released, that the errors are only cropping up now.

The funny thing about statistics is that they are only as good as the representative sample taken, and then they can be interpreted any way you want to. I don't think the handful of WU given here are a good enough representative sample to indicate anything of importance as of yet.
ID: 891492 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 891506 - Posted: 5 May 2009, 6:28:54 UTC - in response to Message 891492.  

..but don't you think if they were software errors that they would have cropped up earlier? I find it hard to believe that, given the length of time that AP v5 has been released, that the errors are only cropping up now.

The funny thing about statistics is that they are only as good as the representative sample taken, and then they can be interpreted any way you want to. I don't think the handful of WU given here are a good enough representative sample to indicate anything of importance as of yet.

To quote a DirecTV commercial: "90% of all statistics can be made to say anything...50% of the time."
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 891506 · Report as offensive
Profile elbea64

Send message
Joined: 16 Aug 99
Posts: 114
Credit: 6,352,198
RAC: 0
Germany
Message 891511 - Posted: 5 May 2009, 7:03:32 UTC

I didn't say it is representative i said it's interesting.

My explanation why it is interesting is very weak regarding interpretation of having a reason why this happens, but it's a strong reason to find it interesting for me.

What i've said is not even a theory, it's a possibility that has the same weight as the possibility of overheated cpus.

There's a argument that speaks against overheatet systems. Such systems will more likely produce errors than valid results that don't validate against others. But that's simply one of many arguments and doesn't mean that it won't be overheatet systems

i don't know, but want to know - that's what i call interesting
ID: 891511 · Report as offensive
Profile ccappel
Avatar

Send message
Joined: 27 Jan 00
Posts: 362
Credit: 1,516,412
RAC: 0
United States
Message 891600 - Posted: 5 May 2009, 14:11:02 UTC - in response to Message 891492.  
Last modified: 5 May 2009, 14:12:00 UTC

..but don't you think if they were software errors that they would have cropped up earlier? I find it hard to believe that, given the length of time that AP v5 has been released, that the errors are only cropping up now.

I'm not entirely ruling heat issues out, but it's not #1 on my suspect list. In addition to being a DBA, I'm also a data analyst, good at spotting trends and patterns in data.

I also find it interesting that both of my inconclusive results that had exact matching credits claimed were from the same tape, 27fe09ac.
"Life is a tragedy for those who feel, and a comedy for those who think."

"I never get into an argument that I cannot win."
ID: 891600 · Report as offensive
Stick Project Donor
Volunteer tester

Send message
Joined: 26 Feb 00
Posts: 100
Credit: 5,283,449
RAC: 5
United States
Message 891639 - Posted: 5 May 2009, 16:04:44 UTC - in response to Message 891600.  
Last modified: 5 May 2009, 16:30:32 UTC

I'm not entirely ruling heat issues out, but it's not #1 on my suspect list.

I don't buy the "heat issues" theory either. It certainly doesn't fit the situation I reported (with 3 inconclusive results). My computer has never shown shown any indications of heat and/or hardware issues and the "Tasks for computer" of my 2 wingmen look pretty clean.

EDIT: However, if I remember correctly, my result was uploaded and reported around the time of the Panic Mode On (15) Server problems reports - maybe server problems were a factor.
ID: 891639 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 891641 - Posted: 5 May 2009, 16:05:18 UTC - in response to Message 891600.  
Last modified: 5 May 2009, 16:08:32 UTC

..but don't you think if they were software errors that they would have cropped up earlier? I find it hard to believe that, given the length of time that AP v5 has been released, that the errors are only cropping up now.

I'm not entirely ruling heat issues out, but it's not #1 on my suspect list. In addition to being a DBA, I'm also a data analyst, good at spotting trends and patterns in data.

I also find it interesting that both of my inconclusive results that had exact matching credits claimed were from the same tape, 27fe09ac.

The credit claims for completed Astropulse tasks will always be identical between hosts if the tasks were sent at about the same time, barring one host quitting early, etc. Variation in credits for tasks sent on different days comes from the server-side rate adjustment.

The inconclusive results are certainly interesting. There are a number of possible contributing factors.

It is possible that one host processing a specific WU twice with the same app might produce two result files which are not strongly similar. There are at least two reasons that could happen. One is that I'm assuming a consumer grade system without ECC memory, so a bit flip can change results. The other is because the host may not be running the exact same code both times. FFTW is used in most of the apps, and at the beginning of processing it creates plans for each FFT length used, by timing several codelets which do the same thing with slightly different code. Those timing tests are brief and can be affected by other activity on the host, so identical choices are not always made. The different codelets produce FFTs which are accurate within very tight tolerances, but not identical. When the app is deciding whether it should report a signal or not, it is comparing two floating point numbers to see which is higher so a signal extremely close to threshold may or may not be reported, or a different 'best' signal may be chosen.

Given two different hosts, differences in the FFTs are even more likely, of course. And there are further minor variations due to slight differences in hardware.

At the Validator, tiny differences in host calculations can also affect the comparison because the AP validator only compares signals which are 1% above the single pulse threshold.
                                                                 Joe
ID: 891641 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 891654 - Posted: 5 May 2009, 16:32:33 UTC - in response to Message 891600.  

... I also find it interesting that both of my inconclusive results that had exact matching credits claimed were from the same tape, 27fe09ac.

Since r112 came out, I have kept my own database of every AP_v5 task I've done with r112, and for about a week and a half, I've got 20-30 tasks claiming the same exact credit, from at least 10 different tapes.

Credit claim doesn't really mean anything when validating (or not validating). I believe the order of priority for the validator is number of pulses, strength of those pulses, and then if at all, credit claim. I don't think credit claim matters if the first two match though, because then people who did a task when v5 first came out would be claiming ~1212, and third/fourth wingmen at this point would be claiming ~1251, but those two will validate fine as long as the pulses and strengths of those pulses match.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 891654 · Report as offensive
Profile ccappel
Avatar

Send message
Joined: 27 Jan 00
Posts: 362
Credit: 1,516,412
RAC: 0
United States
Message 891705 - Posted: 5 May 2009, 22:59:43 UTC - in response to Message 891641.  
Last modified: 5 May 2009, 23:01:44 UTC

Thank you, Josef, for that detailed explanation. Even if the underlying cause for these inconsistencies is never resolved, it still is worthwhile having the opportunity to get a better understanding of the underlying process.

While I've only been crunching since late Feb. after a long hiatus, I had never run into these inconclusive validations on AP for 2 months, and then got 3 within a week. It was this sudden change in events that prompted me to ask about them in the first place.

I see one of my 3 WUs above has validated with a 3rd wingman, and my other wingman now has a validate error. So I'm not concerned now that it was me.
"Life is a tragedy for those who feel, and a comedy for those who think."

"I never get into an argument that I cannot win."
ID: 891705 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 891748 - Posted: 6 May 2009, 0:30:13 UTC
Last modified: 6 May 2009, 0:34:31 UTC

Just to be clear, I did not state that heat issues were the definitive cause of the problem. I only threw that suggestion out there as another possibility. (Talk about interesting findings - interesting that people run away with a claim without examining its context or intent).

Holger Zantl wrote:
I didn't say it is representative i said it's interesting.


You didn't say "representative", but you certainly gave the implication in your last post to me that you spotted a trend among stock apps; a trend from a select group of results proffered in this thread, which suggests you are taking this select group as a representative sample as you've begun building hypothesis for possible explanations. You are, of course, allowed to find this all "interesting", but I'm just trying to keep people focussed on the bigger picture so that they don't get lost in the many, many details.

ccappel wrote:
I'm not entirely ruling heat issues out, but it's not #1 on my suspect list. In addition to being a DBA, I'm also a data analyst, good at spotting trends and patterns in data.


But again, this trend you have spotted is only from a select group of computers. Its entirely possible that you may be on to something. The only thing I am trying to get across, but seem to be viewed as being argumentative instead, is that we must keep in mind that smaller subset of users here before arriving to any definitive conclusions.
ID: 891748 · Report as offensive
Profile ccappel
Avatar

Send message
Joined: 27 Jan 00
Posts: 362
Credit: 1,516,412
RAC: 0
United States
Message 891779 - Posted: 6 May 2009, 1:16:18 UTC - in response to Message 891748.  

Ozzfan wrote:
ccappel wrote:
I'm not entirely ruling heat issues out, but it's not #1 on my suspect list. In addition to being a DBA, I'm also a data analyst, good at spotting trends and patterns in data.

But again, this trend you have spotted is only from a select group of computers. Its entirely possible that you may be on to something. The only thing I am trying to get across, but seem to be viewed as being argumentative instead, is that we must keep in mind that smaller subset of users here before arriving to any definitive conclusions.

Not at all, and I hope I didn't come across as implying that you were being argumentative. I totally agree on the small sample size, which was part of my reason for posting this thread...to find out and alert if my sample was representative of a larger picture, and partly to ease my mind that it wasn't just me.
"Life is a tragedy for those who feel, and a comedy for those who think."

"I never get into an argument that I cannot win."
ID: 891779 · Report as offensive
Profile elbea64

Send message
Joined: 16 Aug 99
Posts: 114
Credit: 6,352,198
RAC: 0
Germany
Message 891867 - Posted: 6 May 2009, 9:14:25 UTC - in response to Message 891748.  

You didn't say "representative", but you certainly gave the implication in your last post to me that you spotted a trend among stock apps; a trend from a select group of results proffered in this thread, which suggests you are taking this select group as a representative sample as you've begun building hypothesis for possible explanations. You are, of course, allowed to find this all "interesting", but I'm just trying to keep people focussed on the bigger picture so that they don't get lost in the many, many details.
The trends i see are always based on the available data which must not be representative. To investigate trends the available data has to be interesting at least for me. I see it like ccappel, i don't want to be the one who produces problems which is very subjective and influences the trend i (may want to) see. But this is all ok for me at this point namely because of the non-representative data.
As available data will grow or better explanations of this are available i will refine my vague point of view, add aditional possibilities or drop others that went impossible.
At the moment i would tend to Josef W. Segur's explanation with the addition of a small "please not me" ;)

Just to be clear, I did not state that heat issues were the definitive cause of the problem. I only threw that suggestion out there as another possibility.
I understood exact this and i see this clearly as a pssibility! I think it's not the strongest yet but far away from being dropped.

Talk about interesting findings - interesting that people run away with a claim without examining its context or intent.
I don't understand what and/or who you mean? I think i've looked into it with a adequate focus.

The only thing I am trying to get across, but seem to be viewed as being argumentative instead
Yes, i felt that way. I said it's interesting - you said there's nothing interesting with a argument i can't agree fully but that sounded like the argument. I tried to explain my point - you made it ridiculous. And now i understand the last quote: a software error isn't my personal favourite but i can think of situations where it shows up that late, perhaps because of some odd WUs going around someone mentioned somewhere i can't find anymore.
If i saw you argumentative by mistake i'm sorry.

There's shouldn't be anything too interesting about that, considering its the CPU that records the results and returns them. An overheating or malfunctioning CPU can produce bad results even if using the same stock app as someone else.
I think i now understand what you wanted to say but earlier i read this as the ultimate argument that there's nothing about it. Nothing offending here, i'm just curious if you can follow my thoughts or did i miss something.
ID: 891867 · Report as offensive
Profile elbea64

Send message
Joined: 16 Aug 99
Posts: 114
Credit: 6,352,198
RAC: 0
Germany
Message 891940 - Posted: 6 May 2009, 15:16:15 UTC
Last modified: 6 May 2009, 15:31:23 UTC

I have one of my inconclusives validated too. It was a stock app against opt app inconclusive and validated against another opt app.

And another one that looks more interesting. These are two opt apps and one rendered to invalid while mine is inconclusive. I think the other was inconclusive too for a while so it could validate inconclusive together with mine
ID: 891940 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 891947 - Posted: 6 May 2009, 15:30:29 UTC - in response to Message 891940.  

Holger, did you check out that first guy? Looks like almost all of his are either inconclusive or invalid. He is Really turning in bad work not a question of optimized Vs stock.


PROUD MEMBER OF Team Starfire World BOINC
ID: 891947 · Report as offensive
Profile elbea64

Send message
Joined: 16 Aug 99
Posts: 114
Credit: 6,352,198
RAC: 0
Germany
Message 891958 - Posted: 6 May 2009, 15:56:02 UTC

That result wasn't looking very interesting so didn't look deeper into it. Now i have PMed him about the problem with his PC.

It's not clear to me if it is a question of stock or opt app. perhaps it's a general problem. The second result indicate a little that it's not a problem of a specific app although it's different to the other inconclusives
ID: 891958 · Report as offensive
Alinator
Volunteer tester

Send message
Joined: 19 Apr 05
Posts: 4178
Credit: 4,647,982
RAC: 0
United States
Message 891975 - Posted: 6 May 2009, 16:16:49 UTC - in response to Message 891940.  

I have one of my inconclusives validated too. It was a stock app against opt app inconclusive and validated against another opt app.

And another one that looks more interesting. These are two opt apps and one rendered to invalid while mine is inconclusive. I think the other was inconclusive too for a while so it could validate inconclusive together with mine


If you look through the results of the first one, this has all the signs of an overheating or another hardware issue. In fact, there was an MB task which errored out with 'Float inexact result <some value> at <mem address>' (first time I ever saw that one!). Most of the AP's appear to be Dash 9's (signal overflows).

For the second case you mentioned, I doubt this one will resurrect. Typically a validate error like this is caused by the backend 'loosing' the output file from the host.

Alinator
ID: 891975 · Report as offensive
Urs Echternacht
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 692
Credit: 135,197,781
RAC: 211
Germany
Message 893716 - Posted: 11 May 2009, 20:51:54 UTC

For the second case you mentioned, I doubt this one will resurrect. Typically a validate error like this is caused by the backend 'loosing' the output file from the host.

Alinator


Found some "validate error"s from ca. 11 May 2009 18:50:57 UTC
http://setiathome.berkeley.edu/workunit.php?wuid=438442777
http://setiathome.berkeley.edu/workunit.php?wuid=438353803
Hopefully the database is not "leaking" too much of our results.
_\|/_
U r s
ID: 893716 · Report as offensive
Profile Bird-Dog
Volunteer tester
Avatar

Send message
Joined: 8 Jun 04
Posts: 121
Credit: 463,173
RAC: 0
United Kingdom
Message 893723 - Posted: 11 May 2009, 21:22:52 UTC - in response to Message 893716.  
Last modified: 11 May 2009, 21:24:55 UTC

I have 26 of those "validate errors" all reported 11 May 18:53:40 UTC.
ID: 893723 · Report as offensive
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : Validation inconclusive


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.