AP validation dilemma

Message boards : Number crunching : AP validation dilemma
Message board moderation

To post messages, you must log in.

AuthorMessage
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14654
Credit: 200,643,578
RAC: 874
United Kingdom
Message 834221 - Posted: 25 Nov 2008, 12:32:40 UTC

Got the first one of an expected class of WUs - WU 364288117.

Ignore the computation error - that's not important. The other two are, to all intents and purposes, identical - both Intel Core2 running Windows XP SP3, BOINC v6.2.19, and the stock application. Fine upstanding SETIzens, the pair of them.

They didn't validate because one of them was computed with AP v4.36, and the other with AP v5.00: we thought they wouldn't validate against each other, and here's the proof.

Because I've been so involved with the testing, I know how to run my task with either v4.36 or v5.00. So I can validate and grant credit to either one of my fellow communards, but probably not both.

Which should I choose, and why?
ID: 834221 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 834226 - Posted: 25 Nov 2008, 12:52:24 UTC - in response to Message 834221.  

Version 5. 4.36 is superseded by new stock, so we must go by the current science.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 834226 · Report as offensive
Wandering Willie
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 136
Credit: 2,127,073
RAC: 0
United Kingdom
Message 834234 - Posted: 25 Nov 2008, 13:18:40 UTC

Hi Richard,

I would say let it run and see what the outcome is after all it could go to both, or give it to AP 5.00 as it is the latest. Smacks of semi favourtism otherwise. The decision in the end is yours and yours alone.

Michael
ID: 834234 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 834260 - Posted: 25 Nov 2008, 14:51:12 UTC - in response to Message 834234.  

Well, my personal opinion is to suspend "production" AP crunching on main as I already said in Lunatics forums.
But note, that it's still my personal opinion only.
While "validation" word under suspiction there is not much to change beside credits will you give "validation" to one or to another.
ID: 834260 · Report as offensive
Profile Leaps-from-Shadows
Volunteer tester
Avatar

Send message
Joined: 11 Aug 08
Posts: 323
Credit: 259,220
RAC: 0
United States
Message 834269 - Posted: 25 Nov 2008, 16:10:23 UTC - in response to Message 834260.  

Well, my personal opinion is to suspend "production" AP crunching on main as I already said in Lunatics forums.
But note, that it's still my personal opinion only.

So you think we shouldn't do any v5 work units at all - with standard or optimized apps?

Have the work units themselves changed? I ask because if they haven't, we could simply switch back to using v4 apps to crunch them.

I just want to make sure I'm not wasting CPU cycles crunching work units that don't add to the science. My RAC would certainly thank me if I stopped crunching Astropulse units, but that's not what this is about...

Cruiser has two v5 units in progress on Main, two in progress on Beta (actually, one completed but not reported and one in progress), plus seven in Main's queue and two in Beta's queue. I have suspended the seven queued units on Main until this is cleared up.
Cruiser
Gateway GT5692 L-f-S Edition
-Phenom X4 9650 CPU
-4GB 667MHz DDR2 RAM
-500GB SATA HD
-Vista x64 SP1
-BOINC 6.2.19 32-bit client
-SSE3 optimized 32-bit apps
ID: 834269 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 834284 - Posted: 25 Nov 2008, 16:48:19 UTC - in response to Message 834269.  
Last modified: 25 Nov 2008, 16:49:17 UTC

So you think we shouldn't do any v5 work units at all - with standard or optimized apps?

Have the work units themselves changed? I ask because if they haven't, we could simply switch back to using v4 apps to crunch them.

I just want to make sure I'm not wasting CPU cycles crunching work units that don't add to the science. My RAC would certainly thank me if I stopped crunching Astropulse units, but that's not what this is about...

Cruiser has two v5 units in progress on Main, two in progress on Beta (actually, one completed but not reported and one in progress), plus seven in Main's queue and two in Beta's queue. I have suspended the seven queued units on Main until this is cleared up.


My response at Lunatics:

Personally I would continue crunching in case Berkeley need data to analyse the performance characteristics themselves. Whether the results are fundamentally correct or not, in scientific terms they still have great value for verification, and further refinement if needed. I don't think stopping crunching them based on the anomalies we've seen so far is necessarily the ideal situation, as they could contain important pointers toward further development, or even indicate some real, little understood phenomenon lurking in some WUs.

Just my 2 cents.

Jason

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 834284 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 834285 - Posted: 25 Nov 2008, 16:49:17 UTC - in response to Message 834269.  
Last modified: 25 Nov 2008, 16:57:00 UTC

Hi, so it wouldn't be 'wise' to install either off the optimized AP apps. right now and wait until this is cleared?
On 1 rig, a P4 D820, I run NON-OPtimized apps. and crunch MB and AP, also EINSTEIN, this way I can view the graphics (screensaver; boincscr.exe).
This is the 'slowest host', I (still) run.
ID: 834285 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 834287 - Posted: 25 Nov 2008, 16:52:57 UTC - in response to Message 834285.  
Last modified: 25 Nov 2008, 16:59:32 UTC

It isn't the [Windows] optimised apps in question, they validate fine against stock so far (as far as we can tell). It's some perceived anomalous behaviour from all the apps with regard to introduction of some software radar blanking yielding unexpected signal counts. There is much more analysis to be done IMO, to determine whether these are truly anomalous, or some other unexpected phenomena.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 834287 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 834296 - Posted: 25 Nov 2008, 17:11:35 UTC - in response to Message 834287.  

1) Opt app now is as good (or as bad) as stock V5, just considerably faster. So if you wanna crunch, do it with opt app.
2) returning to V4.x has no meaning at all cause stock is V5.0 and server validator will mostly refuse validation between v5 and v4 (situation described in first post). So either stop either continue with V5.
ID: 834296 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19100
Credit: 40,757,560
RAC: 67
United Kingdom
Message 834320 - Posted: 25 Nov 2008, 23:04:43 UTC

As the vast majority of users never visit here, so any attempt to stop tasks being processed by V5 is futile.

So all we can do is to try and contact Eric or Josh and get them to sort out this mess.
ID: 834320 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14654
Credit: 200,643,578
RAC: 874
United Kingdom
Message 834985 - Posted: 27 Nov 2008, 21:38:25 UTC

OK, the dilemma is over. I decided to crunch it with a v5 (optimised) application, and the result is:

1056385822 4589383 13 Nov 2008 13:27:21 UTC 25 Nov 2008 11:44:03 UTC Over Success Done 178,391.50 760.39 0.00
1069800282 4292666 25 Nov 2008 11:44:20 UTC 27 Nov 2008 21:26:03 UTC Over Success Done 58,869.05 759.49 759.30
1056385823 4652844 13 Nov 2008 13:27:21 UTC 19 Nov 2008 8:36:21 UTC Over Client error Compute error 31,330.97 153.21 ---
1063278363 4635743 19 Nov 2008 8:36:28 UTC 22 Nov 2008 15:01:33 UTC Over Success Done 225,872.60 759.30 759.30

The first task is the stock v4.36, the last is the stock v5.00

A similar thing happened to the first of Geek@Play's jobs (mis)reported in the AK_V8 issues thread: WU 352088249

1065468267 3218838 21 Nov 2008 8:00:49 UTC 26 Nov 2008 19:12:02 UTC Over Success Done 51,769.95 759.31 759.31
1065468272 3176524 21 Nov 2008 8:00:52 UTC 27 Nov 2008 16:28:39 UTC Over Success Done 211,023.30 759.31 759.31
1030564956 3628086 22 Oct 2008 8:00:38 UTC 21 Nov 2008 8:00:38 UTC Over No reply New 0.00 --- ---
1030564955 1845306 22 Oct 2008 8:00:46 UTC 25 Nov 2008 9:29:55 UTC Over Success Done 534,579.40 755.91 0.00

The first two are v5.00 (optimised and stock, respectively): the final one is v4.36 stock.

I think we can take it that v4.36 results are not even weakly similar to v5.00 results.

Under normal circumstances, I would try to urge Eric to grant credit manually to the v4.36 tasks which have lost out, but all of these will be long purged from the database and untraceable before the long holiday weekend is over.
ID: 834985 · Report as offensive

Message boards : Number crunching : AP validation dilemma


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.