New Greenbank Files

Message boards : Number crunching : New Greenbank Files
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5

AuthorMessage
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1778979 - Posted: 14 Apr 2016, 7:35:01 UTC - in response to Message 1778977.  

Never cared for regulators, lol...


Lol, now where's my Diagram of the Steam engine governor with rotating balls....

Hmmm...rotating balls.
Hmmm.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1778979 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1778996 - Posted: 14 Apr 2016, 8:49:36 UTC - in response to Message 1778979.  
Last modified: 14 Apr 2016, 8:51:32 UTC

Never cared for regulators, lol...


Lol, now where's my Diagram of the Steam engine governor with rotating balls....

Hmmm...rotating balls.
Hmmm.


Yes, mathematically speaking, orbital oscillation in the temporal domain.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1778996 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1779031 - Posted: 14 Apr 2016, 13:55:21 UTC - in response to Message 1778956.  

To put some figures on the difference in runtimes: all tasks are VLAR, no -9 overflow, on I think my most stable machine.

Arecibo (23 tasks): average 5,325.73 seconds, std. dev. 159.50
Guppi (4 tasks): average 4,441.34 seconds, std. dev. 241.30

Not as big a difference as I thought, but noticeable.
ID: 1779031 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 1779035 - Posted: 14 Apr 2016, 15:15:51 UTC - in response to Message 1779031.  

To put some figures on the difference in runtimes: all tasks are VLAR, no -9 overflow, on I think my most stable machine.

Arecibo (23 tasks): average 5,325.73 seconds, std. dev. 159.50
Guppi (4 tasks): average 4,441.34 seconds, std. dev. 241.30

Not as big a difference as I thought, but noticeable.



Interesting to see what credit is granted by Credit Nu?.
ID: 1779035 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1779042 - Posted: 14 Apr 2016, 15:38:58 UTC - in response to Message 1779035.  

Interesting to see what credit is granted by Credit Nu?.

You can watch if you like, but it's a long wait for wingmates.

Valid tasks for computer 7118033

Arecibo - 2 VLAR validated, credit 105.98 and 108.96
Guppi - no validations at all yet.
ID: 1779042 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1779046 - Posted: 14 Apr 2016, 16:08:13 UTC

ID: 1779046 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34253
Credit: 79,922,639
RAC: 80
Germany
Message 1779129 - Posted: 14 Apr 2016, 21:13:43 UTC
Last modified: 14 Apr 2016, 21:14:30 UTC

The problem is to get some for GPU.

I disabled CPU work fetch for 24 hours and got nothing but Arecibo units.


With each crime and every kindness we birth our future.
ID: 1779129 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1779132 - Posted: 14 Apr 2016, 21:31:54 UTC

Of the first 18 that I completed, all are VLARs with an angle range between 0.005947 and 0.010613.

8 of the 18 have ended with -9 overflows.
ID: 1779132 · Report as offensive
Chris Adamek
Volunteer tester

Send message
Joined: 15 May 99
Posts: 251
Credit: 434,772,072
RAC: 236
United States
Message 1779179 - Posted: 14 Apr 2016, 23:52:48 UTC - in response to Message 1779129.  

Same here. Nothing for my ATI cards from there yet.

Chris
ID: 1779179 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1779595 - Posted: 16 Apr 2016, 12:47:59 UTC - in response to Message 1779132.  


8 of the 18 have ended with -9 overflows.


If such ratio will scale then some thresholds should be reconsidered IMO...
ID: 1779595 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1779657 - Posted: 16 Apr 2016, 16:30:58 UTC - in response to Message 1779595.  


8 of the 18 have ended with -9 overflows.


If such ratio will scale then some thresholds should be reconsidered IMO...

It doesn't seem to be scaling. I've now completed 87 guppis but the count of -9 overflows is only up to 18. I have no idea if that still seems high or not.
ID: 1779657 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1779695 - Posted: 16 Apr 2016, 19:43:49 UTC - in response to Message 1779657.  


8 of the 18 have ended with -9 overflows.


If such ratio will scale then some thresholds should be reconsidered IMO...

It doesn't seem to be scaling. I've now completed 87 guppis but the count of -9 overflows is only up to 18. I have no idea if that still seems high or not.


Most likely a reasonable indicator if the noise floor has changed, will be if you can get enough volume to see if the reissue (inconclusive) to pending ratio drops below 5% as with Arecibo tasks and a healthy system + app. Someproportion of that 5% (or less) going inconclusive, would be dodgy wingmen/apps (a sign of project health), natural variation cross platform, and a function of the noise-floor/threholds.

If <=5% go inconclusive and reissue, yet the number of -9's remains high, then 'probably' can be interpreted as many apps+platforms+hosts agreeing that the noise floor is high (so thesholds should be raised)

If, on the other hand, it refuses to converge on <=5% inconclusive, then some app or platform differences may remain (with or without -9 overflows present). Without excess -9's we could guess that there is some error growth going on in one or more calculations, one or more apps (possibly all). With excess -9's the cumulative error could be there also, but buried in the noise floor.

I'm going to guess that this early there could be a bit of both noisy data on low thresholds going on, along with some possible app fine tuning needed if specific offenders can be pinpointed across enough hosts.
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1779695 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1779697 - Posted: 16 Apr 2016, 19:52:18 UTC - in response to Message 1779695.  

If you follow the SoG thread to Beta, there is some indication that there may be a higher error rate than usual, on particular datasets.

I think it's autocorr protection false positive.
We debated with Joe level of this protection some time ago when there were few false positives in usual Arecibo's data. Instead of all other types of protection levels there is no sensible theoretical limits for autocorr value that we can expect from data.
So, chosen one was only our guess what it could be. I already increased it some time ago. Apparently - not high enough. Maybe this sanity check should be disabled completely. I'll try to consult with Eric on this topic again.
ID: 1779697 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1779699 - Posted: 16 Apr 2016, 20:06:00 UTC - in response to Message 1779697.  
Last modified: 16 Apr 2016, 20:06:37 UTC

Hard to say with GBT reobservation data IMO, as to what an appropriate safety limit might be, because self similarity implied by high autocorrelation values I'd imagine might be expected from deliberately staring at a radio source (self similar pattern, natural or otherwise)

So yeah, probably a question for Eric, and lifting that might be the way (assuming it wasn't put there because of some driver/app/platform problem causing weird autocorrelations)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1779699 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1779706 - Posted: 16 Apr 2016, 20:18:55 UTC - in response to Message 1779699.  

Hard to say with GBT reobservation data IMO, as to what an appropriate safety limit might be, because self similarity implied by high autocorrelation values I'd imagine might be expected from deliberately staring at a radio source (self similar pattern, natural or otherwise)

So yeah, probably a question for Eric, and lifting that might be the way (assuming it wasn't put there because of some driver/app/platform problem causing weird autocorrelations)

I think that was the intention, yes. But it raises - again - the problem of distinguishing high-significance autocorrelations caused by weird data, from ditto caused by weird computers. I suspect the answer may need to be "safeties to off" until Eric can characterise the data better for us.
ID: 1779706 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1779720 - Posted: 16 Apr 2016, 21:18:06 UTC - in response to Message 1779699.  

...(assuming it wasn't put there because of some driver/app/platform problem causing weird autocorrelations)

I think it originally stemmed from quite a few rigs with ATI GPUs that were generating bogus -9 overflows with matching 30 autocorr counts that then validated against each other, causing legitimate results to be invalidated. (See Two wrongs make a right.)

Joe suggested the sanity check and, once implemented, it seems to have alleviated that problem.
ID: 1779720 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1779728 - Posted: 16 Apr 2016, 21:47:59 UTC - in response to Message 1779720.  
Last modified: 16 Apr 2016, 21:48:29 UTC


Joe suggested the sanity check and, once implemented, it seems to have alleviated that problem.


Thanks for link to post with original proposal.
Since then level of 100 proved to be too low even for Arecibo's work. Current level of 135 isn't adequate for new GBT data too. I E-mailed Eric to check if we just have some weird batch of GBT data or it's inherent feature of such observation style...
ID: 1779728 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1779920 - Posted: 17 Apr 2016, 17:35:10 UTC - in response to Message 1779657.  


8 of the 18 have ended with -9 overflows.


If such ratio will scale then some thresholds should be reconsidered IMO...

It doesn't seem to be scaling. I've now completed 87 guppis but the count of -9 overflows is only up to 18. I have no idea if that still seems high or not.

A day later and I've now completed 143 guppi VLARs and the total of -9 overflows has only risen to 24, so the ratio seems to be dropping, now down to about 16.8%. For comparison, I took a look at the Arecibo VLARs in my archives for March. Out of 293 that I completed, 45 were -9 overflows, or about 15.4%. So, at least for my small sample sizes, the ratios are similar.

However, what I did notice was that there seems to be a noticeably different distribution in the triggers for those overflows. To wit:

-9 OVERFLOW TRIGGERS
Arecibo VLARs (March, 2016)
30 spikes: 15 WUs
30 autocorrs: 1 WU
30 pulses: 0 WUs
30 triplets: 0 WUs
30 gaussian: 0 WUs
30 mixed signals: 29 WUs

GBT (guppi) VLARs
30 spikes: 5 WUs
30 autocorrs: 3 WU
30 pulses: 2 WUs
30 triplets: 2 WUs
30 gaussian: 0 WUs
30 mixed signals: 12 WUs

I don't know if that piques anybody's curiosity, but I figured as long as I had the data, I'd post it.
ID: 1779920 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1779930 - Posted: 17 Apr 2016, 18:50:31 UTC - in response to Message 1779920.  

Maybe GBT has some different source of artifical signal noise than Arecibo. To have overflowed Triplets and Pulses w/o Spikes is very unusual indeed.
Or, maybe we catch some really strong ET signal here.... fingers crossed :)
ID: 1779930 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1781092 - Posted: 21 Apr 2016, 19:43:28 UTC - in response to Message 1779930.  

Maybe GBT has some different source of artifical signal noise than Arecibo. To have overflowed Triplets and Pulses w/o Spikes is very unusual indeed.
Or, maybe we catch some really strong ET signal here.... fingers crossed :)

Perhaps some different thresholds might be worth considering, as I found that last night's processing yielded a whole bunch of triplets-only overflows. Out of 87 GBT tasks my hosts completed, 49 were 30-triplet overflows, all split from either HIP11048 (30 WUs) or HIP11048_OFF (19 WUs) files.

Prior to those, I'd only seen 3 triplets-only overflows from GBT WUs, all from different sources, and only 3 others all month from Arecibo WUs.
ID: 1781092 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5

Message boards : Number crunching : New Greenbank Files


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.