Very new data...

Message boards : Number crunching : Very new data...
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 992449 - Posted: 28 Apr 2010, 13:42:56 UTC

Noticed this on the splitter this morning (presently for AP):

--> 04ap10aa 50.20 GB

That's the first 2010 "tape" I've seen come over. Hopefully the jump from the thinning Oct, Nov, Dec 2009 remnants is more a matter of random selection than an indicator that there isn't much data in between.

I was beginning to wonder if they were just repeating the 2006 & 2007 data on us :-)
ID: 992449 · Report as offensive
Profile skildude
Avatar

Send message
Joined: 4 Oct 00
Posts: 9541
Credit: 50,759,529
RAC: 60
Yemen
Message 992490 - Posted: 28 Apr 2010, 16:16:38 UTC - in response to Message 992449.  

they shouldn't need to repeat old data. What we've been getting is data that they've been able to software blank where they weren't able to do before. Everything we get is fresh and new.


In a rich man's house there is no place to spit but his face.
Diogenes Of Sinope
ID: 992490 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 995040 - Posted: 9 May 2010, 5:12:31 UTC - in response to Message 992490.  

they shouldn't need to repeat old data... Everything we get is fresh and new.

Everything ... alright, a quick sniff test, the latest tape on the present splitter is --> 18dc09aa

Now go check your job_log_setiathome file.... any old hits ?? ;-) And I don't mean AP.

ID: 995040 · Report as offensive
Luke
Volunteer developer
Avatar

Send message
Joined: 31 Dec 06
Posts: 2546
Credit: 817,560
RAC: 0
New Zealand
Message 995042 - Posted: 9 May 2010, 5:17:11 UTC

Most of my work is either 13ja07 or 30no06.
Newest data I have is from 18no09.

My biggest thought is concerning data, eventually, the rate of data coming from Arecibo falls short of the demands of the ever increasing amount of computers.

Perhaps a resolution jump of 10 is needed. ;)
- Luke.
ID: 995042 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19065
Credit: 40,757,560
RAC: 67
United Kingdom
Message 995063 - Posted: 9 May 2010, 7:51:05 UTC

There was quite a lot of new data, it was dated from 28th March 2010 to 5th April 2010.

I mentioned some of the tasks in my thread on new naming of AP tasks

http://setiathome.berkeley.edu/forum_thread.php?id=59850
ID: 995063 · Report as offensive
Fayvitt
Volunteer tester
Avatar

Send message
Joined: 29 Nov 09
Posts: 217
Credit: 1,190,636
RAC: 0
Australia
Message 995065 - Posted: 9 May 2010, 7:53:08 UTC
Last modified: 9 May 2010, 8:00:15 UTC

So by that reckoning, Seti should not send work if it's not new. After all, i thought it was about the science, not credits.

I have quite a few wu's in my cache atm, 29th dec 06, 29th oct 06. Newest batch by date is 28th mar 07. Surely they've been done 2-3 times by now?

Just a wee suspicion, but i'm probably wrong......as usual ;)
ID: 995065 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19065
Credit: 40,757,560
RAC: 67
United Kingdom
Message 995069 - Posted: 9 May 2010, 8:25:22 UTC - in response to Message 995065.  

So by that reckoning, Seti should not send work if it's not new. After all, i thought it was about the science, not credits.

I have quite a few wu's in my cache atm, 29th dec 06, 29th oct 06. Newest batch by date is 28th mar 07. Surely they've been done 2-3 times by now?

Just a wee suspicion, but i'm probably wrong......as usual ;)

99.99% chance that you are wrong again.

When they had a glut of data, some was put into off-site storage, that they could get back to be processed when there is a problem, like we had recently.

The problem at Arecibo was reported fixed, in Technical news recently. Bent pin on connector apparently.
ID: 995069 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 995076 - Posted: 9 May 2010, 8:48:51 UTC

After all, i thought it was about the science, not credits.

The software (re)blanking of older tapes is well understood -- that's not the issue being pointed out now. And the issue of how much potential piggy-back ALFA time was lost to S@H due to the bent pin is well understood -- that's also not the issue being pointed out now.

What is being pointed out is that a relatively new tape has by repeated (18dc09aa). And if there is one, then perhaps a LOT have been as people tend not to log tape names (but our crunchers do in the JOB LOG).

So it's not an "about the science" discussion, as that tape simply isn't affected by any known need to re-blank/repeat since ALL new tapes since October (when Matt perfected the software blanking and opened the floodgate on the previously unusable 7 months worth of older tapes) are both Hardware & Software blanked by default.

So either a glitch is permitting tapes to be added to the splitters multiple times (no benefit), or internally it's known that data is being repeated (maybe Matt has purpose to repeat some tapes since he's back working on NtPkr). I just want to make sure it's not an effort to reduce the crying whenever data runs low, as I'm happy to add other projects in those situations. I can't check the source code directly to see how the splitter queue works since the web interface has been busted for a while.
ID: 995076 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 995136 - Posted: 9 May 2010, 16:27:08 UTC - in response to Message 995076.  

...since ALL new tapes since October (when Matt perfected the software blanking and opened the floodgate on the previously unusable 7 months worth of older tapes) are both Hardware & Software blanked by default.
...

I believe it's OR rather than AND. The recent blog, WEEKLY MEETING Updates 4/26/10, notes that the project staff is discussing differences between the two methods. That suggests to me that they may well split some data once for each method in order to gather statistics.
                                                            Joe
ID: 995136 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 995165 - Posted: 9 May 2010, 18:32:09 UTC - in response to Message 995136.  

Thx JS, "perfected" was defintely too strong a classification, but I thought the hardware blanking was done auto-magically at the receiver side (Arecibo), thus not something that could be reflectively critiqued (as it can't be backed out of data from Dec 2009). I would have expected such tests on somewhat older tapes (pre-serendip V - June 2009). All speculation I suppose in the absence of informed updates by the S@H team itself.
ID: 995165 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 995258 - Posted: 10 May 2010, 3:45:13 UTC - in response to Message 995165.  

Thx JS, "perfected" was defintely too strong a classification, but I thought the hardware blanking was done auto-magically at the receiver side (Arecibo), thus not something that could be reflectively critiqued (as it can't be backed out of data from Dec 2009). I would have expected such tests on somewhat older tapes (pre-serendip V - June 2009). All speculation I suppose in the absence of informed updates by the S@H team itself.

The hardware blanking is just a signal which they patched into a spare recorder channel, so there are 14 channels of 2 bit coded raw data plus a 15th channel of 1 bit radar indication at the same 2.5MHz sample rate. When a splitter is loading the raw data, it does so through a routine which can use the blanking data to trigger replacement of the raw data with a shaped noise which attempts to simulate the background noise without radar. I believe Matt's software blanking is also recorded as a channel and is later used just like the hardware blanking.

It's possible that Serendip V data is also a useful comparison to see when there was real radar interference...
                                                               Joe
ID: 995258 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 995262 - Posted: 10 May 2010, 4:13:47 UTC - in response to Message 995258.  

So this is like a counter-balance correction applied at time of split, and all 14+1 channels must be recorded simultaneously. Since the splitter queue implies splitting is a channel at a time, that 15th channel must be re-read 14 times itself.

All makes sense, and a few new pieces to the puzzle for me. Thx
ID: 995262 · Report as offensive

Message boards : Number crunching : Very new data...


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.