Take That, FAA Radar! (Sep 23 2009)

Message boards : Technical News : Take That, FAA Radar! (Sep 23 2009)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 935459 - Posted: 23 Sep 2009, 20:46:09 UTC

Had more science database woes at the end of the day yesterday - processes (including splitters) getting logjammed. I'm hoping a couple "update stats" commands will fix all that.

Speaking of splitters, I'm actually running (drumroll please) the first software radar blanked data through a splitter right now, and workunits will be distributed to the public fairly soon. This is still in test phase - we shall see if the software blanking performs better than (or worse, or the same as) the hardware blanking. I'm guess with a couple tweaks here and there my code will be far better.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 935459 · Report as offensive
DJStarfox

Send message
Joined: 23 May 01
Posts: 1066
Credit: 1,226,053
RAC: 2
United States
Message 935488 - Posted: 23 Sep 2009, 23:56:32 UTC - in response to Message 935459.  

Assuming the software blanking works well, would you keep both the hardware and software blanking in the process? I would think a good hardware solution would be ideal, but I will hold my breath for the actual results of your (not so) little test.
ID: 935488 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 935495 - Posted: 24 Sep 2009, 0:11:37 UTC

Yahoo Matt! I have my fingers crossed. Tweak on!

ID: 935495 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 935561 - Posted: 24 Sep 2009, 3:25:25 UTC
Last modified: 24 Sep 2009, 3:26:50 UTC

This is way cool!!

If the radar blanking is done with Matt's new splitter program then a new science app is not needed. Looks like Matt's hard work is going to pay off.

+1 atta boy Matt.
Boinc....Boinc....Boinc....Boinc....
ID: 935561 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21227
Credit: 7,508,002
RAC: 20
United Kingdom
Message 935609 - Posted: 24 Sep 2009, 11:16:32 UTC - in response to Message 935488.  
Last modified: 24 Sep 2009, 11:17:35 UTC

Assuming the software blanking works well, would you keep both the hardware and software blanking in the process? I would think a good hardware solution would be ideal, but I will hold my breath for the actual results of your (not so) little test.

I would guess that the software solution will pick up contamination that is seen as RFI that may well be below the trigger threshold for the hardware RADAR detector.

My second part of a guess is that the hardware blanker will work fine for direct hits and strong reflections, but may well not be sensitive enough for the weaker reflections and multipath effects that s@h will still see. Note that we're looking all the way down for the weakest of the weak signals.

Meanwhile, I'm still a little concerned for how the blanking is done and whether the inserted noise that replaces the RADAR signal will itself introduce artefacts. Some of the Lunatics are looking at modifying the client so that contaminated sections of WUs are completely skipped instead...


Keep searchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 935609 · Report as offensive
PhonAcq

Send message
Joined: 14 Apr 01
Posts: 1656
Credit: 30,658,217
RAC: 1
United States
Message 935677 - Posted: 24 Sep 2009, 18:10:01 UTC - in response to Message 935609.  

The introduction of artifacts have been discussed before, I think I recall. I wonder what the rebuttal was.

On the face of it, I agree with ML1's concern. Taking a swag at encapsulating it, the choice is between blanking the wu and analyzing the data in pieces with a new or altered client, or introducing artificial noise in the blank periods and using the existing client. The added noise must tend to push the detectability of an ET signal down, since (presumably!) ET doesn't exist in the added noise. However, analyzing shorter periods of time (between the RF blanks) also reduces the detectability because the contiguous data runs are shorter. I'm sure the details aren't as simple as I describe here, but I hope that I have the dilemma correct.

Was there a third option considered, such as taking longer time slices? If ET is not in the nuisance RF band, the RF contribution could be rejected via the FFT filtering. Perhaps the RF is too broadbanded? Could we take finer time steps to get around that?

Would wavelet analysis be less sensitive to the RF noise than the standard FFT analysis approach? That was suggested a long time ago, but I never saw a thoughtful response.

Off topic: was the blanker tested out using the beta folks? or isn't this the process for introducing new things like this.
ID: 935677 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21227
Credit: 7,508,002
RAC: 20
United Kingdom
Message 935696 - Posted: 24 Sep 2009, 19:34:17 UTC - in response to Message 935677.  

The introduction of artifacts have been discussed before, I think I recall. I wonder what the rebuttal was.

On the face of it, I agree with ML1's concern. Taking a swag at encapsulating it, the choice is between blanking the wu and analyzing the data in pieces with a new or altered client, or introducing artificial noise in the blank periods and using the existing client. The added noise must tend to push the detectability of an ET signal down, since (presumably!) ET doesn't exist in the added noise. However, analyzing shorter periods of time (between the RF blanks) also reduces the detectability because the contiguous data runs are shorter. I'm sure the details aren't as simple as I describe here, but I hope that I have the dilemma correct.

Was there a third option considered, such as taking longer time slices? If ET is not in the nuisance RF band, the RF contribution could be rejected via the FFT filtering. Perhaps the RF is too broadbanded? Could we take finer time steps to get around that?

Would wavelet analysis be less sensitive to the RF noise than the standard FFT analysis approach? That was suggested a long time ago, but I never saw a thoughtful response.

Off topic: was the blanker tested out using the beta folks? or isn't this the process for introducing new things like this.

I believe you've already been answered elsewhere. Good questions but I'll expect that I'll be giving you the same or similar answers...

Note that the RADAR contaminated part of the WU is completely replaced by artificial pseudo-random noise, so in effect you're chopping up the WU into smaller pieces for the real data that remains. The main difference between that and the Lunatic method is that the Lunatics can avoid a lot of processing of artificially inserted sections of noise to instead literally just 'intelligently' skip them. Either way, you lose data for where the RADAR has obliterated what might have been there.

However, a big positive is that you avoid contaminating the results with lots of junk detections. You also avoid that WU blindly erroring out on the client with a "-9" for the excessive junk load result. Hence, you get a better chance for finding real signals either side of the blasts of RADAR.

Working out the FFT filtering (to detect the RADAR contamination) is what Matt has been working on for some time with the splitters...

And regarding "Would wavelet analysis..."... That has been answered by Eric and others a few times over in that using FFTs are much more suited to the sort of analysis that we are doing here.

Now... If you can put together a wavelets client that offers some good benefits over the existing client, then you will get the chance to put your name on a science paper and get the sort of WOW factor that is so far reserved only for Alex Khan!

Good questions!

Keep searchin',
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 935696 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21227
Credit: 7,508,002
RAC: 20
United Kingdom
Message 935702 - Posted: 24 Sep 2009, 19:59:26 UTC - in response to Message 935677.  

... the RF contribution could be rejected via the FFT filtering. ...

Now, that would be really something if you could do that reliably... Filter to retrieve only the RADAR signal(s) and then exactly subtract those signals from the WU data to produce a 'clean' WU.

However, I suspect that it is very much easier and far more reliable to do the blanking...


I would also suspect that an upgrade to the RF shielding of the ALFA housing and connections would give a better result in the first place. Are any of the other experiments that use ALFA affected by the RADAR contamination? Might they be fixing the problem for us at source?

Keep searchin',
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 935702 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19401
Credit: 40,757,560
RAC: 67
United Kingdom
Message 935794 - Posted: 25 Sep 2009, 7:51:49 UTC - in response to Message 935702.  

Now, that would be really something if you could do that reliably... Filter to retrieve only the RADAR signal(s) and then exactly subtract those signals from the WU data to produce a 'clean' WU.

However, I suspect that it is very much easier and far more reliable to do the blanking...

Now the normal way of doing that is to get the originator to transmit on two separate frequencies, with the signal applied to each transmitter in anti-phase. That way the two receivers receive the wanted signal in anti-phase and the unwanted local interference in-phase. Feed the output of each receiver into a longtail pair combiner and the interference cancels itself out.

Now if we could just contact ET .....

Andy
ID: 935794 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21227
Credit: 7,508,002
RAC: 20
United Kingdom
Message 935819 - Posted: 25 Sep 2009, 13:53:28 UTC - in response to Message 935794.  

Now, that would be really something if you could do that reliably... Filter to retrieve only the RADAR signal(s) and then exactly subtract those signals from the WU data to produce a 'clean' WU.

However, I suspect that it is very much easier and far more reliable to do the blanking...

Now the normal way of doing that is to get the originator to transmit on two separate frequencies, with the signal applied to each transmitter in anti-phase. That way the two receivers receive the wanted signal in anti-phase and the unwanted local interference in-phase. Feed the output of each receiver into a longtail pair combiner and the interference cancels itself out.

Now if we could just contact ET .....

Nice idea but rather complicated by frequency dependant dispersion and phase shifts over the vast distance of space...

I still think the improved shielding at Arecibo and/or switching off the RADARs in the first place would be much more reliable!

Keep searchin',
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 935819 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 935889 - Posted: 25 Sep 2009, 20:31:33 UTC - in response to Message 935702.  

I would also suspect that an upgrade to the RF shielding of the ALFA housing and connections would give a better result in the first place.

If we're looking for how the signal gets into the receiver front end, I suspect that the biggest source of local RF is getting in through the feed antenna.

Eliminate that and you'd have no trouble.

Seriously, though, if the receiver is very sensitive (and it needs to be), anything strong nearby is going to get in to it.
ID: 935889 · Report as offensive
Dena Wiltsie
Volunteer tester

Send message
Joined: 19 Apr 01
Posts: 1628
Credit: 24,230,968
RAC: 26
United States
Message 935912 - Posted: 25 Sep 2009, 21:34:25 UTC - in response to Message 935819.  


I still think the improved shielding at Arecibo and/or switching off the RADARs in the first place would be much more reliable!


Sad part is Arecibo may be switched off before the radar is.
ID: 935912 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 936076 - Posted: 26 Sep 2009, 17:08:50 UTC - in response to Message 935912.  


I still think the improved shielding at Arecibo and/or switching off the RADARs in the first place would be much more reliable!


Sad part is Arecibo may be switched off before the radar is.

Especially since the radar costs a lot less to run, and has a "job" that is generally considered more important than research.
ID: 936076 · Report as offensive
Profile Reuben Gathright
Avatar

Send message
Joined: 8 Mar 01
Posts: 213
Credit: 14,594,579
RAC: 0
United States
Message 936305 - Posted: 27 Sep 2009, 18:44:36 UTC - in response to Message 935459.  

ID: 936305 · Report as offensive
Profile Misfit
Volunteer tester
Avatar

Send message
Joined: 21 Jun 01
Posts: 21804
Credit: 2,815,091
RAC: 0
United States
Message 936355 - Posted: 28 Sep 2009, 0:34:31 UTC - in response to Message 935459.  

Speaking of splitters, I'm actually running (drumroll please) the first software radar blanked data through a splitter right now, and workunits will be distributed to the public fairly soon. This is still in test phase - we shall see if the software blanking performs better than (or worse, or the same as) the hardware blanking. I'm guess with a couple tweaks here and there my code will be far better.

- Matt

Does that mean it'll come to BETA first?
me@rescam.org
ID: 936355 · Report as offensive
RoosStar

Send message
Joined: 16 Oct 99
Posts: 51
Credit: 12,900,339
RAC: 20
Netherlands
Message 936406 - Posted: 28 Sep 2009, 10:30:31 UTC - in response to Message 936355.  
Last modified: 28 Sep 2009, 10:33:13 UTC

That is not intended. See Matts post Take Two
ID: 936406 · Report as offensive

Message boards : Technical News : Take That, FAA Radar! (Sep 23 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.