Take That, FAA Radar! (Sep 23 2009)

Message boards : Technical News : Take That, FAA Radar! (Sep 23 2009)

Author	Message
Matt Lebofsky Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 1 Mar 99 Posts: 1375 Credit: 74,079 RAC: 0	Message 935459 - Posted: 23 Sep 2009, 20:46:09 UTC
	Had more science database woes at the end of the day yesterday - processes (including splitters) getting logjammed. I'm hoping a couple "update stats" commands will fix all that. Speaking of splitters, I'm actually running (drumroll please) the first software radar blanked data through a splitter right now, and workunits will be distributed to the public fairly soon. This is still in test phase - we shall see if the software blanking performs better than (or worse, or the same as) the hardware blanking. I'm guess with a couple tweaks here and there my code will be far better. - Matt ____________ -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
	ID: 935459 ·

DJStarfox Send message Joined: 23 May 01 Posts: 1040 Credit: 485,117 RAC: 157	Message 935488 - Posted: 23 Sep 2009, 23:56:32 UTC - in response to Message 935459.
	Assuming the software blanking works well, would you keep both the hardware and software blanking in the process? I would think a good hardware solution would be ideal, but I will hold my breath for the actual results of your (not so) little test.
	ID: 935488 ·

Bill Walker Send message Joined: 4 Sep 99 Posts: 2861 Credit: 1,778,308 RAC: 223	Message 935495 - Posted: 24 Sep 2009, 0:11:37 UTC
	Yahoo Matt! I have my fingers crossed. Tweak on! ____________
	ID: 935495 ·

Geek@Play Volunteer tester Send message Joined: 31 Jul 01 Posts: 2414 Credit: 79,016,596 RAC: 0	Message 935561 - Posted: 24 Sep 2009, 3:25:25 UTC Last modified: 24 Sep 2009, 3:26:50 UTC
	This is way cool!! If the radar blanking is done with Matt's new splitter program then a new science app is not needed. Looks like Matt's hard work is going to pay off. +1 atta boy Matt. ____________ Boinc....Boinc....Boinc....Boinc....
	ID: 935561 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7112 Credit: 3,681,408 RAC: 916	Message 935609 - Posted: 24 Sep 2009, 11:16:32 UTC - in response to Message 935488. Last modified: 24 Sep 2009, 11:17:35 UTC
	Assuming the software blanking works well, would you keep both the hardware and software blanking in the process? I would think a good hardware solution would be ideal, but I will hold my breath for the actual results of your (not so) little test. I would guess that the software solution will pick up contamination that is seen as RFI that may well be below the trigger threshold for the hardware RADAR detector. My second part of a guess is that the hardware blanker will work fine for direct hits and strong reflections, but may well not be sensitive enough for the weaker reflections and multipath effects that s@h will still see. Note that we're looking all the way down for the weakest of the weak signals. Meanwhile, I'm still a little concerned for how the blanking is done and whether the inserted noise that replaces the RADAR signal will itself introduce artefacts. Some of the Lunatics are looking at modifying the client so that contaminated sections of WUs are completely skipped instead... Keep searchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 935609 ·

PhonAcq Send message Joined: 14 Apr 01 Posts: 1620 Credit: 20,858,897 RAC: 4,460	Message 935677 - Posted: 24 Sep 2009, 18:10:01 UTC - in response to Message 935609.
	The introduction of artifacts have been discussed before, I think I recall. I wonder what the rebuttal was. On the face of it, I agree with ML1's concern. Taking a swag at encapsulating it, the choice is between blanking the wu and analyzing the data in pieces with a new or altered client, or introducing artificial noise in the blank periods and using the existing client. The added noise must tend to push the detectability of an ET signal down, since (presumably!) ET doesn't exist in the added noise. However, analyzing shorter periods of time (between the RF blanks) also reduces the detectability because the contiguous data runs are shorter. I'm sure the details aren't as simple as I describe here, but I hope that I have the dilemma correct. Was there a third option considered, such as taking longer time slices? If ET is not in the nuisance RF band, the RF contribution could be rejected via the FFT filtering. Perhaps the RF is too broadbanded? Could we take finer time steps to get around that? Would wavelet analysis be less sensitive to the RF noise than the standard FFT analysis approach? That was suggested a long time ago, but I never saw a thoughtful response. Off topic: was the blanker tested out using the beta folks? or isn't this the process for introducing new things like this.
	ID: 935677 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7112 Credit: 3,681,408 RAC: 916	Message 935696 - Posted: 24 Sep 2009, 19:34:17 UTC - in response to Message 935677.
	The introduction of artifacts have been discussed before, I think I recall. I wonder what the rebuttal was. On the face of it, I agree with ML1's concern. Taking a swag at encapsulating it, the choice is between blanking the wu and analyzing the data in pieces with a new or altered client, or introducing artificial noise in the blank periods and using the existing client. The added noise must tend to push the detectability of an ET signal down, since (presumably!) ET doesn't exist in the added noise. However, analyzing shorter periods of time (between the RF blanks) also reduces the detectability because the contiguous data runs are shorter. I'm sure the details aren't as simple as I describe here, but I hope that I have the dilemma correct. Was there a third option considered, such as taking longer time slices? If ET is not in the nuisance RF band, the RF contribution could be rejected via the FFT filtering. Perhaps the RF is too broadbanded? Could we take finer time steps to get around that? Would wavelet analysis be less sensitive to the RF noise than the standard FFT analysis approach? That was suggested a long time ago, but I never saw a thoughtful response. Off topic: was the blanker tested out using the beta folks? or isn't this the process for introducing new things like this. I believe you've already been answered elsewhere. Good questions but I'll expect that I'll be giving you the same or similar answers... Note that the RADAR contaminated part of the WU is completely replaced by artificial pseudo-random noise, so in effect you're chopping up the WU into smaller pieces for the real data that remains. The main difference between that and the Lunatic method is that the Lunatics can avoid a lot of processing of artificially inserted sections of noise to instead literally just 'intelligently' skip them. Either way, you lose data for where the RADAR has obliterated what might have been there. However, a big positive is that you avoid contaminating the results with lots of junk detections. You also avoid that WU blindly erroring out on the client with a "-9" for the excessive junk load result. Hence, you get a better chance for finding real signals either side of the blasts of RADAR. Working out the FFT filtering (to detect the RADAR contamination) is what Matt has been working on for some time with the splitters... And regarding "Would wavelet analysis..."... That has been answered by Eric and others a few times over in that using FFTs are much more suited to the sort of analysis that we are doing here. Now... If you can put together a wavelets client that offers some good benefits over the existing client, then you will get the chance to put your name on a science paper and get the sort of WOW factor that is so far reserved only for Alex Khan! Good questions! Keep searchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 935696 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7112 Credit: 3,681,408 RAC: 916	Message 935702 - Posted: 24 Sep 2009, 19:59:26 UTC - in response to Message 935677.
	... the RF contribution could be rejected via the FFT filtering. ... Now, that would be really something if you could do that reliably... Filter to retrieve only the RADAR signal(s) and then exactly subtract those signals from the WU data to produce a 'clean' WU. However, I suspect that it is very much easier and far more reliable to do the blanking... I would also suspect that an upgrade to the RF shielding of the ALFA housing and connections would give a better result in the first place. Are any of the other experiments that use ALFA affected by the RADAR contamination? Might they be fixing the problem for us at source? Keep searchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 935702 ·

WinterKnight Volunteer tester Send message Joined: 18 May 99 Posts: 6799 Credit: 17,051,394 RAC: 28,850	Message 935794 - Posted: 25 Sep 2009, 7:51:49 UTC - in response to Message 935702.
	Now, that would be really something if you could do that reliably... Filter to retrieve only the RADAR signal(s) and then exactly subtract those signals from the WU data to produce a 'clean' WU. However, I suspect that it is very much easier and far more reliable to do the blanking... Now the normal way of doing that is to get the originator to transmit on two separate frequencies, with the signal applied to each transmitter in anti-phase. That way the two receivers receive the wanted signal in anti-phase and the unwanted local interference in-phase. Feed the output of each receiver into a longtail pair combiner and the interference cancels itself out. Now if we could just contact ET ..... Andy
	ID: 935794 ·

ML1 Volunteer tester Send message Joined: 25 Nov 01 Posts: 7112 Credit: 3,681,408 RAC: 916	Message 935819 - Posted: 25 Sep 2009, 13:53:28 UTC - in response to Message 935794.
	Now, that would be really something if you could do that reliably... Filter to retrieve only the RADAR signal(s) and then exactly subtract those signals from the WU data to produce a 'clean' WU. However, I suspect that it is very much easier and far more reliable to do the blanking... Now the normal way of doing that is to get the originator to transmit on two separate frequencies, with the signal applied to each transmitter in anti-phase. That way the two receivers receive the wanted signal in anti-phase and the unwanted local interference in-phase. Feed the output of each receiver into a longtail pair combiner and the interference cancels itself out. Now if we could just contact ET ..... Nice idea but rather complicated by frequency dependant dispersion and phase shifts over the vast distance of space... I still think the improved shielding at Arecibo and/or switching off the RADARs in the first place would be much more reliable! Keep searchin', Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3)
	ID: 935819 ·

1mp0£173 Volunteer tester Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0	Message 935889 - Posted: 25 Sep 2009, 20:31:33 UTC - in response to Message 935702.
	I would also suspect that an upgrade to the RF shielding of the ALFA housing and connections would give a better result in the first place. If we're looking for how the signal gets into the receiver front end, I suspect that the biggest source of local RF is getting in through the feed antenna. Eliminate that and you'd have no trouble. Seriously, though, if the receiver is very sensitive (and it needs to be), anything strong nearby is going to get in to it. ____________
	ID: 935889 ·

Dena Wiltsie Send message Joined: 19 Apr 01 Posts: 987 Credit: 392,779 RAC: 176	Message 935912 - Posted: 25 Sep 2009, 21:34:25 UTC - in response to Message 935819.
	I still think the improved shielding at Arecibo and/or switching off the RADARs in the first place would be much more reliable! Sad part is Arecibo may be switched off before the radar is. ____________
	ID: 935912 ·

1mp0£173 Volunteer tester Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0	Message 936076 - Posted: 26 Sep 2009, 17:08:50 UTC - in response to Message 935912.
	I still think the improved shielding at Arecibo and/or switching off the RADARs in the first place would be much more reliable! Sad part is Arecibo may be switched off before the radar is. Especially since the radar costs a lot less to run, and has a "job" that is generally considered more important than research. ____________
	ID: 936076 ·

Reuben Gathright Send message Joined: 8 Mar 01 Posts: 213 Credit: 14,594,579 RAC: 0	Message 936305 - Posted: 27 Sep 2009, 18:44:36 UTC - in response to Message 935459.
	Engage H@x0r power! Good luck! ____________ Overclock with the MSI G31M3-L and Intel E8600 3.33Ghz Intel D865GLC Socket 478 Motherboard ~How To Overclock The Eee ASUS 1005HA Netbook To 1.9Ghz~
	ID: 936305 ·

Misfit Volunteer tester Send message Joined: 21 Jun 01 Posts: 21680 Credit: 2,443,662 RAC: 0	Message 936355 - Posted: 28 Sep 2009, 0:34:31 UTC - in response to Message 935459.
	Speaking of splitters, I'm actually running (drumroll please) the first software radar blanked data through a splitter right now, and workunits will be distributed to the public fairly soon. This is still in test phase - we shall see if the software blanking performs better than (or worse, or the same as) the hardware blanking. I'm guess with a couple tweaks here and there my code will be far better. - Matt Does that mean it'll come to BETA first? ____________
	ID: 936355 ·

RoosStar Send message Joined: 16 Oct 99 Posts: 48 Credit: 2,756,909 RAC: 355	Message 936406 - Posted: 28 Sep 2009, 10:30:31 UTC - in response to Message 936355. Last modified: 28 Sep 2009, 10:33:13 UTC
	That is not intended. See Matts post Take Two ____________
	ID: 936406 ·

Message boards : Technical News : Take That, FAA Radar! (Sep 23 2009)