Drifting RFI: this time for sure

Message boards : Nebula : Drifting RFI: this time for sure
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile David Anderson
Volunteer moderator
Project administrator
Project developer
Avatar

Send message
Joined: 13 Feb 99
Posts: 99
Credit: 390,229
RAC: 0
Message 1851474 - Posted: 26 Feb 2017, 0:40:36 UTC

The drifting RFI algorithm removed non-RFI signals in a band of 1 KHz or so around the actual RFI. With the goal of being more selective, I made one more (hopefully the last) change to the drifting RFI algorithm, represented in the following picture:

We're trying to decide whether the signal at the common vertex of the triangles is drifting RFI. Before we only looked at the triangles above the signal (i.e. later in time). Now we're looking at the earlier triangles too, and demanding that both the upper triangle and the corresponding lower triangle (A and B in this case) have a statistical excess of signals. Actually, "corresponding" plus or minus one, to allow for bends in the RFI.

In other words, the algorithm is now time-symmetric.

This achieves the goal of removing less non-RFI. However, it more than double the size of the sliding window of signals, up to 150K signals or so, and the performance went way down, to the point where it would take about a week to run RFI removal.

The slow-down was mostly in R-tree access. Even log(N) can be significant. So instead of maintaining a single R-tree over the whole 2.5 MHz frequency band, I split it up into 2,500 R-trees, each for a 1 KHz band. I could do this because our access to the trees is localized in frequency.

That did the trick and now the time to do RFI removal is down to about 5 hours!
ID: 1851474 · Report as offensive

Message boards : Nebula : Drifting RFI: this time for sure


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.