Instructions for evaluating Nebula output

Each Nebula run generates various output, most notably lists of birdies, multiplets, and pixels. The web interface lets you examine these lists, and "drill down", e.g. to see the list of signals the comprise a multiplet, or to see a waterfall plot of the signals surrounding a given signal.

Nebula's algorithms - e.g. for RFI removal and multiplet finding - are developed in large part heuristically: we look at the output, find places where the algorithms didn't work well, and improve them accordingly. This page describes how to do this. Everyone is invited to help (though you'll have to learn about Nebula in detail in order to help effectively).

As you identify problems in the output, use Nebula's Bookmark feature to record them, and email David Anderson.

Look at birdies

Birdies are surrogate ET signals. We want them to have multiplets, and we want the multiplet scores to be high.

Are birdie signals being flagged as RFI incorrectly?

Look at birdies with lots of RFI signals. Look at the waterfall plots for those signals. Make sure that RFI removal is working correctly. The birdie signals by themselves should not trigger any of the RFI algorithms, but they might be located in the time/freq vicinity of actual RFI.

Are there birdies that should have multiplets but don't?

If a birdie has more than a few signals, it should be detected with a multiplet of the same type (bary/nonbary) and possibly the other type as well.

If not, go to its pixel page and click "Scoring details" (generates lots of cryptic output; I can explain it).

Are birdie multiplets as high scoring as possible?

Essentially, this means checking that a birdie's multiplets include as many of the birdie signals as they should.

Recall that both birdies and multiplets can be bary or non-bary. A non-bary birdie may have signals from several observations, which may be far apart in bary frequency. A non-bary multiplet for such a birdie should include signals from all the observations.

A bary multiplet for a bary birdie should include all its signals, except for those that overlap in time (i.e. signals with different FFT lengths at about the same time).

Look at high-scoring non-birdie multiplets

Are we failing to reject RFI?

Look at high-scoring non-birdie multiplets. Look at the waterfall plots of their signals. Check for possible RFI. NOTE: we don't yet have instructions for how to identify RFI. The gist of it is: if a group of signals is likely RFI if

Are we scoring multiplets appropriately?

Look at lots of non-birdie multiplets. Look at things like their time entropy, signal power, freq stddev, etc. Look for multiplets that seem to be scored too high or low.




 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.