Instructions for evaluating Nebula output

Each Nebula run generates various output, most notably lists of birdies, multiplets, and pixels. The web interface lets you examine these lists, and "drill down", e.g. to the list of signals that comprise a multiplet, or to a waterfall plot of the signals surrounding a given signal.

Nebula's algorithms - e.g. for RFI removal and multiplet finding - are developed in large part heuristically: we look at the output and find places where the algorithms didn't work well. This page describes how to do this. Everyone is invited to help (though you'll have to learn about Nebula in detail in order to help effectively).

As you identify problems in the output, use Nebula's Bookmark feature to record them, and email David Anderson.

Look at birdies

Birdies are surrogate ET signals, so we want Nebula to detect them; i.e. they should have high-ranking multiplets.

Do the birdie signals look plausible?

Look at a few birdies. Look at the plot of their signals (the "plot" button next to # spikes). Over short time scales, the frequences should be close (within a few dozen Hz). For bary birdies, all the signals should be close.

Are birdie signals being flagged as RFI incorrectly?

Look at birdies with lots of RFI signals. Look at the waterfall plots for those signals. Make sure that RFI removal is working correctly. The birdie signals by themselves should not trigger any of the RFI algorithms, but they might be located in the time/freq vicinity of actual RFI.

Are there birdies that should have multiplets but don't?

If a birdie has more than a few signals, it should be detected with a multiplet of the same type (bary/nonbary) and possibly the other type as well.

If not, go to its pixel page and click "Scoring details" (generates lots of cryptic output; I can explain it).

Are birdie multiplets as high scoring as possible?

Essentially, this means checking that a birdie's multiplets include as many of the birdie signals as they should. The birdie page shows a list of multiplets. Each of them as a "plot" link that shows the signals of both the birdie and the multiplet. Ideally these should be more or less the same.

Recall that both birdies and multiplets can be bary or non-bary. A non-bary birdie may have signals from several observations, which may be far apart (up to 150 KHz) in bary frequency. A non-bary multiplet for such a birdie should include signals from all the observations.

A bary multiplet for a bary birdie should include all its signals, except for those that overlap in time (i.e. signals with different FFT lengths at about the same time).

Look at high-scoring non-birdie multiplets

Are we failing to reject RFI?

Look at high-scoring non-birdie multiplets. Look at the waterfall plots of their signals. Check for possible RFI. NOTE: we don't yet have instructions for how to identify RFI. The gist of it is: if a group of signals is likely RFI if

Are we scoring multiplets appropriately?

Look at lots of non-birdie multiplets. Look at things like their time entropy, signal power, freq stddev, etc. Look for multiplets that seem to be scored too high or low.




 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.