(The title refers to a group of one-act plays by David Ives.)

A while back, Eric and I had the idea of "observation pruning": removing spikes whose duration (a function of their FFT length) is too short for the observation in which they occur.

For example, suppose a 13-second spike S (the longest spike, hence the narrowest bandwidth) occurs at a time when the telescope beam is moving at a speed where it crosses a point in the sky in 1 second. Then it's unlikely that the source of S is a cosmic signal; it's more likely RFI or noise. And if S is from a cosmic signal, that signal will probably be detected with greater power at a shorter FFT length.

Anyway, a few weeks ago I wrote some code to do observation pruning. But - surprise! - I didn't get it right the first time.

I decided to do observation pruning in 'nebula_score', the program that finds and scores multiplets, because that's the only place where we do things on a per-pixel basis, and fetching a pixel's observation history is expensive.

For a given pixel P, nebula_score reads and processes detections from a sky "disc" that includes parts of the 8 adjacent pixels. In my first attempt, I was comparing the intervals of these detections with the observation intervals of P (for the beam in which the detection occurs). This removed too many detections - not surprisingly, since the adjacent pixels have different observation intervals.

In my second attempt, I merged (for each beam) the observation intervals of all 9 pixels and compared against that. But this didn't help at all! I had a panicky feeling that there was some insidious problem in how I was computing the intervals, or in the data itself. I started working on tools for studying this.

But then I looked at the code for a while, and I realized the problem was that, after concatenating the 9 lists of observation intervals, I wasn't time-sorting the result, as required by the pruning logic.

A few seconds later, I realized that merging the lists was the wrong idea to begin with! I should just compare each detection against the observation intervals for the pixel that it's in. D'oh!

It wasn't immediately clear how to implement this efficiently, but eventually I figured out a way that, although not quite optimal, required changing only a few lines of code. Sweet.

I tested this on a few pixels and it seems to work; it removes about 10% of spikes, which is about what I expected. I did a scoring run using the new version of nebula_score; it's online now.

This experience highlights a big problem in Nebula: we're not doing design review or code review. The only programmers are Eric and me. Each of us has other projects - Eric has ICON, I have BOINC - as well as our Nebula work. Neither of us has quite enough time to look carefully at what the other one is doing. We pay a price for this: we make mistakes, and it costs us time. Eric has been mired in birdie_gen for months now, while I lurch along with RFI and multiplet stuff. And, or course, the real fear is that we're making undetected mistakes.

In case you, Dear Reader, are in a position to help, the Nebula source code repo is here. Warning: the learning curve is huge.