Wow! A lot has happened since we last spoke. Democracy prevailed in the U.S., for the time being. And COVID vaccination has started - Yay!

Lots of progress on Nebula too. We solved a nagging issue, and in the course of doing so we improved our understanding of what SETI@home is able to detect.

The nagging issue has to do with birdies - specifically, where we position them in the sky. We were creating 4000 birdies, and putting each one in a pixel in which spikes had been detected. The choice of pixel, and the position within the pixel, were random.

A Nebula program, birdie_gen.cpp, generate spikes for the birdies by scanning the trajectories of the 7 beams over the ~15 years of observation, seeing when a beam gets close to the birdie, and computing a set of spikes that the SETI@home client might have found.

A bit surprisingly, we ended up making any spikes at all for only 2700 out of the 4000 birdies. Then Eric made a bunch of changes to make the simulation more realistic:

• Taking into account the difference between signal bandwidth and width of the FFT bin.
  
• Taking into account the movement of the beam during the duration of the FFT bin.
  
• Modeling transmitters that are the surface of a planet and are therefore detectable only ~half the time.
  

After these changes only 1300 of the birdies had any spikes! This was no good - our plans to estimate sensitivity require finding birdies, and we can't find a birdie if it doesn't have any spikes.

So we started thinking about where we were putting the birdies. Pixel observation varies widely. Some pixels have been observed thousands of times and/or for long periods. But many have been observed only once or twice, for a few seconds, If we put a birdie in such a pixel, we might not generate any spikes, especially if the birdie and the beam path are in opposite corners of the pixel.

So I restricted birdies to pixels with at least 60 seconds of observing time. This helped, but only slightly.

Then Eric made a key realization: we can "detect" a birdie best at the FFT length whose frequency resolution is closest to the birdie's intrinsic bandwidth. But this will happen only if we have an observation that includes the time interval of the FFT bin. So if the birdie bandwidth is small (say, .05 Hz) we'll detect it best at our longest FFT length: 128K samples, or 13 seconds, or .07 Hz. But we'll detect it with maximum power only if our observation includes an entire FFT bin. A 26-second observation is guaranteed to include one 13-second FFT bin; a 39-second observation is guaranteed to contain two.

Now, a pixel could have 60 seconds of observing time, but if that consists of a lot of 1 or 2 second observations, we probably won't generate any spikes for that narrow-bandwidth birdie.

This was like a light bulb going on. We've always thought of observing time as the key factor in sensitivity - in other words, that we could say "for pixels that we've observed for at least 10 minutes, our sensitivity is X". But what matters - at least for narrow-bandwidth signals, which are inherently easier to find because they're less like noise - is not total time but rather just the intervals that are long enough to hold the optimal FFT bin.

Nebula has a program "pointing.cpp" that analyzes the beam pointing trajectories and generates per-pixel observing information. I extended this so that for each pixel it computes the length of the longest and send-longest observations.

Recall that a multiplet must have at least two detections. So we can't "find" a birdie unless it has at least two spikes. To make this likely, we need to put it in a pixel with sufficiently long observations for the birdie's intrinsic bandwidth. Specifically: if the duration of the closest FFT length is T, then we want a pixel for which

longest observation >= 3T
or
second longest observation >= 2T

I made this change to birdie_gen. We also had a discussion about planetary transmitters. If the transmitter is hidden half the time, it might not be on during the critical observation, in which case our careful selection of pixel is for naught. Dan argued that if ET is serious about being detected, they'd put two transmitters on opposite sides of the planet, so that one of them is always visible. We decided to go with this.

We re-ran birdie_gen with high hopes! But alas there was no improvement.

Then, today, Eric found two critical bugs in the function (which I'd written) to get information about a pixel's observations: the arguments to fseek() were reversed, and one of them was a file offset that was overflowing 32 bits. After fixing these problems, we're now generating spikes for 3945 of the 4000 birdies, which is plenty. Phew!

---

This means we have to recalibrate our view of what SETI@home is good at. Previously, we thought: SETI@home has covered a lot of the sky - about 23% - and we did 128K FFTs everywhere, so we can find narrow-band signals in 23% of the sky. But that's not the case. We can actually hope to find narrow-band signals only in the parts of the sky where we had long enough observations. And for 128K FFTs, that's only 0.5% of the sky.

That's a little disappointing, I guess. Of course, we're sensitive to broad-band signals and pulsed signals over a wide area of sky.

But there's an important lesson here for future radio SETI sky surveys: if possible, make each observation long enough for your longest FFT. I'm sure Dan and Eric already knew this, but since we never controlled the pointing of Arecibo, we didn't think about it (until recently).

BTW: covering the Arecibo sky with 13-second observations - given the ALFA beam size - would take about a year, according to my calculations.