As described in my previous blog post, we recently realized that our search for narrow-band signals was less effective during observation periods where the telescope beams are moving fast, and that such periods comprise the majority of our data. People then pointed out that - since much of the SETI@home front-end computation involves looking for narrow-band signals - it seems like we wasted a lot of computing time.

I'd like to respond to this. I take SETI@home - and volunteer computing in general - very seriously. When people run a BOINC project, they invest time and energy, and their electric bills may increase. BOINC projects have an obligation to use their contributions efficiently.

In this case, we could have been more efficient, but the computing time spent doing long FFTs on short observations wasn't wasted. First, we detected narrow-band RFI (which generally doesn't depend on pointing) during these periods; this is important e.g. for identifying RFI "zones". Second, our current notion of "observation" is conservative; we count it as an observation only when the beam is close to the pixel center. If an ET signal is powerful, it could be detected even if the beam is farther away. So the fraction of the sky where we can detect powerful narrow-band signals is larger than the number I gave.

But it's true that it took us a long time to realize that narrow-band signals and short observations don't go well together. Here's a long-winded explanation of why this happened:

SETI@home "piggybacks" on Arecibo; other science projects (like pulsar search and Hydrogen mapping) control the telescope pointing. When we designed SETI@home, and for the first 7 years of data collection, we used a dedicated "flat feed" antenna, which has a relatively large beam. When the main antenna (in the Gregorian dome) is tracking a point in the sky, the flat feed's beam moves across the sky at about twice the sidereal rate (i.e. the rate of Earth's rotation).

Given this slew rate and beam diameter, it takes the flat feed's beam about 13 seconds to pass over a point. Frequency resolution is important when looking for narrow-band signals. We chose our longest FFT length - i.e. our best frequency resolution - accordingly: 128K samples at 9765.625 samples/sec is 13.4 seconds. We tailored our search algorithm to the data source.

In 2006 we switched from the flat feed to the new ALFA receiver, which has some big advantages:

• It uses very low noise cryogenic receivers.
  
• It has a larger collecting area.
  
• It has 7 beams, which lets us cover the sky faster and also lets us do "multi-beam" RFI detection, which rejects similar signals that occur in 2 beams at the same time.
  

When we switched to ALFA we didn't know what its patterns of pointings would be like. It turned out to move faster, on average, than the flat feed had; we didn't initially know this. And because the beams are smaller, observations are shorter. Our application kept using long FFT lengths, even on data from short observations. We could have made the application more efficient by doing only shorter FFT lengths for such data.

Around the same time, SETI@home transitioned to a sort of "maintenance mode": we focused on system administration - keeping our servers and databases running - and porting the application to GPUs and Android. This kept us busy; we had full-time jobs doing non-SETI things, and SETI@home's funding sources (other than donations) had dried up. SETI@home kept accumulating detections (spikes, Gaussians etc.), but we didn't study them.

In 2016, I started Nebula, and we began to analyze detections. It took a couple of years to get the Nebula pipeline (RFI removal and multiplet finding) working. We created the "birdie" mechanism, which lets us test algorithms. Everything we looked at required rethinking and often replacement. At the same time, we undertook the huge project of downsizing our server complex.

So it wasn't until recently that - while trying to figure out why narrow-band birdies weren't producing spikes - we realized that looking for narrow-band signals works best when you have long observations. In retrospect we could have figured this out much earlier.

Anyway, the lessons I've learned from this include:

1. Don't start doing large-scale computing until your entire scientific pipeline is in place, or at least enough of it to let you assess the scientific value of the computing results.
   
2. Run the pipeline as soon as the computing results start to come in; don't wait until the computing is all done.
   
3. Make sure you have enough funding to do 1) and 2).