|
1)
Message boards :
News :
YouTube video on SETI@home
(Message 2153977)
Posted 1 Feb 2026 by David Anderson
Post: Anton Petrov made <a href=https://www.youtube.com/watch?v=Gt07R_amRT8>a nice video</a> about SETI@home and the recent papers. |
|
2)
Message boards :
News :
UPI story on SETI@home
(Message 2153958)
Posted 30 Jan 2026 by David Anderson
Post: Check out <a href=https://www.upi.com/Top_News/US/2026/01/30/popular-project-eye-space-signals-ending-hunt-continues/9001769613937/>a story by United Press International</a> about SETI@home. |
|
3)
Message boards :
News :
SETI@home papers accepted for publication
(Message 2149919)
Posted 18 Jun 2025 by David Anderson
Post: <div style="max-width: 700px;">Two papers on SETI@home will be published in <a href=https://iopscience.iop.org/journal/1538-3881>The Astronomical Journal</a>, a well-regarded scientific journal: <li> <a href=https://setiathome.berkeley.edu/SETI_Home_instrument_rev2_final.pdf>SETI@home: Data Acquisition and Front-End Processing</a> describes SETI@home's data recorder, splitter, and client program. It covers the five detection types, their parameters and statistics, and the algorithm for finding them. <li> <a href=https://setiathome.berkeley.edu/SETI_Home_Nebula_final.pdf>SETI@home: Data Analysis and Findings</a> describes the back end (Nebula) and its results: RFI removal, candidate finding and ranking. It explains how artificial signals, or 'birdies', were used to optimize algorithms and estimate overall sensitivity. For details, see <a href=https://setiathome.berkeley.edu/forum_thread.php?id=86160>an entry in the Nebula blog</a>. </div> |
|
4)
Message boards :
Nebula :
SETI@home papers accepted for publication
(Message 2149918)
Posted 18 Jun 2025 by David Anderson
Post: <div style="max-width: 700px;">Great news! Two papers on SETI@home will be published in <a href=https://iopscience.iop.org/journal/1538-3881>The Astronomical Journal</a>, a well-regarded scientific journal: <li> <a href=https://arxiv.org/abs/2506.14718>SETI@home: Data Acquisition and Front-End Processing</a> describes SETI@home's data recorder, splitter, and client program. It covers the five detection types, their parameters and statistics, and the algorithm for finding them. <li> <a href=https://arxiv.org/abs/2506.14737>SETI@home: Data Analysis and Findings</a> describes the back end (Nebula) and its results: RFI removal, candidate finding and ranking. It explains how artificial signals, or 'birdies', were used to optimize algorithms and estimate overall sensitivity. It took a long time to write the papers. We started in 2018. In March 2024, after a two-year hiatus, Eric and I resumed work on them. We met twice a week, in person (that was key). By December 2024 the papers were in pretty good shape, and we submitted them to the journal. The referee reports came back the following month. They were positive, but had long lists of suggested changes and additions. The referees did an excellent job; they deserve big thanks. We revised the papers, addressing all the referees' suggestions. These changes greatly improved the papers. We resubmitted them in April 2025, and they were accepted in June 2025. They'll appear together in a future issue of the journal, TBD. There will probably be another paper at some point. Using the FAST observatory in China, we're reobserving the 92 top-ranking candidates found by Nebula. Eric Korpela, Dan Werthimer, and Wei Liu are working on this together with colleagues in China; I don't think they'll need my help. BTW: I think it's unlikely that an ET signal will emerge from this; none of the candidates found by Nebula really stood out. <hr>There are two general kinds of science papers. A "technical report" describes an experimental setup and the data it produces. The paper graphs the data. It suggests conclusions but doesn't prove them. It presents engineering, not science. On the other hand, a "research paper" poses and answers scientific questions. These answers must be proved, or at least backed up, with statistically valid data analyses. Eric and I tried to write research papers, not technical reports. The central thing we tried to prove: if there were certain types of ET radio signals, above certain power levels, in certain areas of the sky, SETI@home would have discovered them. Making this rigorous was tricky. To do so, we made the distinction between: <li> 'Event sensitivity': the minimum power of momentary signals the hardware can reliably pick up. <li> 'Candidate sensitivity': the minumum power of signals (possibly of multi-year duration) that reliably make it through RFI rejection and candidate selection, and into the final list that human experts look at. For the purpose of finding ET signals, candidate sensitivity is what matters. As far as I know, SETI@home is the first radio SETI project to define this quantity, much less estimate it (which we did). As we worked on the papers, we decided that each Conclusions section should have three parts: <li> What SETI@home did that was new and different from other projects. <li> If we could start over, what we'd do differently. <li> Lessons for future radio SETI projects, especially sky surveys. This helped bring our ideas into focus. We did lots of brainstorming, from which new ideas magically emerged. Super fun. I encourage you to read the Conclusions sections of each paper. They're fairly non-technical. One conclusion of the Nebula paper (as mentioned in previous blog entries) is that commensal observing — recording data while the telescope is being used for other purposes — isn't ideal for radio SETI sky surveys. The telescope often moves too fast to provide the long observations needed for narrow frequency channels (which are needed for high sensitivity). Also, the slew rate varies widely during commensal observing, and it's hard to develop RFI-removal and candidate-finding algorithms that work for a wide range of slew rates. It's better if the telescope moves in a slow, regular pattern. We concluded that future SETI@home-type projects should ideally get at least two years of dedicated telescope time (this is unlikely in the near future). I'm very happy about how the papers came out. Maybe I'm biased, but I think they're the best radio SETI papers ever written. <hr>The publication of the papers gives a purpose and meaning to the vast volunteer effort (and electric bills, and carbon emissions) that went into SETI@home. For me, it's a deeply satisfying conclusion to 25 years of hard work. This includes 15 years developing the original SETI@home, then developing BOINC and porting SETI@home to it. This was a lot of work, but it was fun; I enjoy developing new software. I spent the next 7 years (2016 to 2022) developing Nebula. I've <a href=https://continuum-hypothesis.com/software.php>written a lot of code in my life</a>, but this was the hardest thing I've ever done (see earlier blog entries for details). My brain-power, patience, concentration, and motivation were pushed to their limits. Eric and Dan helped a lot with developing RFI and candidate scoring algorithms, but the programming and debugging was a solitary endeavor. There were some fun Eureka! moments, but mostly it was a long, hard slog. <hr>I haven't forgotten SETI@home's original spirit and ideals; they kept me going through all this. Intelligence and its products (science, philosophy, arts) are the greatest things I know of. Hence the question of whether intelligence exists outside Earth — and if so, the nature of its civilization and collective knowledge — is the most compelling one I can imagine (more so, for example, than the various open questions in physics and cosmology). Lots of people feel the same way. There's a species-wide eagerness to find other intelligent life and transcend the loneliness of our tiny planet drifting in the void (see Carl Sagan's <a href=https://www.planetary.org/worlds/pale-blue-dot>Pale Blue Dot</a> essay). I think of SETI@home as a million people reaching out together into space, hoping to find other minds. SETI@home is perhaps quixotic, but it's unparalleled among human endeavors. Let's reflect on this, and be proud of what we accomplished together. </div> |
|
5)
Message boards :
News :
RIP Jimmy Carter
(Message 2144633)
Posted 30 Dec 2024 by David Anderson
Post: Carter wrote the following on June 16, 1977 and placed it in Voyager 1, which is the most distant human-made object from Earth: <i>This Voyager spacecraft was constructed by the United States of America. We are a community of 240 million human being among the more than 4 billion who inhabit the planet Earth. We human beings are still divided into nation states, but these states are rapidly becoming a single global civilization. We cast this message into the cosmos. It is likely to survive a billion years into our future, when our civilization is profoundly altered and the surface of the Earth may be vastly changed. Of the 200 billion stars in the Milky Way galaxy, some – perhaps many – may have inhabited planet and spacefaring civilizations. If one such civilization intercepts Voyager and can understand these recorded contents, here is our message: “This is a present from a small distant world, a token of our sounds, our science, our images, our music, our thoughts and our feelings. We are attempting to survive our time so we may live into yours. We hope someday, having solved the problem we face, to join a community of galactic civilizations. This record represents our hope and our determination, and our good will in a vast and awesome universe.” --- Jimmy Carter, President of the United States of America, the White House, June 16, 1977</i> |
|
6)
Message boards :
News :
Nebula progress report
(Message 2115485)
Posted 3 Mar 2023 by David Anderson
Post: Check out our latest newsletter: Final update. |
|
7)
Message boards :
Nebula :
Final update [not actually]
(Message 2115484)
Posted 3 Mar 2023 by David Anderson
Post: <div style="width:640px">Sorry for the long gap between reports. Since last April, my colleagues (Dan Werthimer, Eric Korpela, Jeff Cobb, Wei Liu) have done two re-observation sessions at FAST, looking at a few dozen of the top-scoring multiplets found by Nebula. So far it's been all barycentric spike/gaussian multiplets. With our remaining observing time we'll look at some pulse/triplet and autocorr multiplets, and some non-barycentric, as well. We haven't examined the re-observation data yet, and it's unclear how we're going to do this. For barycentric multiplets I think the plan is to manually look at waterfall plots, possibly of the raw FFT data, possibly after processing by the SETI@home client (running on our own computers). For non-barycentric this isn't feasible - the frequency range is big and there's lots of RFI. I don't know; I'm out of the loop at this point. Whether or not we find ET, there's still the matter of writing papers describing what we did, and (in the case of Nebula) giving sensitivity bounds. We had planned to write two papers, one about the front end and one about the back end (Nebula). These papers are about 80% done, but there has been no progress on them in over a year. I can't go into details. I've done everything I can to complete these papers, but I can't do it by myself. My involvement in SETI@home has ended. I'd like to thank everyone who has participated, especially those who have read and commented on these Nebula reports. Since then, I worked for a while on <a href=https://github.com/panoseti/panoseti/wiki>PanoSETI</a>, an optical SETI project. I've also worked on two music-related projects: <a href=https://music-match.org/>Music Match<a> and <a href=https://github.com/davidpanderson/numula/wiki>Numula</a>. I'm also working as a contractor for <a href=http://imslp.org/wiki/Main_Page>IMSLP</a>, the online classical music score library. And I continue to run the <a href=https://boinc.berkeley.edu/>BOINC project</a>. </div> |
|
8)
Message boards :
Nebula :
Update
(Message 2097897)
Posted 16 Apr 2022 by David Anderson
Post: <div style="width:640px">Nebula has done its job - we have millions of "multiplets" - candidate signals - in 6 categories, ordered by 7 different score functions. Dan and Eric will, at some point, manually examine lots of these and pick 100 or so for re-observation at FAST, where we've been granted 20 hours of observing time. To do this re-observation , we need to build a new data recorder - hardware and software. Much of this work is being done by Wei Liu, an astronomy post-doc who's visiting here for 2 years. We recently had an in-person meeting with one of the leaders of FAST to discuss the technical details of the re-observation. I'm working on the paper about Nebula and its results. This is going slowly. I've recently started going over the paper with Bruce Allen (leader of Einstein@home), whose input and questions are very valuable. </div> |
|
9)
Message boards :
News :
Nebula progress report
(Message 2088645)
Posted 21 Nov 2021 by David Anderson
Post: Check out our latest newsletter: New zones, a milestone, and next steps. |
|
10)
Message boards :
Nebula :
New zones, a milestone, and next steps
(Message 2088641)
Posted 21 Nov 2021 by David Anderson
Post: <div style="width:640px">My <a href=https://setiathome.berkeley.edu/forum_thread.php?id=85783>last missive</a> described how we changed zone RFI removal to work for pulses. In this case the "zones" are ranges of pulse period rather than frequency: a lot of the RFI is aviation radar that results in pulses with periods that are multiples of 12 seconds. To figure things out, Eric looked at histograms of pulse counts as a function of period. Since then, he has repeated this exercise for triplets, and then autocorrs. The details were different in each case, but the basic idea was the same. Once this was done - a couple of weeks ago - I did another Nebula run on the full sky - i.e. finding and scoring multiplets in all pixels. Eric, Dan and I then examined the top-scoring pulse/triplet and autocorr multiplets, looking for RFI that should have been removed by the zone filter. Good news! Although some multiplets still had this sort of RFI, many of them didn't - enough that we'll be able to find plenty of multiplets that are worth re-observing. So until further notice we're done with computing. No more algorithm-fiddling or scoring runs. Nebula has done its job. This is a huge milestone for SETI@home, and for me personally. I've been working on Nebula for 5 years, and it's been some of the most challenging work - algorithm design, programming, and debugging - I've done. The immediate next step is for us to go through the top-scoring multiplets - of all the various detection types, bary/non-bary, and scoring variants - and pick 100 or so to re-observe at FAST. That's about how many we'll have time for in the 24 hours of observing time that we've been granted - we'll observe each one for a minute or two, and it can take several minutes to slew from one sky location to the next. After that, we'll need to adapt our computers at FAST (which currently run SERENDIP spectrometers) to produce the time-domain data needed for SETI@home. Then we have to figure out how to analyze this data; we'll probably do the first part using the existing SETI@home client running on cluster nodes either in China or at the Atlas cluster in Hannover. After that we'll need take these detections and decide whether they "confirm" the corresponding multiplet. We haven't figured the details. In the case of barycentric multiplets - where we know what frequency to look at - this might involve manually looking at waterfall plots of the new detections. For non-barycentric multiplets - where the frequency could be anywhere in a wide range - we could add the new detections to the SETI@home detections, re-run Nebula (at least the multiplet-finding part) and see if it finds the original multiplets with additional detections that increase the score. </div> |
|
11)
Message boards :
News :
Nebula progress report
(Message 2085144)
Posted 28 Sep 2021 by David Anderson
Post: Check out our latest newsletter: Finding a Pulse. |
|
12)
Message boards :
Nebula :
Finding a pulse
(Message 2085143)
Posted 28 Sep 2021 by David Anderson
Post: <div style="width:640px">Here, "pulse" refers to the detection type, though I should add that SETI@home itself still has a pulse in spite of the low rate of progress over the last couple of months. I've become involved in another SETI project, <a href=https://oirlab.ucsd.edu/PANOSETI.html>PanoSETI</a>. And there are other factors. Anyway, the good news is that I think we're done with narrow-band signals (spikes and Gaussians). We're happy with RFI removal, and with multiplet finding and scoring. So we've turned our attention to pulsed signals (pulses and triplets). Until recently we've ignored these to some extent, because the birdie mechanism doesn't extend to pulsed signals so we don't have it to guide us. Recently, when Eric examined the top-scoring pulse/triplet multiplets, he found that they consisted of detection that were zone RFI: in particular, their periods were multiples of 12s, which is the period of some kind of radar that pollutes our data. Now, we have a "zone" RFI filter that is supposed to find stuff like this. We wrote this filter for narrow-band signals, where the zones are in frequency space: TV station sidebands and the like. We used the same algorithm for pulses and triplets, but using zones in period (i.e. the period of the pulse, or the spacing of the triplet) rather than frequency. (Actually, the zones are in log(period)). But it turned out - once Eric looked that the aforementioned top-scoring multiplets - that this didn't work. The reason is that the periods of pulses and triplets are (unlike spike/gaussian frequency) not smoothly distributed. They're contentrated at particular values that arise from our FFT lengths and the algorithms we use to find pulses and triplets. Our zone-finding algorithm wasn't finding RFI - it was just finding groups of detections at these periods. So Eric put on his thinking cap, looked at a lot of histograms in log(period) space, and came up with new zone-finding algorithms for pulses and triplets. I don't know exactly what these algorithms are - Eric did the work in IDL and just sent me a list of zones. Anyway, we're still working out some kinks, but this looks like the right way to go. We'll know when we re-run the pipeline and look at the new pulse/triplet multiplets. In other news: our application for time at <a href=https://en.wikipedia.org/wiki/Five-hundred-meter_Aperture_Spherical_Telescope>FAST</a> to re-observe candidates was approved - I may have mentioned this in an earlier post - and we're scrambling to make a data recorder to use at FAST. We can't - for various reasons - use the same approach we used at Arecibo. After we record this data, BTW, we'll analyze it with the SETI@home client program. But we'll do this on cluster nodes, not home PCs. There won't be very much data, and it will be easier to handle that way. But who knows - maybe this collaboration will lead to an eventual un-hibernation. </div> |
|
13)
Message boards :
News :
Nebula progress report
(Message 2081557)
Posted 4 Aug 2021 by David Anderson
Post: Another in the All in the Timing series. |
|
14)
Message boards :
Nebula :
All in the Timing V
(Message 2081556)
Posted 4 Aug 2021 by David Anderson
Post: <div style="width:640px">We (mostly Eric) examined lots of top-ranking multiplets from the recent all-pixels scoring run. Many of the multiplets were clearly RFI, and we'll be tweaking the filters a bit to reduce the number of such cases. In addition, we found a multiplet whose time factor seemed way too high. We figured out why. Here's what happened: when looking for multiplets in a pixel P, we assemble the detections from a "disk" that includes all of P and parts of the 8 adjacent pixels. For each pixel, we know the time intervals during which we observed it (more precisely: the intervals during which a beam was within a half beam-width of the pixel center). The time factor compares the time we observed P (the central pixel) with the time covered by detections. For example, if we observed for 26 seconds and the multiplet has two 13-second spikes, that would get a high score. If we observed for 1000 seconds, it would get a lower score. The problem is that the observation times of adjacent pixels can be wildly different. In the problem case, the central pixel was observed for 0.3 seconds and one of the adjacent pixels was observed for 1000s of seconds. The multiplet included spikes from the adjacent pixel (in fact, it only had spikes from that pixel). Their duration was way more than 0.3 seconds, so the multiplet (incorrectly) got a high time factor. How to address this? One approach is to include, in the calculation of time factor, the observation intervals of the adjacent pixels as well as those of P (or perhaps, for a given multiplet, just the pixels for which the multiplet has detections). But this is flawed: the detection disk includes only a part of each adjacent pixel; we shouldn't include times when the beam didn't overlap this part. Another approach is to include, in the calculation of time factor, only the multiplet's detections that are in the central pixel. I think this is the way to go; it's certainly the simplest option. It would (correctly) give a low time factor score in the above example. If a multiplet consists primarily of detections from an adjacent pixel, this approach would tend to give it a low time factor score. But that's OK - when we score the adjacent pixel we'll see a (probably better) version of this multiplet, and it would get a higher time factor in that pixel. A related issue: during multiplet finding we do something called "observation consistency pruning", which means e.g. ignoring a 13-second spike that occurs during a 1 sec observation. We do this only for detections that lie in the central pixel, because that's the only pixel for which we've read the observation intervals. In theory we could do this for adjacent pixels too, but I don't think it's worth it. </div> |
|
15)
Message boards :
News :
Nebula progress report
(Message 2080046)
Posted 15 Jul 2021 by David Anderson
Post: Oops! Fixed. -- David |
|
16)
Message boards :
News :
Nebula progress report
(Message 2079760)
Posted 12 Jul 2021 by David Anderson
Post: ... wherein it's asserted that we're in the home stretch. |
|
17)
Message boards :
Nebula :
The home stretch
(Message 2079759)
Posted 12 Jul 2021 by David Anderson
Post: <div style="width:640px">There, I said it: after 25 years working on SETI@home, and 5 years working on Nebula, we're finally on the home stretch. The finish line (producing a final candidate list, and writing a paper) is in view, maybe a few months off. It's been a slog, and I'm eager to have it behind me. <h3>A full-sky Nebula run</h3> Over the last several years (as chronicled here) we've been developing and refining algorithms for detecting RFI, generating birdies, finding signal candidates (multiplets) and scoring these candidates. For these purposes it wasn't necessary to look for multiplets in all 15 million pixels; I generally looked in only 256K per cycle, sometimes 1M. Finding multiplets uses lots of computing and produces lots of files (2 per pixel). I didn't want to overstay my welcome at the Atlas computing cluster. But a few weeks ago we decided that these algorithms had reached the point of being good enough, and I did, for the first time, a Nebula run that scored all 15 million pixels. This went faster than I thought it would. I processed batches of 1.25M pixels at a time, and each batch took only a few hours, running on about 1000 cluster nodes. It turned out that the "multiplet uniqueness" step - ensuring that multiplets in adjacent pixels are disjoint - would take pretty long, like a week. So I did a minor rewrite of this program to increase its efficiency; now it's down to about a day. Also, copying the 30M or so files from Atlas (Germany) to Centurion (Berkeley) took a couple of days. It seemed to confuse rsync; I had to do a couple of tries to get everything copied. <h3>Keeping birdies separate</h3> Until now, we've processed and scored birdie detections and real detections together. I decided to change things so that we do two separate Nebula runs: a) using only real (non-birdie) detections, and scoring all pixels; b) using real and birdie detections, and scoring only birdie pixels. There are two reasons for this: <ul> <li> A birdie could mask (i.e. prevent us from detecting) an actual ET signal in the same pixel. The odds of this are small - with 3,000 birdies, only .02% of pixels have a birdie - but still. <li> We may want to change the number or parameters of our birdies, perhaps repeatedly. It would be good to be able to do this without rescoring all pixels (and transferring all the files). </ul> It took me a while to figure out a clean way to keep the two runs separate, in terms of files. I was already keeping all score-related files in a subdirectory, score/. What I settled on is: <ul> <li> On Atlas, everything stays the same. When we do a run, either birdie or non-birdie, the results go in score/. <li> On Centurion, we have a new directory score_birdie/. After a birdie run, we copy score/ on Atlas to score_birdie/ on Centurion. </ul> This meant changing the scripts and PHP pages that run on Centurion to look for birdie-related data in score_birdie/, rather than in specially-named files in score/. This wasn't hard to do, and actually makes things a bit simpler. <h3>Human rating of multiplets</h3> The final output of Nebula is a bunch of score-ranked lists of multiplets. We already know that some fraction of these will be RFI of the sort that's apparent to a human observer, but hard to identify algorithmically. That's OK. The goal our RFI algorithms is not to remove 100% of RFI; doing so would probably remove ET signals too. The goal is to remove enough RFI that a good fraction (say, at least half) of the top-ranking multiplets are not obvious RFI. The final (post-Nebula) stages of SETI@home are <ul> <li> Manually examine the top few thousand multiplets, in all the various categories and score variants, and remove the ones that are obvious RFI. At least initially we'll do this as a group, on Zoom, to make sure we agree on what constitutes obvious RFI. <li> Make a list of the multiplets that remain; re-observe those spots in the sky (hopefully using FAST), analyze the resulting data (probably using the SETI@home client running on a cluster) and see if we find detections consistent with the multiplets. If we do, maybe that's ET. </ul> So I extended our existing "bookmark" system to let you rate multiplets. When you bookmark a multiplet you can now give it a 0-10 rating, as well as a comment. The web page for each multiplet shows the ratings that have been reported so far. This mechanism is intended for use by our group (me, Eric, Dan, Jeff) but any SETI@home user can browse and rate multiplets. Feel free to do so! </div> |
|
18)
Message boards :
News :
Nebula progress report
(Message 2077682)
Posted 10 Jun 2021 by David Anderson
Post: Check out <a href=https://setiathome.berkeley.edu/forum_thread.php?id=85745&postid=2077681#2077681>recent advances in drifting RFI removal</a>. |
|
19)
Message boards :
Nebula :
Drifting (on a sea of...)
(Message 2077681)
Posted 10 Jun 2021 by David Anderson
Post: <div style="width:640px">We've been working almost entirely on refining the drifting RFI algorithm. This - I hope - is the last big piece before we do our final Nebula run and finish the paper. <p> The goal, as with all RFI algorithms, is to: <ul> <li> <b>Remove all the RFI</b>. We've <a href=https://setiathome.berkeley.edu/nebula/bookmark.php?action=list>bookmarked</a> a lot of examples of drifting RFI. There's a spectrum: some examples are clearly RFI, others are less obvious. When we change the algorithm, we look at these examples to make sure we're still removing at least the obvious ones. We also look at the top-ranking spike/gaussian multiplets. It's OK if a small fraction of these are RFI; we can skip over them in the manual inspection process. But if most of them are RFI we need to change the algorithm. <li> <b>Remove only RFI</b>. We don't want the algorithm to remove an ET signal. To check for this, we see what fraction of birdie spikes are flagged as drifting RFI. This is inevitably nonzero - some birdie detections happen to lie in regions of drifting RFI - but it shouldn't exceed a few percent. Also, we monitor the fraction of all spikes flagged as drifting RFI; 10% is plausible; 20% is probably too high. </ul> Recall that the drifting algorithm takes a "vertex" detection D and forms two fans of triangles in time/frequency space, emanating from D in the positive and negative time directions. We count the number of detections in each triangle that are "far" from D (a couple of beam widths or more) and compute a probability for each triangle; the more detections in the triangle, the lower the probability. <p> Recent changes: <ul> <li>Previously, we looked at the product of the probabilities of opposing triangles; if this was below a threshold, we flagged both of them as RFI. This didn't work well in some cases; we tried various alternatives. Our current approach uses two thresholds. If a triangle's probability is below 1e-8, we flag it as RFI. If a triangle and an opposing triangle are both below 1e-4, we flag them both. <li>Previously, when we flagged a triangle as RFI, we flagged all the detections in the triangle, including its vertex. But there are cases (including lots of birdies) where a triangle has lots of close, non-RFI detections, that happen to be far from the vertex detection; these were erroneously getting flagged. So we changed to a "vertex-only" scheme where - if any triangle is flagged as RFI - we flag only the vertex detection, not the detections in the triangles. <li>The vertex-only policy introduced a wrinkle: before we apply the drifting algorithm, we group detections into "clusters" of nearly-identical signals. From each cluster, one detection is picked as the "master". Only the master detections are used in the drifting algorithm; including the non-masters would skew the statistics. Previously, we flagged non-master detections lying in flagged triangles. With the vertex-only policy, we must do things differently: when we flag a vertex detection, we flag the other detections in its cluster. This required adding a data structure to keep track of clusters. <li>Previously, "clusters" meant detections within 1 second and 1 Hz. This wasn't appropriate for short or long FFT lengths. We changed it to use rectangles based on FFT bin sizes. </ul> With these changes, the algorithm is working pretty well. It removes only 2.9% of birdie spikes and 11.8% of all spikes, it removes the obvious examples, and few of the top multiplets are RFI. So maybe we're done with drifting. <p> Other news: we're talking with astronomers at FAST about using Nebula as part of SETI sky survey there. Needless to say, that would be very exciting! </div> |
|
20)
Message boards :
News :
Nebula progress report
(Message 2074148)
Posted 24 Apr 2021 by David Anderson
Post: Check out recent data analysis progress: Reobservation and drifting RFI. |
©2026 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.