Nebula: Completing the SETI@home pipeline

Message boards : News : Nebula: Completing the SETI@home pipeline
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Profile David Anderson
Volunteer moderator
Project administrator
Project developer
Avatar

Send message
Joined: 13 Feb 99
Posts: 99
Credit: 390,229
RAC: 0
Message 1829052 - Posted: 7 Nov 2016, 21:48:28 UTC

Read about Nebula, a new and faster back end for SETI@home. Nebula removes RFI and finds persistent signals. Its goal is to let us finish the current SETI@home experiment.
ID: 1829052 · Report as offensive
BoincSpy
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 107
Credit: 26,335,742
RAC: 36,752
Canada
Message 1829124 - Posted: 8 Nov 2016, 5:15:23 UTC - in response to Message 1829052.  

Hi All,

I am a bit confused by a couple of things after reading the Nebula article:

1) Does seti@home client computers feed the Nebula back end? There are hints that it will but does not explicitly call it out that as the section of Flattening the table hierarchy there is a step on unloading the tape so would like a clarification on why use the tape again as the data on the tape has already been sent to seti@home clients.

2) If David is not working on the seti@home client because of funding issues. Who will continue to work the seti@home changes that were outlined in the article. Volunteers?

Thanks for listening.

BoincSpy still alive.
ID: 1829124 · Report as offensive
Profile mr.mac52Project Donor
Avatar

Send message
Joined: 18 Mar 03
Posts: 52
Credit: 195,274,817
RAC: 102,755
United States
Message 1829128 - Posted: 8 Nov 2016, 5:32:16 UTC - in response to Message 1829052.  

I will admit that I did not read all the materials pointed to in the various places.

That said, I find myself troubled by the statement: "Its goal is to let us finish the current SETI@home experiment".

This sounds like the end of SETI@home.

I'm a bit tired and low on my glucose levels and I hope I'm not jumping to conclusions so please tell me that I'm off in the weeds.

John
ID: 1829128 · Report as offensive
Profile JakeTheDog
Avatar

Send message
Joined: 3 Nov 13
Posts: 138
Credit: 2,302,099
RAC: 280
United States
Message 1829133 - Posted: 8 Nov 2016, 6:42:37 UTC

"Publishing this paper won't mark the end of SETI@home. It may continue indefinitely. However, it will probably undergo a major transformation soon - new telescopes, new frequency bands, new front-end algorithms - that will effectively start a new experiment. We need to finish the original experiment."

Well, no matter what happens to SETI@home, I'm glad all the results will now undergo some kind of comprehensive analysis.
ID: 1829133 · Report as offensive
Profile [STS]LoB
Volunteer tester

Send message
Joined: 23 May 99
Posts: 42
Credit: 2,765,070
RAC: 1,969
Germany
Message 1829134 - Posted: 8 Nov 2016, 6:46:07 UTC - in response to Message 1829128.  

I will admit that I did not read all the materials pointed to in the various places.

That said, I find myself troubled by the statement: "Its goal is to let us finish the current SETI@home experiment".

This sounds like the end of SETI@home.

I'm a bit tired and low on my glucose levels and I hope I'm not jumping to conclusions so please tell me that I'm off in the weeds.

John


Publishing this paper won't mark the end of SETI@home. It may continue indefinitely. However, it will probably undergo a major transformation soon - new telescopes, new frequency bands, new front-end algorithms - that will effectively start a new experiment. We need to finish the original experiment.


http://setiathome.berkeley.edu/nebula_web/goals.php
ID: 1829134 · Report as offensive
Profile ZalsterProject Donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 3990
Credit: 208,939,431
RAC: 54,812
United States
Message 1829139 - Posted: 8 Nov 2016, 7:01:21 UTC - in response to Message 1829128.  
Last modified: 8 Nov 2016, 7:01:53 UTC

I will admit that I did not read all the materials pointed to in the various places.

That said, I find myself troubled by the statement: "Its goal is to let us finish the current SETI@home experiment".

This sounds like the end of SETI@home.

I'm a bit tired and low on my glucose levels and I hope I'm not jumping to conclusions so please tell me that I'm off in the weeds.

John


Unfortunately, I think you read it right John.

But as stated, all experiments have to end at some point and results published.

But I think this opens a way for a new Seti@home (with funding probably from Breakthrough Listen for paid employees) to take over the daily requirements of keeping it going. It would allow the current scientists to move onto newer projects and bring in younger ones to continue their work (ie cut their teeth on a stable platform)

What will the future look like, who can tell.

Zalster
ID: 1829139 · Report as offensive
Profile tullioProject Donor
Volunteer moderator
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 6296
Credit: 1,676,778
RAC: 1,339
Italy
Message 1829153 - Posted: 8 Nov 2016, 8:06:08 UTC
Last modified: 8 Nov 2016, 8:07:08 UTC

As an old Einstein@home aficionado, I am glad of this cooperation between SETI@home and the Albert Einstein Institut of the Max Planck Gesellshaft. I have seen some of my Einstein results validated against results from Atlas members.
Tullio
ID: 1829153 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 40036
Credit: 34,769,143
RAC: 63,511
United Kingdom
Message 1829155 - Posted: 8 Nov 2016, 8:12:55 UTC
Last modified: 8 Nov 2016, 8:14:37 UTC

Firstly I think that is an excellent report by Dr Anderson. Although technical it is written in a way that someone like myself, a non scientist, can follow and understand the issues involved. But certain points stood out to me.

I've been working on a project called Nebula. Its goal is to finish the current SETI@home experiment: to complete the data analysis pipeline, and to write a paper describing the experiment and saying either that we found ET or that we were unable to find an ET signal in the data.

So yes this looks like the end of the current Seti@home project as we know it.

The underlying problem was that the SETI@home project has never had adequate funding, and the people who could have completed and improved the back end (like Jeff and Eric) were often involved in "fire-fighting" - dealing with hardware and sysadmin problems to keep the front end working.

That in a nutshell has been the perennial problem, scientists working as computer technicians.

So in 2007 Jeff Cobb began working on a new approach called the Near-Time Persistency Checker, or NTPCkr. The idea was to keep track of the set of pixels for which new signals had arrived recently, and to do RFI removal and scoring for these pixels. Eventually we realized that even this was too slow to keep up with the arrival of signals, and it was shelved.

Well a definitive statement at last. We were initially told that a nitpicker machine would be $30,000 but there was no funding.

So for a number of years - 2003 to 2015 - SETI@home has continued to accumulate signals, with no viable strategy for back-end processing.

A situation I have pointed out a number of times and which has led to many people leaving for other Boinc projects.

I proposed working for SETI@home with the goal of making the back end functional. It was clear that we needed to moving things off our own computers, onto a scalable platform such as a commercial "cloud". So I called this new project Nebula.

This is where the Einstein Atlas comes in to do what nitpicker was supposed to?

A science experiment must have a conclusion, positive or negative. After 18 years, SETI@home does not yet have one. My goal in developing Nebula is to finish what we started in 1998 - to reach a point where we can write a scientific "results" paper

I have to agree. No point in continuing to store results and not doing anything with them.

In addition to providing a conclusion to SETI@home, this paper would expose the SETI@home methods and algorithms to the world, so that people outside our group (other SETI researchers, signal-processing experts, etc.) can critique them and suggest improvements. Perhaps our data contains an ET signal but our current algorithms can't find it.

I think "share" would have been better than "expose", but this could have been done 5 years ago couldn't it?

Publishing this paper won't mark the end of SETI@home. It may continue indefinitely. However, it will probably undergo a major transformation soon - new telescopes, new frequency bands, new front-end algorithms - that will effectively start a new experiment. We need to finish the original experiment.

From this I assume that us the crunchers are still needed to receive data, process, and send back, but all under a different umbrella to that at present.

My conclusions are that Dr Anderson is basically correct in his overview of the current Seti@home project. Time to call a halt. Review and move on. But persistent hardware and software problems continue at the CoLo with the Seti kit which cannot be ignored.

I am not disheartened at all, nor should you be.
ID: 1829155 · Report as offensive
Profile JimbocousProject Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1001
Credit: 90,758,926
RAC: 87,471
United States
Message 1829169 - Posted: 8 Nov 2016, 9:31:31 UTC - in response to Message 1829155.  
Last modified: 8 Nov 2016, 9:32:47 UTC

I am not disheartened at all, nor should you be.

As long as there's data I can crunch, I will keep heart.
At the point where downloads cease, I'd be po'd and trying to decide what to do with 5 crunchers containing 15 $$$ GPUs that I'll have no further use for ...
I didn't read a vision for how S@H folks and their capabilities fit into the future scheme, in this dissertation, and that's disappointing.
ID: 1829169 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 40036
Credit: 34,769,143
RAC: 63,511
United Kingdom
Message 1829178 - Posted: 8 Nov 2016, 10:46:33 UTC - in response to Message 1829169.  
Last modified: 8 Nov 2016, 10:54:44 UTC

I didn't read a vision for how S@H folks and their capabilities fit into the future scheme, in this dissertation, and that's disappointing.

I take your point, but I did say

Publishing this paper won't mark the end of SETI@home. It may continue indefinitely. However, it will probably undergo a major transformation soon - new telescopes, new frequency bands, new front-end algorithms - that will effectively start a new experiment. We need to finish the original experiment.

From this I assume that us the crunchers are still needed to receive data, process, and send back, but all under a different umbrella to that at present.

No succinct reply yet to that, but I think the point is that Seti proved that, distributed computing involving the general public was a viable proposition compared to expensive super computer time. So much so, that BOINC was devised as an umbrella for currently over 50 scientific projects. You can help provide clean water, search for cancer cures, gravity waves, mathematical matters, clues to the origins of the universe, all sorts of worthwhile things.

We now have GBT data and Guppi work in addition to Arecibo. Just because Seti as we know it might transition into something blander or not as attractive, is no reason to not use 5 top rate crunchers for the benefit of mankind. I know my farm will continue somewhere or other, most likely at Seti MkII :-))
ID: 1829178 · Report as offensive
Cameron
Avatar

Send message
Joined: 27 Nov 02
Posts: 90
Credit: 2,323,187
RAC: 3,478
Australia
Message 1829193 - Posted: 8 Nov 2016, 13:50:03 UTC

My inital interpretation is that the current SETI Experiment is involving the Aricebo dataset to draw the conclusion from.

Although I would like to hear about how SETI@Home will transition.

Will we crunch Breakthough with V8 while they code S@H3?
Will They Shutdown for a few months to transistion the software?

I'd expect to hear more details next year.
ID: 1829193 · Report as offensive
[VENETO] boboviz
Volunteer tester

Send message
Joined: 5 Oct 99
Posts: 8
Credit: 530,048
RAC: 95
Italy
Message 1829280 - Posted: 9 Nov 2016, 9:04:57 UTC

Publishing this paper won't mark the end of SETI@home. It may continue indefinitely. However, it will probably undergo a major transformation soon - new telescopes, new frequency bands, new front-end algorithms - that will effectively start a new experiment. We need to finish the original experiment.


And we are ready to crunch on this new!! :-)
ID: 1829280 · Report as offensive
Profile dschwartingProject Donor
Avatar

Send message
Joined: 7 Sep 99
Posts: 94
Credit: 40,158,163
RAC: 19,234
Germany
Message 1829370 - Posted: 9 Nov 2016, 17:27:35 UTC

 
It was very exciting for me to read the Nebula article from David P. Anderson. Thanks for this great report!

So far as I understood it's time to bring up the back-end processing (which was never done by the seti@home clients) to a new level (faster processing speed, higher accuracy, extended possibilities to detect ET-Signals and more). The SETI@home front-end processing done by our seti@home clients will may be changed in future but it will continue to be necessary (please reply if I'm wrong).

I'm very glad about this upcoming step (evolution) because stagnation is regression.

I am grateful to participate in this exciting project.
 
SETIspirit Homepage
ID: 1829370 · Report as offensive
Nighthawk[TECH]

Send message
Joined: 7 May 01
Posts: 5
Credit: 2,078,360
RAC: 0
Canada
Message 1829455 - Posted: 9 Nov 2016, 22:39:31 UTC

I had read approximately one-third of the way through the article when I came across the assertion that GPS satellites are geostationary (in the section describing RFI sources). This is incorrect. Communication satellites are (in general) geostationary, while GPS satellites have an orbital period of ~12hrs.

With regard to those dismayed crunchers who are concerned about an impending end to the project, I believe David Anderson addressed this by stating in the article that this is the end of the original experiment, but that further (and expanded) data acquisition and analysis is almost a certainty.
ID: 1829455 · Report as offensive
Profile zubr2009Project Donor
Avatar

Send message
Joined: 3 Mar 13
Posts: 24
Credit: 147,854
RAC: 0
Belarus
Message 1830053 - Posted: 12 Nov 2016, 12:39:13 UTC

That's right, David!
It is time to end the wandering in the "mist" and draw conclusions, we find them or not!
But in any case they have and perhaps much closer than we think and where looking.
If we come to them with the phrase "hello humanoids!", I am sure one day they declared to us with the idea, "Well that was not expected?" )))
Горы, нет ничего лучше гор. :)
Mountains, there is nothing better than mountains. :)
Life is good when you live without haste! ))
ID: 1830053 · Report as offensive
Profile ML1
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 9370
Credit: 7,025,239
RAC: 5,145
United Kingdom
Message 1830054 - Posted: 12 Nov 2016, 12:42:47 UTC
Last modified: 12 Nov 2016, 12:45:08 UTC

WOW! Real Science in action, and a very good move and fantastic new development.


Shame that developments like this are pushed after the all too usual crisis of no funding for science and where the science is left to external random goodwill and the randomness of having to go begging.

Great shame for the circumstances.

Excellent very logical and opportune development to make s@h a big success.


Oh, and Trump owes Germany a BIG favour for keeping USA cutting edge astronomy exploration alive.

Congratulations all round :-)

Keep searchin!!!!
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 1830054 · Report as offensive
frogola

Send message
Joined: 15 Oct 15
Posts: 1
Credit: 1,903,589
RAC: 2,522
United States
Message 1830117 - Posted: 12 Nov 2016, 19:22:29 UTC - in response to Message 1829052.  

Thanks for the readable, detailed and thorough article on your well thought-out plan for updating and hopefully completing the SETI@home project David. Keep up the good work.
ID: 1830117 · Report as offensive
Profile Crunch3rProject Donor
Volunteer tester
Avatar

Send message
Joined: 15 Apr 99
Posts: 1546
Credit: 3,438,544
RAC: 0
Germany
Message 1830131 - Posted: 12 Nov 2016, 20:46:28 UTC - in response to Message 1830117.  
Last modified: 12 Nov 2016, 20:48:34 UTC

OK, i've read the whole "nebula" link and to me it seems that nebula will finally conclude and finnish S@H. Which is past due....

Given the fact, that Seti@Home never analysed the data send back from it's clients, it's about time we arrive at a conclusion.

Special thanks goes out to Bruce Allen, who i've personally had the honor to meet at an einstein@home event back in 2006.

https://www.flickr.com/photos/20418602@N00/sets/72157594427277300/detail/

He's a great guy and without a doubt, one of the most interactive boinc project admins i had the pleasure to meet in person or work with/talk to.

Join BOINC United now!
ID: 1830131 · Report as offensive
kittyman
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 49050
Credit: 877,947,989
RAC: 199,879
United States
Message 1830195 - Posted: 13 Nov 2016, 3:18:58 UTC
Last modified: 13 Nov 2016, 3:32:48 UTC

As one who has spent a great deal of time and money on the current version of Seti, I await the results of the 'final processing' of the data we have all worked so long and hard to accumulate.

Furthermore, the kitties are optimistic and excited about the next phase of Seti research.

And you should know this..........
I feel I should make this statement as one who has done so much toward an apparently dead end.

I personally would like to thank Eric, Jeff, Matt and Dr. Anderson for all of THEIR time and effort spent in support of what we crunchers have been able to do. Without their endless efforts for so many years, this would not have been possible.

I am not nor shall be disappointed if our years of current work turns up nothing.
That is part of the scientific process. And truth be told, most of us expected to find nothing, even though we believe there IS life beyond our solar system.

Rest assured, if possible, the kitties shall be involved in the next generation of this most important scientific project. And I hope that my fellow Setizens, and the generations of Setizens to follow in our footsteps shall continue this search when I have left this planet in earthly form to continue the search as a soul unfettered by the boundaries that bind a living human being. I shall then be able to travel personally to other worlds and see for myself what might be there. And that to me, is a most exciting possibility.

Meow!
A kitty keeps loneliness away.
More meowing, less hissing. I speak meow, do you?

Have made friends in this life.
Most were cats.
ID: 1830195 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 40036
Credit: 34,769,143
RAC: 63,511
United Kingdom
Message 1830283 - Posted: 13 Nov 2016, 12:11:46 UTC
Last modified: 13 Nov 2016, 12:18:10 UTC

It is a shame is it not to see negativity? We all know that it is a very small needle in a very big haystack, but if you don't look then you will never find.
ID: 1830283 · Report as offensive
1 · 2 · 3 · Next

Message boards : News : Nebula: Completing the SETI@home pipeline


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.