June Update (Jun 02 2010)

Message boards : Technical News : June Update (Jun 02 2010)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1000083 - Posted: 2 Jun 2010, 19:31:08 UTC

Another monthly-ish report.

First the good news, before it seems like I only want to blather about the funny/annoying stuff. Jeff has been hammering on the NTPCkr to incorporate the newer RFI removal code. Before the plumbing was of the form: signals come in, pixels are scored, the best ones are displayed for the public to see. Now the plumbing is: signals come in, pixels are scored, the best ones are displayed in a sort of "preview" form and sent into the RFI loop, which then forces the pixels to be rescored (after bad signals are removed), and if they still happen to be in the top ten they'll have all the associated plots for the public to analyze.

I'm also still pecking away at data integrity tests. I have the "birdie injector" (which sticks fake signals in the raw data) working to some extent. After some full tests we're finding these birdies in the results reported back from the clients - though it seems that we might have to add another retroactive signal correction in the future. Don't worry - if this in fact true it's not a big deal. It's easy to fix and there's no lost scientific integrity. Other than that, there's continuing testing happening in my copious free time. I also wrote up a scientific newsletter about radar blanking.

Of course, our server woes have peaked a bit recently, coinciding nicely with the holiday weekend and a mass e-mail. The mass mail was part of the problem actually - there was a link to several video files which were much larger than I assumed. Like hundreds of megabytes larger. So that made our web site (and the whole lab's internet connection) a little bit sluggish. Oopsie.

But the two machines prone to random crashes did just that. First mork went down taking the BOINC user database with it. Recovery was easy enough, but then the next day ptolemy went down taking everything with it. Dan actually came up on Sunday to power cycle the thing by hand (with my guidance via phone).

Yeah - it's on our list of over 200 critical things to get a remote power strip installed on ptolemy. I'd rather we just have systems that didn't crash. Unfortunately the functionalities of these machines are such that transferring them to other machines is impossible. However, we do have thumper... and marvin...

I've been hoping to reorganize thumper and upgrade its OS for some time now. We finally had the window to do that this past month, but there was one hangup after another postponing this project. Meanwhile we have marvin set up for test database purposes. It's more or less a functional equivalent of thumper, but with a lot less drive spindles. Still, the plan now is to burn marvin in, move the science database there (temporarily if not permanently), and then thumper is free to be completely wiped clean. Maybe we'll make thumper the new ptolemy and retire old ptolemy. That'll be one less crashy server to deal with. As for replacing mork... well we need another system with a bunch of CPUs, many disk spindles, and at least 64GB of memory. Not happening any time soon.

Let's see... other projects... oh yeah - we're now merging the spike tables. We had to split the spike signal a while back due to reaching some logical constraint in the database. After the dust settled on various other projects we're ready to make that one whole spike table again. Easier said than done, but what isn't around her? Anywho.. this was why the spike signal counts on the science status page seemed a little off for a while.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1000083 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1000107 - Posted: 2 Jun 2010, 20:41:22 UTC - in response to Message 1000083.  

Thanks for the update Matt, and for all the hard work keeping the Servers running,

Claggy
ID: 1000107 · Report as offensive
Luke
Volunteer developer
Avatar

Send message
Joined: 31 Dec 06
Posts: 2546
Credit: 817,560
RAC: 0
New Zealand
Message 1000126 - Posted: 2 Jun 2010, 21:09:48 UTC

Thank you Matt for the update, any is appreciated. All the servers are apparently running normally, we can upload, download and report tasks. All is fine and happy.

On a completely unrelated note... It was mentioned on the old list of things SETI@home would desire to implement is a sky search in the southern hemisphere, possibly using te parks observatory in Australia? Was this idea shelved? Or is it still in the back of your minds if Arecibo runs out of funding?
- Luke.
ID: 1000126 · Report as offensive
Profile Byron Leigh Hatch @ team Carl Sagan
Volunteer tester
Avatar

Send message
Joined: 5 Jul 99
Posts: 4548
Credit: 35,667,570
RAC: 4
Canada
Message 1000143 - Posted: 2 Jun 2010, 22:19:44 UTC

Thanks for the update Matt. You scientist At SETI@home are the best.
ID: 1000143 · Report as offensive
Profile Bill Walker
Avatar

Send message
Joined: 4 Sep 99
Posts: 3868
Credit: 2,697,267
RAC: 0
Canada
Message 1000146 - Posted: 2 Jun 2010, 22:38:03 UTC

Thank you Matt, and please tell Dan thanks too.

ID: 1000146 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31006
Credit: 53,134,872
RAC: 32
United States
Message 1000162 - Posted: 3 Jun 2010, 0:35:38 UTC - in response to Message 1000083.  

Thanks for the update. It is much appreciated.

Went by the NTPCKR page a couple days ago but there are no links to the waterfall plots available, in case Jeff didn't know that.

Keep up the good work.

ID: 1000162 · Report as offensive
Profile gizbar
Avatar

Send message
Joined: 7 Jan 01
Posts: 586
Credit: 21,087,774
RAC: 0
United Kingdom
Message 1000273 - Posted: 3 Jun 2010, 9:07:12 UTC

Thanks Matt, and Dan for dragging himself in on a Sunday!

Nice to get an update, and hope your productivity has increased since you're not posting here all the time...

But, who's Marvin? He's not on the server list!

regards, Gizbar.



A proud GPU User Server Donor!
ID: 1000273 · Report as offensive
Fred W
Volunteer tester

Send message
Joined: 13 Jun 99
Posts: 2524
Credit: 11,954,210
RAC: 0
United Kingdom
Message 1000289 - Posted: 3 Jun 2010, 10:18:38 UTC - in response to Message 1000273.  

But, who's Marvin? He's not on the server list!



He's hiding. That's what paranoia does... ;-P

F.
ID: 1000289 · Report as offensive
ront

Send message
Joined: 25 Aug 01
Posts: 77
Credit: 386,336
RAC: 0
United States
Message 1000312 - Posted: 3 Jun 2010, 13:19:41 UTC

May I add my thanks and gratitude for the update and the outstanding dedication of all of you to keep this project online.

Blessing and Peace

ront
ID: 1000312 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1000362 - Posted: 3 Jun 2010, 16:48:36 UTC

Answering some questions:

1. Not sure about other projects, but the SETI@home science database is roughly 1.25 terabytes and the Astropulse database is just under 0.5 terabytes. Both are of course still growing. There are billions of signals in there, which is why matching them together in various permutations is such a slow, arduous problem.

2. Southern hemisphere search is on indefinite hold. Not enough resources. That, or other telescopes, could be used if Arecibo goes away, but we'd prefer to keep it up at Arecibo as long as possible.

3. No waterfall plots on the NTPCkr page because these "top candidates" haven't gone through RFI excision yet, and when they do they'll be rescored. If they manage to still be in the top ten (highly unlikely) *then* waterfalls will be made (it's silly to keep making waterfalls full of RFI a computer can get rid of eventually on its own).

4. Marvin is a functional equivalent of thumper but with only 8 drives (instead of 48) - one of many Intel donations. We used it as a compute server, then set it up as a database test server, now maybe an actual database server. I only post the servers on the status page that are public facing (or process public data). We actually have about, let's see... over 20 servers around here (including desktops and backend compute servers).

5. To be clear, I personally am not a scientist, though I learn a lot via osmosis hanging around them that sometimes people think I know what I'm talking about. I more of a math/computer geek.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1000362 · Report as offensive
Profile Byron Leigh Hatch @ team Carl Sagan
Volunteer tester
Avatar

Send message
Joined: 5 Jul 99
Posts: 4548
Credit: 35,667,570
RAC: 4
Canada
Message 1000384 - Posted: 3 Jun 2010, 18:30:05 UTC

Thank you Matt. You scientist At SETI@home show outstanding dedication to keep this project on line. Best wishes and peace Byron.
ID: 1000384 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 1000390 - Posted: 3 Jun 2010, 19:04:54 UTC

I believe you are being too modest Matt. You have the "Project Scientist" title so you are one.

Thanks for everything you do to keep us updated.

Boinc....Boinc....Boinc....Boinc....
ID: 1000390 · Report as offensive
Profile SciManStev Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Jun 99
Posts: 6658
Credit: 121,090,076
RAC: 0
United States
Message 1000466 - Posted: 4 Jun 2010, 0:02:27 UTC

Thank you very much for the news. Knowing a little of what goes on gives us high motivation to continue being a part of this fantastic search. I see every one of you as a scientist and a problem solver. We will crunch on!

Steve
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website
ID: 1000466 · Report as offensive
Profile Norm Hill

Send message
Joined: 6 Sep 99
Posts: 9
Credit: 10,396,539
RAC: 0
United States
Message 1000467 - Posted: 4 Jun 2010, 0:03:19 UTC - in response to Message 1000362.  

Matt,
Half the reason I continue to contribute my spare CPU cycles to SETI is that I am truly interested in what is going on behind the scenes. I enjoyed hearing the daily goings on around the lab. You were a little frustrated in your posts sometimes but that goes with being a Type-A personality. Please reconsider going back to a daily or weekly posting. There a lot SETI fans who are actually interested in what is going on!
ID: 1000467 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31006
Credit: 53,134,872
RAC: 32
United States
Message 1000481 - Posted: 4 Jun 2010, 0:48:03 UTC - in response to Message 1000362.  

5. To be clear, I personally am not a scientist, though I learn a lot via osmosis hanging around them that sometimes people think I know what I'm talking about. I more of a math/computer geek.

- Matt

What makes a scientist? A fancy degree? Can name a long list that didn't have one. Make the majority of money at it? Can name another long list that didn't. Publishing papers? Haven't you done that?

Matt, you may not be an astronomer, but you are a scientist, perhaps computer scientist with a specialty of distributed processing. Anyone who can keep Seti going is a cutting edge wonder.


ID: 1000481 · Report as offensive
Profile Byron Leigh Hatch @ team Carl Sagan
Volunteer tester
Avatar

Send message
Joined: 5 Jul 99
Posts: 4548
Credit: 35,667,570
RAC: 4
Canada
Message 1000489 - Posted: 4 Jun 2010, 1:33:17 UTC - in response to Message 1000362.  

Answering some questions:

.....

5. To be clear, I personally am not a scientist, though I learn a lot via osmosis hanging around them that sometimes people think I know what I'm talking about. I more of a math/computer geek.

- Matt

I know that Matt, but still ... I consider you to be a scientist. Thank you Matt, David A, Dan, Jeff, Josh Von Korff, Robert Bankay, Kevin Douglas, and anyone I forgot ... You scientist At SETI@home show outstanding dedication to keep this project on line. Best wishes and peace Byron.

ID: 1000489 · Report as offensive
Profile Jon Golding
Avatar

Send message
Joined: 20 Apr 00
Posts: 105
Credit: 841,861
RAC: 0
United Kingdom
Message 1000593 - Posted: 4 Jun 2010, 11:43:43 UTC

Jeff has been hammering on the NTPCkr to incorporate the newer RFI removal code. Before the plumbing was of the form: signals come in, pixels are scored, the best ones are displayed for the public to see. Now the plumbing is: signals come in, pixels are scored, the best ones are displayed in a sort of "preview" form and sent into the RFI loop, which then forces the pixels to be rescored (after bad signals are removed), and if they still happen to be in the top ten they'll have all the associated plots for the public to analyze.


So, is the RFI removal script now running constantly in the background, or does this script only get implemented every day or so?
ID: 1000593 · Report as offensive
Luke
Volunteer developer
Avatar

Send message
Joined: 31 Dec 06
Posts: 2546
Credit: 817,560
RAC: 0
New Zealand
Message 1000601 - Posted: 4 Jun 2010, 12:13:50 UTC

Thanks for answering my question Matt. :)
- Luke.
ID: 1000601 · Report as offensive
Richard
Avatar

Send message
Joined: 10 Jul 99
Posts: 19
Credit: 17,341,684
RAC: 0
Argentina
Message 1000664 - Posted: 4 Jun 2010, 16:23:20 UTC - in response to Message 1000467.  

Matt,
Half the reason I continue to contribute my spare CPU cycles to SETI is that I am truly interested in what is going on behind the scenes. I enjoyed hearing the daily goings on around the lab. You were a little frustrated in your posts sometimes but that goes with being a Type-A personality. Please reconsider going back to a daily or weekly posting. There a lot SETI fans who are actually interested in what is going on!


I agree 100% with this Member. I think there is a lot of people who enjoyed the almost daily posts of Matt, not matter if there are good or bad news, and nowadays whitout this posts, feels a little "abandoned" (to say) by the project. I agree that the Science is the most important part of the project, but I believe that participate is not only crunching numbers but also knowing what is happening behind the curtains with our efforts.
I think that there outside is a lot of people like me who enjoyed to read each morning the "coloured" comments of Matt (he would be a writer) and feels a little frustrated by the lack of them.
ID: 1000664 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 36772
Credit: 261,360,520
RAC: 489
Australia
Message 1000672 - Posted: 4 Jun 2010, 17:44:33 UTC - in response to Message 1000664.  

Matt,
Half the reason I continue to contribute my spare CPU cycles to SETI is that I am truly interested in what is going on behind the scenes. I enjoyed hearing the daily goings on around the lab. You were a little frustrated in your posts sometimes but that goes with being a Type-A personality. Please reconsider going back to a daily or weekly posting. There a lot SETI fans who are actually interested in what is going on!


I agree 100% with this Member. I think there is a lot of people who enjoyed the almost daily posts of Matt, not matter if there are good or bad news, and nowadays whitout this posts, feels a little "abandoned" (to say) by the project. I agree that the Science is the most important part of the project, but I believe that participate is not only crunching numbers but also knowing what is happening behind the curtains with our efforts.
I think that there outside is a lot of people like me who enjoyed to read each morning the "coloured" comments of Matt (he would be a writer) and feels a little frustrated by the lack of them.

I'm in the same boat as these members plus can it be possible to setup a fault/error reporting message board?

BTW you guys do do great work.
ID: 1000672 · Report as offensive
1 · 2 · Next

Message boards : Technical News : June Update (Jun 02 2010)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.