Data Dump (May 17 2016)

Message boards : Technical News : Data Dump (May 17 2016)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1441
Credit: 217,165
RAC: 0
United States
Message 1788204 - Posted: 17 May 2016, 23:00:11 UTC

I haven't written in a long long while, with good reason: As of December 2015 I moved entirely to working on Breakthrough Listen, and Jeff and Eric heroically picked up all the slack. Of course we are all one big SETI family here at Berkeley and the many projects overlap, so I'm still helping out on various SETI@home fronts. But keeping Breakthrough moving forward has been occupying most of my time, and thus I'm not doing any of the day-to-day stuff that was fodder for many past tech news items.

That said, I thought it would be fun to chime in again on some random subset of things. I guess I could look and see what Eric already reported on over the past few months, but I won't so I apologize if there's any redundancy.

First, thanks to gaining access on a free computing cluster (off campus) and a simultaneous influx of free time from Dave and Eric we are making some huge advances in reducing the science database. All the database performance problems I've been moaning about for years are finally getting solved, or worked around, basically. This is really good news.

Second, obviously we are also finally splitting Green Bank data for SETI@home. While Jeff and Eric are managing all that, it's up to me to pass data from the Breakthrough Listen pipeline at Green Bank to our servers here at Berkeley. This is no small feat. We're collected 250TB of data a day with Breakthrough Listen - and maybe eventually recording as much as 1000TB (or 1PB a day). When we aren't collecting data we need every cycle we have to reduce the data to some manageable size. It's still in my court to figure out how to get some of this unreduced data to Berkeley. Shipping disks is not possible, or at least as easy as it is at Arecibo (because we aren't recording to single, shippable disks, but to giant arrays that aren't going anywhere. We may be able to do the data transfers over the net, and in theory have 10GB links between Berkeley and Green Bank, but in practice there'sa 1GB chokepoint somewhere. We're still figuring that out, but we have lots of data queued up so no crisis... yet.

Third, our new (not so new anymore) server centurion has been a life-saver, taking over as both the splitter for Green Bank data (turns out we needed a hefty server like this - the older servers would have fallen behind demand quickly) as well as our web server muarae1 when that system went bonkers around the start of the year. Well, we finally got a new muarae1 so centurion is back to being centurion - a dedicated splitter, a storage server, and potentially a science database clone and analysis machine. We also got a muarae2 server which is a back up (and eventual replacement) for the seti.berkeley.edu web server.

Fourth, our storage server bucket is having fits. All was well for a while but this is an old clunker of a machine so it's no surprise its internal disk controllers are misbehaving (we've seen similar behavior on similar oldSun servers). No real news here, as it doesn't have any obvious effect on public data services, but it means that Jeff and I have to wake up early and meet at the data center to deal with it tomorrow.

And on that note.. there's lots more of course but I should get back to it...
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1788204 · Report as offensive    Reply Quote
Profile Mr. KevvyCrowdfunding Project Donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 1536
Credit: 308,700,306
RAC: 556,552
Canada
Message 1788250 - Posted: 18 May 2016, 1:55:21 UTC

Dunno if you will see this, but may as well ask: with this huge volume of data, how much of it are we volunteers currently able to process? Are we keeping up or falling behind to what fraction?

And, sorry to sound like a broken record, but please let GPUUG know of any hardware needs! Haven't been tapped for a good fundraiser in a while, even though those we did were quite successful.
“Never doubt that a small group of thoughtful, committed citizens can change the world; indeed, it's the only thing that ever has.”
--- Margaret Mead

ID: 1788250 · Report as offensive    Reply Quote
Filipe

Send message
Joined: 12 Aug 00
Posts: 181
Credit: 8,414,603
RAC: 174
Portugal
Message 1788384 - Posted: 18 May 2016, 10:50:37 UTC

we are making some huge advances in reducing the science database


Can you explain more on this? And what are the implications for Nitpick?
ID: 1788384 · Report as offensive    Reply Quote
Chris Oliver
Avatar

Send message
Joined: 4 Jul 99
Posts: 56
Credit: 87,630,361
RAC: 34,372
United Kingdom
Message 1788775 - Posted: 19 May 2016, 19:25:53 UTC - in response to Message 1788204.  

Good to here from you - I was beginning to wonder if the aliens had kidnapped you !!!

Thanks again for the update.
ID: 1788775 · Report as offensive    Reply Quote
Profile Alex Striker

Send message
Joined: 11 Jan 04
Posts: 31
Credit: 19,157,615
RAC: 27,003
Denmark
Message 1790028 - Posted: 24 May 2016, 12:19:30 UTC

Many thanks for the long waited update Matt :-)
Happy Crunching

/Alex Striker, founder of:
Team Striker SETI/BOINC

English version webpage
us on Facebook
ID: 1790028 · Report as offensive    Reply Quote
Profile Michael Hoffmann
Volunteer tester

Send message
Joined: 4 Jun 08
Posts: 25
Credit: 2,897,190
RAC: 1,549
Germany
Message 1819329 - Posted: 24 Sep 2016, 13:00:00 UTC

Last update as of May 2016 - I mean: really? It's not that I'd expect a weekly update, but once every half year seems a little bit weak.
Om mani padme hum.
ID: 1819329 · Report as offensive    Reply Quote
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7471
Credit: 89,758,078
RAC: 13,189
Australia
Message 1819901 - Posted: 26 Sep 2016, 13:13:02 UTC - in response to Message 1819329.  
Last modified: 26 Sep 2016, 13:19:23 UTC

Last update as of May 2016 - I mean: really? It's not that I'd expect a weekly update, but once every half year seems a little bit weak.


More a case of legitimate logistics in my opinion. We're seeing the volunteer community spread pretty thinly as well. Just a matter of lots of announcements about dollars, but no apparent investment in what really matters (building the future).

[Edit:] Ivory backscratchers maybe ?
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1819901 · Report as offensive    Reply Quote
Profile edjcox
Avatar

Send message
Joined: 20 May 99
Posts: 91
Credit: 5,330,283
RAC: 5,844
United States
Message 1831691 - Posted: 21 Nov 2016, 5:09:11 UTC

I guess this whole thing boils down to the guys at SETI ignoring the public that is crunching their data for them. I expect more akin to what I get at Einstein at home. Regular updates and Kudo's for accomplishments shared with the crunching public.

So now I see an request for donors. Let me understand that the NSF contributes nothing to SETI at Home? Is there a misunderstanding at the NSF as to where SETI money goes?

Anyone have any insight into this?

Someone should clean up this site. Archive old stuff and get rid of it unless one goes into the archives. Just not cleanly run.
Never engage stupid people at their level, they then have the home court advantage.....
ID: 1831691 · Report as offensive    Reply Quote
oromis95

Send message
Joined: 2 Feb 16
Posts: 1
Credit: 4,410
RAC: 0
Italy
Message 1831915 - Posted: 22 Nov 2016, 15:50:32 UTC

No offense, but I've been kind of following you guys for a long time now. My computer is working for you guys while I study every day at the public library, with a GTX 970 and an i7 6700HQ, but you guys really don't give us any feedback on the results, just: "here is our infrastructure", or "here is more data to process". The thing is, you never tell us how well we are doing (or how badly), and you never tell us if anything interesting pops up or not. I feel like this is a waste of time. I think I'll be moving elsewhere, just like most other people have done.
ID: 1831915 · Report as offensive    Reply Quote
Profile Wiggo "Socialist"
Avatar

Send message
Joined: 24 Jan 00
Posts: 12133
Credit: 157,052,775
RAC: 118,684
Australia
Message 1831919 - Posted: 22 Nov 2016, 22:46:58 UTC

I guess this whole thing boils down to the guys at SETI ignoring the public that is crunching their data for them. I expect more akin to what I get at Einstein at home. Regular updates and Kudo's for accomplishments shared with the crunching public.

So now I see an request for donors. Let me understand that the NSF contributes nothing to SETI at Home? Is there a misunderstanding at the NSF as to where SETI money goes?

Anyone have any insight into this?

Someone should clean up this site. Archive old stuff and get rid of it unless one goes into the archives. Just not cleanly run.


No offense, but I've been kind of following you guys for a long time now. My computer is working for you guys while I study every day at the public library, with a GTX 970 and an i7 6700HQ, but you guys really don't give us any feedback on the results, just: "here is our infrastructure", or "here is more data to process". The thing is, you never tell us how well we are doing (or how badly), and you never tell us if anything interesting pops up or not. I feel like this is a waste of time. I think I'll be moving elsewhere, just like most other people have done.


Have either of you read this? Nebula: Completing the SETI@home pipeline

Cheers.
ID: 1831919 · Report as offensive    Reply Quote
Profile edjcox
Avatar

Send message
Joined: 20 May 99
Posts: 91
Credit: 5,330,283
RAC: 5,844
United States
Message 1835355 - Posted: 11 Dec 2016, 10:56:27 UTC

So here we are December. Last Tech update May.

Having worked in R&D all my life I'll say this. You guys would not work for me as you do not keep most of those crunching your data informed as to what is happening. You ask for more funding from us but don't really tell us where your funds are going and how they are utilized.

If you want to solicit funds then make those contributing excited about the project by telling us where your heading and what the project has accomplished. This should occur every two weeks or so. I have had every PC of mine working this since 1999. Somehow I don't feel the work accomplished anything. You certainly havn't improved the effort much. It seems so random and hit an miss. You have a orb of space to look at yet there seems to be no attack plan just scan whatever comes about at the GS site and hope maybe a strip of sky might have the remnants of an intelligible signal in it originating from out there.. No way of checking the sky again until another random strip coincidentally crosses the same area.

Well. Its in your court. Get it together and talk to the other half of your team. Us...
Never engage stupid people at their level, they then have the home court advantage.....
ID: 1835355 · Report as offensive    Reply Quote
mrncugo
Avatar

Send message
Joined: 23 Dec 16
Posts: 5
Credit: 5,146
RAC: 67
Russia
Message 1842179 - Posted: 14 Jan 2017, 16:02:50 UTC - in response to Message 1835355.  

Launch...
Question about the client run Seti@Home the first time I can`t run, anyone encountered such a problem that occurs.
Before starting the client noise no connection with server os (for beginners).
How long it takes to run the required points to be credited?
ID: 1842179 · Report as offensive    Reply Quote
wulf 21

Send message
Joined: 18 Apr 09
Posts: 83
Credit: 19,667,749
RAC: 7,382
Germany
Message 1842274 - Posted: 14 Jan 2017, 22:19:31 UTC - in response to Message 1842179.  
Last modified: 14 Jan 2017, 22:21:59 UTC

Well, such discussion is best placed in the number crunching forum. This forum is for news from project technicians and discussion about that.

But...

Before starting the client noise no connection with server os (for beginners).

This is normal at first startup or after reboot and should disappear after some time. Technically the BOINC engine (boinc.exe in task manager) and the graphical surface called BOINC Manager (boincmgr.exe) are two different programs, so users could actually run boinc on one computer and manage the program from a manager on an entirely different computer.

The message appears when the BOINC Mangaer is fully started up before the BOINC engine is started up and ready. If the message does keep appearing for longer than some minutes, something may be wrong with the installation.


How long it takes to run the required points to be credited?


Take a look at your task page (click on project website --> my tasks in the boinc manager), and click a little though it, so the internal workings may become a little more clear.

But basically: Your computer needs to finish a task first (status of task = in progress). Depending on which task it may take some minutes (if processed on GPU) to some hours (processed on CPU). After that, it will be uploaded to boinc and reported. Someone else needs to finish the same task (status = validation pending). If you were slower and the other one already was ready, validation will be done immediately, otherwise you need to wait. This can be quite a long time (even several weeks are possible). Worst case would be that the other one never finishes, then you will need to wait until the other task times out and another machine processes it.

If there are two independent results for the same task, validation will be done. If it is successful (2x same result), you will receive your credit. If results didn't match, the task will be sent to yet another machine that decides which one of the original 2 machines was correct.

So, long story short: The time to recieve credit for a specific task of your machine can be anything from some minutes to some months. But if you process enough tasks every day chances will be high that you may receive the credit for at least some of them the same day.
ID: 1842274 · Report as offensive    Reply Quote
mrncugo
Avatar

Send message
Joined: 23 Dec 16
Posts: 5
Credit: 5,146
RAC: 67
Russia
Message 1842493 - Posted: 15 Jan 2017, 22:10:23 UTC - in response to Message 1842274.  

Sorry, but it probably comes to program setiathome_win_3_08 it is most likely an individual client (its no boinc), its client seti.
Your do not know what can be a problem when connecting to a server?
Can this only applies to beginners?
ID: 1842493 · Report as offensive    Reply Quote
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,115,101
RAC: 799
United Kingdom
Message 1842581 - Posted: 16 Jan 2017, 11:57:39 UTC - in response to Message 1842493.  
Last modified: 16 Jan 2017, 12:09:26 UTC

Sorry, but it probably comes to program setiathome_win_3_08 it is most likely an individual client (its no boinc), its client seti.
Your do not know what can be a problem when connecting to a server?
Can this only applies to beginners?

That is a Seti Classic program, Seti clients are only run though Boinc now, and have been since Dec 2005:

http://seticlassic.ssl.berkeley.edu

On December 15, 2005, after 6 years of operation, Classic SETI@home sent out its last workunit. On December 22, we stopped receiving results. The current SETI@home project is up and running. Please join us! The new software uses the BOINC distributed computing platform, which enables your computer to work on other scientific endeavors like SETI@home if you wish. We could all use your CPU cycles! It's easy to switch over - just follow the instructions on the new SETI@home web site. We hope to see everyone there!
After the last result is received, the stats for this project will be frozen but will continue to be available on the web. It may take several weeks to clean everything up and syncronize the classic stats with the current SETI@home project.
Thanks to all original SETI@home participants for their tremendous dedication to the project. You made SETI@home into something of lasting significance.

Download a suitable Boinc for your computer, and add/attach it to Setiathome, and when Boinc contacts the project it will send the correct Seti apps/clients for your computer:

https://setiathome.berkeley.edu/join.php

Claggy
ID: 1842581 · Report as offensive    Reply Quote
mrncugo
Avatar

Send message
Joined: 23 Dec 16
Posts: 5
Credit: 5,146
RAC: 67
Russia
Message 1842838 - Posted: 18 Jan 2017, 8:30:09 UTC - in response to Message 1842581.  

Quote:
''Classic SETI@home sent out its last workunit. On December 22, we stopped receiving results.''
That is, a program that works separately from Boinc '' seti athome_win_3_08 '' - it is now frozen???
ID: 1842838 · Report as offensive    Reply Quote
Profile Wiggo "Socialist"
Avatar

Send message
Joined: 24 Jan 00
Posts: 12133
Credit: 157,052,775
RAC: 118,684
Australia
Message 1842844 - Posted: 18 Jan 2017, 9:30:02 UTC - in response to Message 1842838.  

Quote:
''Classic SETI@home sent out its last workunit. On December 22, we stopped receiving results.''
That is, a program that works separately from Boinc '' seti athome_win_3_08 '' - it is now frozen???

Obsolete. ;-)

Cheers.
ID: 1842844 · Report as offensive    Reply Quote
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 8476
Credit: 106,456,742
RAC: 71,338
Australia
Message 1842847 - Posted: 18 Jan 2017, 9:50:15 UTC - in response to Message 1842838.  

Quote:
''Classic SETI@home sent out its last workunit. On December 22, we stopped receiving results.''
That is, a program that works separately from Boinc '' seti athome_win_3_08 '' - it is now frozen???

It was replaced by this project, Seti@home running under BOINC.
Grant
Darwin NT
ID: 1842847 · Report as offensive    Reply Quote
mrncugo
Avatar

Send message
Joined: 23 Dec 16
Posts: 5
Credit: 5,146
RAC: 67
Russia
Message 1843014 - Posted: 19 Jan 2017, 9:47:23 UTC - in response to Message 1842847.  

Now it is clear that this version is '' seti athome_win_3_08 '' is now outdated!
Thanks for the info-roads!
ID: 1843014 · Report as offensive    Reply Quote
Profile Bill

Send message
Joined: 18 May 99
Posts: 21
Credit: 4,639,977
RAC: 1,247
United States
Message 1861152 - Posted: 13 Apr 2017, 3:12:11 UTC

Well we are about 40 days from a one year anniversary of the Last Technical News update.

just inquiring is there any one except remote volunteers still working this project ?

The whole world has volunteers but we have no one with a degree in Berkley that can type an update. Sorry but that is a measured fact.

Thanks
Bill F.
SETI since May 18th 1999
ID: 1861152 · Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Technical News : Data Dump (May 17 2016)


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.