Swimming in Data (Jun 20 2012)

Message boards : Technical News : Swimming in Data (Jun 20 2012)

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1441
Credit: 213,689
RAC: 0
United States
Message 1248935 - Posted: 20 Jun 2012, 21:53:23 UTC

Some news. Yesterday we had our usual weekly outage, and shortly after the floodgates opened again bruno (the upload server) crashed. Except we quickly found it didn't actually crash. It was turned off. By the web-enabled power strip. For no apparent reason. We turned it back on and everything was okay, but now it seems like we have a flaky web-enabled power strip on our hands. It is interesting to note that this power strip was plugged into the same breaker as thinman - the previous webserver system that died during that last unexpected power issues. So maybe some funky voltage clobbered this strip as well. Well, we have a spare one which works so no big shakes there. And yes, we ruled out foul play.

As for the crashy desktop machines, I may have fixed one. The theory being, oddly enough, too much thermal grease was employed thus reducing the effectiveness of the heat sink. Oops. Well, I'm not quite convinced that was the problem, and we're burning it in now. If it survives a week without crashing, great. The other system is not doing as well. I think we're aiming to get insurance money from the university to cover the cost of these systems killed or injured during these outages. Meanwhile, we're operational, so no real disaster.

In better news, georgem is now not only hosting all the workunits and running some backend BOINC services and scientific analysis processes, but it's also hosting all the data (~13TB) from a recent survey of the galactic center collected at Green Bank Telescope. Several grad students will be processing this data on georgem itself.

Also paddym has been cleared to finally reformat all its drives into a giant RAID10, and we can now start the process of duplicating the whole SETI@home informix science database on oscar over there. As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

Oh yeah we also found a bug that had been causing a lot of Astropulse splitters to fail, thus reducing the amount of AP workunits being sent out. This has been fixed, and so expect more AP work.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1248935 · Report as offensive
ClaggyProject Donor
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4623
Credit: 46,352,417
RAC: 3,002
United Kingdom
Message 1248943 - Posted: 20 Jun 2012, 22:01:12 UTC - in response to Message 1248935.  

Thanks for the update Matt,

Claggy
ID: 1248943 · Report as offensive
JohnDKCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 944
Credit: 87,997,413
RAC: 47,117
Denmark
Message 1248959 - Posted: 20 Jun 2012, 22:17:15 UTC

Yes thanks, no least the Astropulse splitters news, I gave up doing AP only on my notebook, couldn't get any cache before the AP stopped being sent, may try again now :)
ID: 1248959 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 2871
Credit: 10,622,211
RAC: 332
United States
Message 1248976 - Posted: 20 Jun 2012, 23:13:43 UTC

Speaking of AP, can those last one or two completed v505's be manually kicked and cleared so the status page can reflect v6 statistics, or is there more to it than that?
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1248976 · Report as offensive
ClaggyProject Donor
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4623
Credit: 46,352,417
RAC: 3,002
United Kingdom
Message 1249346 - Posted: 21 Jun 2012, 11:10:55 UTC

Again speaking of Astropulse, can the sequence of validation be fixed so some pending validations don't sit around indefinitely, for example:

Workunit 955322258

Claggy
ID: 1249346 · Report as offensive
Profile Michel448a
Volunteer tester
Avatar

Send message
Joined: 27 Oct 00
Posts: 1201
Credit: 2,891,635
RAC: 0
Canada
Message 1249365 - Posted: 21 Jun 2012, 11:47:12 UTC - in response to Message 1248935.  
Last modified: 21 Jun 2012, 11:47:40 UTC

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?
ID: 1249365 · Report as offensive
Profile Julie
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 28 Oct 09
Posts: 33251
Credit: 9,032,399
RAC: 8,582
Belgium
Message 1249459 - Posted: 21 Jun 2012, 17:10:20 UTC

Thanx for the update Matt.

rOZZ
MUSIC
ID: 1249459 · Report as offensive
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 18644
Credit: 21,479,234
RAC: 19,486
United States
Message 1249502 - Posted: 21 Jun 2012, 18:03:38 UTC - in response to Message 1249365.  

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.

ID: 1249502 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45941
Credit: 815,362,607
RAC: 125,176
United States
Message 1249505 - Posted: 21 Jun 2012, 18:06:09 UTC
Last modified: 21 Jun 2012, 18:06:58 UTC

Thank you for the updates and interesting news on the new servers.
Hope the insurance angle works out and the project gets reimbursed for some of the damage caused by the power problems!
Meow!
Always remember.....kitties are all Angels with fur.

Have made friends in this life.
Most were cats.
ID: 1249505 · Report as offensive
Profile Dimly Lit Lightbulb 😀Project Donor
Volunteer tester
Avatar

Send message
Joined: 30 Aug 08
Posts: 14363
Credit: 2,924,262
RAC: 3,491
United Kingdom
Message 1249506 - Posted: 21 Jun 2012, 18:08:47 UTC

Thanks for the news Matt.
ID: 1249506 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 38185
Credit: 21,398,303
RAC: 27,793
United Kingdom
Message 1249568 - Posted: 21 Jun 2012, 19:51:17 UTC

A most positive report, and very welcome too. Many thanks Matt.
ID: 1249568 · Report as offensive
Profile Ex
Volunteer tester
Avatar

Send message
Joined: 12 Mar 12
Posts: 2895
Credit: 2,080,145
RAC: 421
United States
Message 1249782 - Posted: 22 Jun 2012, 4:59:59 UTC

Thanks Matt, and Gary for the elaboration on the Kepler.
-Dave #2

3.2.0-33
ID: 1249782 · Report as offensive
KB7RZF
Volunteer tester
Avatar

Send message
Joined: 15 Aug 99
Posts: 9467
Credit: 3,112,738
RAC: 0
United States
Message 1249797 - Posted: 22 Jun 2012, 5:33:45 UTC

Very good news Matt! Thank you for the updates!
ID: 1249797 · Report as offensive
Profile Slavac
Volunteer tester
Avatar

Send message
Joined: 27 Apr 11
Posts: 1932
Credit: 17,952,639
RAC: 0
United States
Message 1249877 - Posted: 22 Jun 2012, 9:15:58 UTC - in response to Message 1249502.  

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.


To add onto this a bit, PaddyM's drives were shipped off to the GBT (their second trip to the telescope) and came back nearly filled with data. That's 21, 2TB drives of data.

We'll process GBT on our PC's eventually, but I just wanted to give our users a scale of the data being processed in house vs archived for eventual S@H volunteer processing.


Executive Director GPU Users Group Inc. -
brad@gpuug.org
ID: 1249877 · Report as offensive
Profile Dimly Lit Lightbulb 😀Project Donor
Volunteer tester
Avatar

Send message
Joined: 30 Aug 08
Posts: 14363
Credit: 2,924,262
RAC: 3,491
United Kingdom
Message 1250101 - Posted: 22 Jun 2012, 20:21:19 UTC - in response to Message 1249877.  

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.


To add onto this a bit, PaddyM's drives were shipped off to the GBT (their second trip to the telescope) and came back nearly filled with data. That's 21, 2TB drives of data.

We'll process GBT on our PC's eventually, but I just wanted to give our users a scale of the data being processed in house vs archived for eventual S@H volunteer processing.

42TB, round about five and a half million* astropulses. At one and a half astropulses a day that would keep me crunching for bit :)

*maybe inaccurate as I'm bad at mathematics :)
ID: 1250101 · Report as offensive
Scott Campbell

Send message
Joined: 27 Jun 99
Posts: 4
Credit: 9,648,843
RAC: 465
United States
Message 1250399 - Posted: 23 Jun 2012, 13:12:11 UTC - in response to Message 1248935.  

Matt are we down again on the 23rd? I am down to 5 tasks left and they are small ones. It seems I haven't received anything since the 17th.
Thanks, Scott
ID: 1250399 · Report as offensive
ClaggyProject Donor
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4623
Credit: 46,352,417
RAC: 3,002
United Kingdom
Message 1250413 - Posted: 23 Jun 2012, 13:52:37 UTC - in response to Message 1250399.  
Last modified: 23 Jun 2012, 13:57:21 UTC

Matt are we down again on the 23rd? I am down to 5 tasks left and they are small ones. It seems I haven't received anything since the 17th.
Thanks, Scott

According to your host's task page you're received at least 78 tasks since the 17th, have 17 in progess tasks, and received 8 tasks today:

All tasks for computer 4964303

Have you supplied suitable cache settings for Boinc 7? (there have been changes to the scheduler for Boinc 7)

Claggy
ID: 1250413 · Report as offensive
Profile Keith T.
Volunteer tester
Avatar

Send message
Joined: 23 Aug 99
Posts: 743
Credit: 244,276
RAC: 0
United Kingdom
Message 1251815 - Posted: 26 Jun 2012, 14:25:58 UTC

Still 37 Astropulse V5 results stuck in the pipe. The last one was reported around 6th June. I think these must need some manual intervention by project staff to validate and purge. NC thread http://setiathome.berkeley.edu/forum_thread.php?id=67920 refers to this.

Keith.
ID: 1251815 · Report as offensive
ClaggyProject Donor
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4623
Credit: 46,352,417
RAC: 3,002
United Kingdom
Message 1251839 - Posted: 26 Jun 2012, 22:19:01 UTC - in response to Message 1251815.  

Still 37 Astropulse V5 results stuck in the pipe. The last one was reported around 6th June. I think these must need some manual intervention by project staff to validate and purge. NC thread http://setiathome.berkeley.edu/forum_thread.php?id=67920 refers to this.

Keith.

That's 37 Astropulse v505 results that are stuck, Astropulse v5 has been obsolete for a couple of years now:

Setiathome Applications

Claggy
ID: 1251839 · Report as offensive
Nick
Avatar

Send message
Joined: 11 Oct 11
Posts: 4344
Credit: 2,621,696
RAC: 568
United Kingdom
Message 1252683 - Posted: 28 Jun 2012, 16:01:49 UTC

Thanks Mat for your update, I was wondering why I was not getting any Astropulse
units.

The Kite Fliers

--------------------
Kite fliers: An imaginary club of solo members, those who don't yet
belong to a formal team so "fly their own kites" - as the saying goes.
ID: 1252683 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Swimming in Data (Jun 20 2012)


 
©2016 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.