Swimming in Data (Jun 20 2012)


log in

Advanced search

Message boards : Technical News : Swimming in Data (Jun 20 2012)

1 · 2 · Next
Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 1248935 - Posted: 20 Jun 2012, 21:53:23 UTC

Some news. Yesterday we had our usual weekly outage, and shortly after the floodgates opened again bruno (the upload server) crashed. Except we quickly found it didn't actually crash. It was turned off. By the web-enabled power strip. For no apparent reason. We turned it back on and everything was okay, but now it seems like we have a flaky web-enabled power strip on our hands. It is interesting to note that this power strip was plugged into the same breaker as thinman - the previous webserver system that died during that last unexpected power issues. So maybe some funky voltage clobbered this strip as well. Well, we have a spare one which works so no big shakes there. And yes, we ruled out foul play.

As for the crashy desktop machines, I may have fixed one. The theory being, oddly enough, too much thermal grease was employed thus reducing the effectiveness of the heat sink. Oops. Well, I'm not quite convinced that was the problem, and we're burning it in now. If it survives a week without crashing, great. The other system is not doing as well. I think we're aiming to get insurance money from the university to cover the cost of these systems killed or injured during these outages. Meanwhile, we're operational, so no real disaster.

In better news, georgem is now not only hosting all the workunits and running some backend BOINC services and scientific analysis processes, but it's also hosting all the data (~13TB) from a recent survey of the galactic center collected at Green Bank Telescope. Several grad students will be processing this data on georgem itself.

Also paddym has been cleared to finally reformat all its drives into a giant RAID10, and we can now start the process of duplicating the whole SETI@home informix science database on oscar over there. As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

Oh yeah we also found a bug that had been causing a lot of Astropulse splitters to fail, thus reducing the amount of AP workunits being sent out. This has been fixed, and so expect more AP work.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4140
Credit: 33,544,766
RAC: 25,104
United Kingdom
Message 1248943 - Posted: 20 Jun 2012, 22:01:12 UTC - in response to Message 1248935.

Thanks for the update Matt,

Claggy

JohnDKProject donor
Volunteer tester
Avatar
Send message
Joined: 28 May 00
Posts: 862
Credit: 46,657,254
RAC: 74,044
Denmark
Message 1248959 - Posted: 20 Jun 2012, 22:17:15 UTC

Yes thanks, no least the Astropulse splitters news, I gave up doing AP only on my notebook, couldn't get any cache before the AP stopped being sent, may try again now :)

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2287
Credit: 8,807,813
RAC: 4,138
United States
Message 1248976 - Posted: 20 Jun 2012, 23:13:43 UTC

Speaking of AP, can those last one or two completed v505's be manually kicked and cleared so the status page can reflect v6 statistics, or is there more to it than that?
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4140
Credit: 33,544,766
RAC: 25,104
United Kingdom
Message 1249346 - Posted: 21 Jun 2012, 11:10:55 UTC

Again speaking of Astropulse, can the sequence of validation be fixed so some pending validations don't sit around indefinitely, for example:

Workunit 955322258

Claggy

Profile Michel448a
Volunteer tester
Avatar
Send message
Joined: 27 Oct 00
Posts: 1201
Credit: 2,891,635
RAC: 0
Canada
Message 1249365 - Posted: 21 Jun 2012, 11:47:12 UTC - in response to Message 1248935.
Last modified: 21 Jun 2012, 11:47:40 UTC

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?
____________

Profile JulieProject donor
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 28 Oct 09
Posts: 21992
Credit: 3,922,526
RAC: 6,368
Belgium
Message 1249459 - Posted: 21 Jun 2012, 17:10:20 UTC

Thanx for the update Matt.
____________


rOZZ

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12722
Credit: 7,238,062
RAC: 16,523
United States
Message 1249502 - Posted: 21 Jun 2012, 18:03:38 UTC - in response to Message 1249365.

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.

____________

Profile Zapped "Sixth Sense" Sparky
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 30 Aug 08
Posts: 8370
Credit: 1,295,459
RAC: 973
United Kingdom
Message 1249506 - Posted: 21 Jun 2012, 18:08:47 UTC

Thanks for the news Matt.
____________
In an alternate universe, it was a ZX81 that asked for clothes, boots and motorcycle.

Client error 418: I'm a teapot

Tropical Goldfish Fish 15: Squeaky bras 'R us

Illusions of normality sufferer

Profile Chris SProject donor
Volunteer tester
Avatar
Send message
Joined: 19 Nov 00
Posts: 32073
Credit: 13,756,268
RAC: 26,108
United Kingdom
Message 1249568 - Posted: 21 Jun 2012, 19:51:17 UTC

A most positive report, and very welcome too. Many thanks Matt.

Profile Ex
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 12 Mar 12
Posts: 2895
Credit: 1,797,015
RAC: 1,328
United States
Message 1249782 - Posted: 22 Jun 2012, 4:59:59 UTC

Thanks Matt, and Gary for the elaboration on the Kepler.
____________
-Dave #2

3.2.0-33

KB7RZF
Volunteer tester
Avatar
Send message
Joined: 15 Aug 99
Posts: 9463
Credit: 3,112,738
RAC: 39
United States
Message 1249797 - Posted: 22 Jun 2012, 5:33:45 UTC

Very good news Matt! Thank you for the updates!
____________

Profile Slavac
Volunteer tester
Avatar
Send message
Joined: 27 Apr 11
Posts: 1932
Credit: 17,952,639
RAC: 0
United States
Message 1249877 - Posted: 22 Jun 2012, 9:15:58 UTC - in response to Message 1249502.

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.


To add onto this a bit, PaddyM's drives were shipped off to the GBT (their second trip to the telescope) and came back nearly filled with data. That's 21, 2TB drives of data.

We'll process GBT on our PC's eventually, but I just wanted to give our users a scale of the data being processed in house vs archived for eventual S@H volunteer processing.
____________


Executive Director GPU Users Group Inc. -
brad@gpuug.org

Profile Zapped "Sixth Sense" Sparky
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 30 Aug 08
Posts: 8370
Credit: 1,295,459
RAC: 973
United Kingdom
Message 1250101 - Posted: 22 Jun 2012, 20:21:19 UTC - in response to Message 1249877.

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.


To add onto this a bit, PaddyM's drives were shipped off to the GBT (their second trip to the telescope) and came back nearly filled with data. That's 21, 2TB drives of data.

We'll process GBT on our PC's eventually, but I just wanted to give our users a scale of the data being processed in house vs archived for eventual S@H volunteer processing.

42TB, round about five and a half million* astropulses. At one and a half astropulses a day that would keep me crunching for bit :)

*maybe inaccurate as I'm bad at mathematics :)
____________
In an alternate universe, it was a ZX81 that asked for clothes, boots and motorcycle.

Client error 418: I'm a teapot

Tropical Goldfish Fish 15: Squeaky bras 'R us

Illusions of normality sufferer

Scott Campbell
Send message
Joined: 27 Jun 99
Posts: 4
Credit: 6,144,556
RAC: 5,918
United States
Message 1250399 - Posted: 23 Jun 2012, 13:12:11 UTC - in response to Message 1248935.

Matt are we down again on the 23rd? I am down to 5 tasks left and they are small ones. It seems I haven't received anything since the 17th.
Thanks, Scott
____________

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4140
Credit: 33,544,766
RAC: 25,104
United Kingdom
Message 1250413 - Posted: 23 Jun 2012, 13:52:37 UTC - in response to Message 1250399.
Last modified: 23 Jun 2012, 13:57:21 UTC

Matt are we down again on the 23rd? I am down to 5 tasks left and they are small ones. It seems I haven't received anything since the 17th.
Thanks, Scott

According to your host's task page you're received at least 78 tasks since the 17th, have 17 in progess tasks, and received 8 tasks today:

All tasks for computer 4964303

Have you supplied suitable cache settings for Boinc 7? (there have been changes to the scheduler for Boinc 7)

Claggy

Profile Keith T.
Volunteer tester
Avatar
Send message
Joined: 23 Aug 99
Posts: 738
Credit: 232,725
RAC: 10
United Kingdom
Message 1251815 - Posted: 26 Jun 2012, 14:25:58 UTC

Still 37 Astropulse V5 results stuck in the pipe. The last one was reported around 6th June. I think these must need some manual intervention by project staff to validate and purge. NC thread http://setiathome.berkeley.edu/forum_thread.php?id=67920 refers to this.

Keith.

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4140
Credit: 33,544,766
RAC: 25,104
United Kingdom
Message 1251839 - Posted: 26 Jun 2012, 22:19:01 UTC - in response to Message 1251815.

Still 37 Astropulse V5 results stuck in the pipe. The last one was reported around 6th June. I think these must need some manual intervention by project staff to validate and purge. NC thread http://setiathome.berkeley.edu/forum_thread.php?id=67920 refers to this.

Keith.

That's 37 Astropulse v505 results that are stuck, Astropulse v5 has been obsolete for a couple of years now:

Setiathome Applications

Claggy

NickProject donor
Avatar
Send message
Joined: 11 Oct 11
Posts: 3978
Credit: 2,033,826
RAC: 956
United Kingdom
Message 1252683 - Posted: 28 Jun 2012, 16:01:49 UTC

Thanks Mat for your update, I was wondering why I was not getting any Astropulse
units.

____________
The Kite Fliers

--------------------
Kite fliers: An imaginary club of solo members, those who don't yet
belong to a formal team so "fly their own kites" - as the saying goes.

1 · 2 · Next

Message boards : Technical News : Swimming in Data (Jun 20 2012)

Copyright © 2014 University of California