Swimming in Data (Jun 20 2012)

Message boards : Technical News : Swimming in Data (Jun 20 2012)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1248935 - Posted: 20 Jun 2012, 21:53:23 UTC

Some news. Yesterday we had our usual weekly outage, and shortly after the floodgates opened again bruno (the upload server) crashed. Except we quickly found it didn't actually crash. It was turned off. By the web-enabled power strip. For no apparent reason. We turned it back on and everything was okay, but now it seems like we have a flaky web-enabled power strip on our hands. It is interesting to note that this power strip was plugged into the same breaker as thinman - the previous webserver system that died during that last unexpected power issues. So maybe some funky voltage clobbered this strip as well. Well, we have a spare one which works so no big shakes there. And yes, we ruled out foul play.

As for the crashy desktop machines, I may have fixed one. The theory being, oddly enough, too much thermal grease was employed thus reducing the effectiveness of the heat sink. Oops. Well, I'm not quite convinced that was the problem, and we're burning it in now. If it survives a week without crashing, great. The other system is not doing as well. I think we're aiming to get insurance money from the university to cover the cost of these systems killed or injured during these outages. Meanwhile, we're operational, so no real disaster.

In better news, georgem is now not only hosting all the workunits and running some backend BOINC services and scientific analysis processes, but it's also hosting all the data (~13TB) from a recent survey of the galactic center collected at Green Bank Telescope. Several grad students will be processing this data on georgem itself.

Also paddym has been cleared to finally reformat all its drives into a giant RAID10, and we can now start the process of duplicating the whole SETI@home informix science database on oscar over there. As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

Oh yeah we also found a bug that had been causing a lot of Astropulse splitters to fail, thus reducing the amount of AP workunits being sent out. This has been fixed, and so expect more AP work.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1248935 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1248943 - Posted: 20 Jun 2012, 22:01:12 UTC - in response to Message 1248935.  

Thanks for the update Matt,

Claggy
ID: 1248943 · Report as offensive
JohnDK Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 1222
Credit: 451,243,443
RAC: 1,127
Denmark
Message 1248959 - Posted: 20 Jun 2012, 22:17:15 UTC

Yes thanks, no least the Astropulse splitters news, I gave up doing AP only on my notebook, couldn't get any cache before the AP stopped being sent, may try again now :)
ID: 1248959 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1248976 - Posted: 20 Jun 2012, 23:13:43 UTC

Speaking of AP, can those last one or two completed v505's be manually kicked and cleared so the status page can reflect v6 statistics, or is there more to it than that?
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1248976 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1249346 - Posted: 21 Jun 2012, 11:10:55 UTC

Again speaking of Astropulse, can the sequence of validation be fixed so some pending validations don't sit around indefinitely, for example:

Workunit 955322258

Claggy
ID: 1249346 · Report as offensive
Profile Julie
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 28 Oct 09
Posts: 34060
Credit: 18,883,157
RAC: 18
Belgium
Message 1249459 - Posted: 21 Jun 2012, 17:10:20 UTC

Thanx for the update Matt.
rOZZ
Music
Pictures
ID: 1249459 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30975
Credit: 53,134,872
RAC: 32
United States
Message 1249502 - Posted: 21 Jun 2012, 18:03:38 UTC - in response to Message 1249365.  

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.

ID: 1249502 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1249505 - Posted: 21 Jun 2012, 18:06:09 UTC
Last modified: 21 Jun 2012, 18:06:58 UTC

Thank you for the updates and interesting news on the new servers.
Hope the insurance angle works out and the project gets reimbursed for some of the damage caused by the power problems!
Meow!
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1249505 · Report as offensive
Profile Dimly Lit Lightbulb 😀
Volunteer tester
Avatar

Send message
Joined: 30 Aug 08
Posts: 15399
Credit: 7,423,413
RAC: 1
United Kingdom
Message 1249506 - Posted: 21 Jun 2012, 18:08:47 UTC

Thanks for the news Matt.

Member of the People Encouraging Niceness In Society club.

ID: 1249506 · Report as offensive
Profile Ex: "Socialist"
Volunteer tester
Avatar

Send message
Joined: 12 Mar 12
Posts: 3433
Credit: 2,616,158
RAC: 2
United States
Message 1249782 - Posted: 22 Jun 2012, 4:59:59 UTC

Thanks Matt, and Gary for the elaboration on the Kepler.
#resist
ID: 1249782 · Report as offensive
KB7RZF
Volunteer tester
Avatar

Send message
Joined: 15 Aug 99
Posts: 9549
Credit: 3,308,926
RAC: 2
United States
Message 1249797 - Posted: 22 Jun 2012, 5:33:45 UTC

Very good news Matt! Thank you for the updates!
ID: 1249797 · Report as offensive
Profile Slavac
Volunteer tester
Avatar

Send message
Joined: 27 Apr 11
Posts: 1932
Credit: 17,952,639
RAC: 0
United States
Message 1249877 - Posted: 22 Jun 2012, 9:15:58 UTC - in response to Message 1249502.  

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.


To add onto this a bit, PaddyM's drives were shipped off to the GBT (their second trip to the telescope) and came back nearly filled with data. That's 21, 2TB drives of data.

We'll process GBT on our PC's eventually, but I just wanted to give our users a scale of the data being processed in house vs archived for eventual S@H volunteer processing.


Executive Director GPU Users Group Inc. -
brad@gpuug.org
ID: 1249877 · Report as offensive
Profile Dimly Lit Lightbulb 😀
Volunteer tester
Avatar

Send message
Joined: 30 Aug 08
Posts: 15399
Credit: 7,423,413
RAC: 1
United Kingdom
Message 1250101 - Posted: 22 Jun 2012, 20:21:19 UTC - in response to Message 1249877.  

As well it's already actually serving a mysql database containing Kepler data, also collected at the GBT, which we're soon to use old SERENDIP code to analyze in-house.

- Matt


i guess , we, people, wont process Kepler candidates at all ?

We will, just no rush. They have to rewrite the multi-beam and astropulse code to accept the different data format from GBT. That needs cash to pay the programmers. Then it rolls out on the Beta project.


To add onto this a bit, PaddyM's drives were shipped off to the GBT (their second trip to the telescope) and came back nearly filled with data. That's 21, 2TB drives of data.

We'll process GBT on our PC's eventually, but I just wanted to give our users a scale of the data being processed in house vs archived for eventual S@H volunteer processing.

42TB, round about five and a half million* astropulses. At one and a half astropulses a day that would keep me crunching for bit :)

*maybe inaccurate as I'm bad at mathematics :)

Member of the People Encouraging Niceness In Society club.

ID: 1250101 · Report as offensive
Scott Campbell

Send message
Joined: 27 Jun 99
Posts: 4
Credit: 9,913,372
RAC: 0
United States
Message 1250399 - Posted: 23 Jun 2012, 13:12:11 UTC - in response to Message 1248935.  

Matt are we down again on the 23rd? I am down to 5 tasks left and they are small ones. It seems I haven't received anything since the 17th.
Thanks, Scott
ID: 1250399 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1250413 - Posted: 23 Jun 2012, 13:52:37 UTC - in response to Message 1250399.  
Last modified: 23 Jun 2012, 13:57:21 UTC

Matt are we down again on the 23rd? I am down to 5 tasks left and they are small ones. It seems I haven't received anything since the 17th.
Thanks, Scott

According to your host's task page you're received at least 78 tasks since the 17th, have 17 in progess tasks, and received 8 tasks today:

All tasks for computer 4964303

Have you supplied suitable cache settings for Boinc 7? (there have been changes to the scheduler for Boinc 7)

Claggy
ID: 1250413 · Report as offensive
Profile Keith T.
Volunteer tester
Avatar

Send message
Joined: 23 Aug 99
Posts: 962
Credit: 537,293
RAC: 9
United Kingdom
Message 1251815 - Posted: 26 Jun 2012, 14:25:58 UTC

Still 37 Astropulse V5 results stuck in the pipe. The last one was reported around 6th June. I think these must need some manual intervention by project staff to validate and purge. NC thread http://setiathome.berkeley.edu/forum_thread.php?id=67920 refers to this.

Keith.
ID: 1251815 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1251839 - Posted: 26 Jun 2012, 22:19:01 UTC - in response to Message 1251815.  

Still 37 Astropulse V5 results stuck in the pipe. The last one was reported around 6th June. I think these must need some manual intervention by project staff to validate and purge. NC thread http://setiathome.berkeley.edu/forum_thread.php?id=67920 refers to this.

Keith.

That's 37 Astropulse v505 results that are stuck, Astropulse v5 has been obsolete for a couple of years now:

Setiathome Applications

Claggy
ID: 1251839 · Report as offensive
Nick
Avatar

Send message
Joined: 11 Oct 11
Posts: 4344
Credit: 3,313,107
RAC: 0
United Kingdom
Message 1252683 - Posted: 28 Jun 2012, 16:01:49 UTC

Thanks Mat for your update, I was wondering why I was not getting any Astropulse
units.

The Kite Fliers

--------------------
Kite fliers: An imaginary club of solo members, those who don't yet
belong to a formal team so "fly their own kites" - as the saying goes.
ID: 1252683 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1254230 - Posted: 1 Jul 2012, 17:19:18 UTC

Alert! Even though everything shows as "running" on the "server Status" page, uploads are very slow - and usually don't get through at all.

Sunday 1 July, 2012
.

Hello, from Albany, CA!...
ID: 1254230 · Report as offensive
tjcares

Send message
Joined: 6 Feb 11
Posts: 5
Credit: 141,065
RAC: 0
United States
Message 1254370 - Posted: 1 Jul 2012, 23:25:10 UTC
Last modified: 2 Jul 2012, 0:09:45 UTC

I also can't upload at all SETI@Home Enhanced 6.03 and AstroPulse ver 6 6.01

Have a quad core x64 unit here.

Seems fixed. Thanks:)
ID: 1254370 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Swimming in Data (Jun 20 2012)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.