The Black Fruit (Apr 07 2011)

Message boards : Technical News : The Black Fruit (Apr 07 2011)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1094515 - Posted: 7 Apr 2011, 22:42:01 UTC

Turns out one of the inputs to our data recorder got messed up. I don't have the details, but this only happened recently - we haven't yet gotten the raw data from Arecibo to split into workunits, etc. And the problem will be fixed soon if not already. The key now is to add some smarts to our scripts to avoid splitting the particular beam on that particular day or so. I guess this is a feature we've been meaning to add, and now we have a reason.

I've been messing a lot with gprof the past couple of days trying to find the subroutines causing the greatest slowdowns in our final analysis code. This actually wasn't getting me very far - I just now had to resort to some fprintf's with microsecond-level timestamps. No big news - we're database i/o constrained but now I have some numbers. We're brainstorming ideas about how to keep only the stuff we need in memory (as opposed to whole tables or indexes). Also I've been getting back to work cleaning up the NTPCkr pages for ultimate public consumption (and ultimately public assistance in helping us find and score detections). Slow, steady progress.

Otherwise all systems are go - bulking up the data pipeline before the weekend, queues are draining, etc. The mysql replica seems to be chugging along just fine. We'll see how it goes through the weekend before we call that project done.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1094515 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1094523 - Posted: 7 Apr 2011, 23:37:04 UTC - in response to Message 1094515.  

Thanks for the update Matt,

Claggy
ID: 1094523 · Report as offensive
S@NL - Nick Succorso

Send message
Joined: 24 May 00
Posts: 15
Credit: 23,074,763
RAC: 0
Netherlands
Message 1094636 - Posted: 8 Apr 2011, 9:33:26 UTC

Thanks for all your efforts to keep us up to date, Matt.

However, I have a question that was stated before by someone else, but which most likely disappeared into the bit-bucket. What happened to the pending credits?
I notice the number of them is only getting lower, so my assumption is that you keep the recent pending credits stacked away in some cubbyhole hidden from the public (us, your faithful assistents out there in the cold). Could you give us an idea about the reasons why you keep them hidden and whether they will come back online again? Thanks for your response on this! Cheers, Nick
ID: 1094636 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1094646 - Posted: 8 Apr 2011, 10:55:06 UTC - in response to Message 1094636.  

Thanks for all your efforts to keep us up to date, Matt.

However, I have a question that was stated before by someone else, but which most likely disappeared into the bit-bucket. What happened to the pending credits?
I notice the number of them is only getting lower, so my assumption is that you keep the recent pending credits stacked away in some cubbyhole hidden from the public (us, your faithful assistents out there in the cold). Could you give us an idea about the reasons why you keep them hidden and whether they will come back online again? Thanks for your response on this! Cheers, Nick

That's actually a change implemented by the BOINC developers, not Matt. See this post in Number Crunching for details.
ID: 1094646 · Report as offensive
MarkJ Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 08
Posts: 1139
Credit: 80,854,192
RAC: 5
Australia
Message 1094650 - Posted: 8 Apr 2011, 11:39:41 UTC

Hi Matt,

Thanks for the regular updates.

Can you ask Eric to liase with Pappa regarding the next upgrades needed. I'm thinking hard disks are probably the next thing to tackle, but you may have something else needing more urgent attention.

Cheers,
MarkJ
ID: 1094650 · Report as offensive
S@NL - Nick Succorso

Send message
Joined: 24 May 00
Posts: 15
Credit: 23,074,763
RAC: 0
Netherlands
Message 1094676 - Posted: 8 Apr 2011, 13:18:35 UTC - in response to Message 1094646.  

Thanks for all your efforts to keep us up to date, Matt.

However, I have a question that was stated before by someone else, but which most likely disappeared into the bit-bucket. What happened to the pending credits?
I notice the number of them is only getting lower, so my assumption is that you keep the recent pending credits stacked away in some cubbyhole hidden from the public (us, your faithful assistents out there in the cold). Could you give us an idea about the reasons why you keep them hidden and whether they will come back online again? Thanks for your response on this! Cheers, Nick

That's actually a change implemented by the BOINC developers, not Matt. See this post in Number Crunching for details.

Thanks Richard, I am aware that Matt doesn't know the answers to all questions, but I supposed he knows a lot of them ;)

I did check the thread you pointed out, but that doesn't really give the answer either, does it? It merely states the same question in different words and putting it in the context of the new credit system. A proper explanation is not given by anyone. Well, that's the way it is, obviously.

In any case, visiting the thread also introduced me to the fact of this new credit system being implemented, of which I was not aware. This probably is my own fault, as I am not following this particular forum on a regular basis.

Still, with all repect to the people involved, a little more effort by the Seti lab or Berkeley to communicate these important things to us, would not do any harm, would it? Instead of digging through the long and winding threads on a forum and per chance come upon things that were changed. I know, I know: too few people and too low budgets.
ID: 1094676 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14679
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1094683 - Posted: 8 Apr 2011, 13:33:39 UTC - in response to Message 1094676.  
Last modified: 8 Apr 2011, 13:34:05 UTC

Thanks Richard, I am aware that Matt doesn't know the answers to all questions, but I supposed he knows a lot of them ;)

Actually, I think Matt himself has said that he isn't necessarily aware of all the things that change in BOINC - that's a different project from SETI@home, even if they do live on the same campus. My post was at least as much intended to let him know the source of the problem you'd raised, as it was to give you the answer directly. Might save Matt a bit of time searching and asking questions, too!

And also, I wasn't referring either of you to the thread in general, but a particular post by Joe Segur which contains the precise technical answer: changeset [trac]changeset:23118[/trac].
ID: 1094683 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1095690 - Posted: 10 Apr 2011, 16:24:43 UTC

Mmm, the SSL is two buildings... and even if the BOINC people were in the same room as SETI, Matt is probably so busy keeping S@H running that he doesn't/can't keep up with the BOINC details.
.

Hello, from Albany, CA!...
ID: 1095690 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1096209 - Posted: 11 Apr 2011, 20:10:52 UTC - in response to Message 1095690.  

Mmm, the SSL is two buildings... and even if the BOINC people were in the same room as SETI, Matt is probably so busy keeping S@H running that he doesn't/can't keep up with the BOINC details.


Honestly, Dave's desk is literally right next to mine (albeit we're not facing each other), but (a) our schedules are not in sync so we're only both here a small fraction of the time and (b) we tend to be lost in whatever we're working on to discuss. And if we do discuss anything it's either (a) server related, (b) music related, or (c) hiking related. I kind of don't want to know about BOINC. I try to treat it like an object in the programming sense - keeping all the private stuff SETI@home shouldn't have to care about private. Otherwise I'd be responsible for that stuff too. I got plenty to worry about. Well of course I still end up knowing a lot about BOINC but I may not be up to speed on latest problems/features/advancements/plans so I'd rather just not say anything lest I confuse people.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1096209 · Report as offensive
Patrick M. ORourke
Volunteer tester
Avatar

Send message
Joined: 21 Aug 02
Posts: 28
Credit: 643,752
RAC: 0
United States
Message 1096507 - Posted: 12 Apr 2011, 14:09:19 UTC - in response to Message 1096209.  

When you said the project will be done, is this why I am only getting a few work units on one of my computers?
Patrick M. O'Rourke


ID: 1096507 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1096719 - Posted: 13 Apr 2011, 6:02:11 UTC - in response to Message 1096507.  

When you said the project will be done, is this why I am only getting a few work units on one of my computers?

Post your question in Number Crunching and somebody may be able to help you figure out what your difficulty is.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1096719 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1098480 - Posted: 17 Apr 2011, 17:45:37 UTC

from the "Client Connection Stats" page:

last updated: October 21 2010 19:25:01

[irony]
Something appears to be broken…
[/irony]
.

Hello, from Albany, CA!...
ID: 1098480 · Report as offensive
John M. Kendall
Avatar

Send message
Joined: 15 May 99
Posts: 30
Credit: 11,800,561
RAC: 164
United States
Message 1098569 - Posted: 18 Apr 2011, 1:31:16 UTC

Pending Credits: So it is just me? My number of pending credits has been slipping steadily away since March 8. Back then I had 25 thousand and now it is down to 811!
At first I thought with the new SETI processors and all those folks at home with new computers the pending back log was being processed faster. I take it that is not the case?
ID: 1098569 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51477
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1098572 - Posted: 18 Apr 2011, 1:39:10 UTC - in response to Message 1098569.  

Pending Credits: So it is just me? My number of pending credits has been slipping steadily away since March 8. Back then I had 25 thousand and now it is down to 811!
At first I thought with the new SETI processors and all those folks at home with new computers the pending back log was being processed faster. I take it that is not the case?

Nope...
The pending credits page is no longer being supported by Boinc.
What you see there will dwindle as the WUs are returned by other crunchers and are retired.
Pending credits are no longer being added.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1098572 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30981
Credit: 53,134,872
RAC: 32
United States
Message 1098624 - Posted: 18 Apr 2011, 4:53:32 UTC - in response to Message 1098572.  

Pending Credits: So it is just me? My number of pending credits has been slipping steadily away since March 8. Back then I had 25 thousand and now it is down to 811!
At first I thought with the new SETI processors and all those folks at home with new computers the pending back log was being processed faster. I take it that is not the case?

Nope...
The pending credits page is no longer being supported by Boinc.
What you see there will dwindle as the WUs are returned by other crunchers and are retired.
Pending credits are no longer being added.

Kind of.

If you go to your all tasks page, you will find a link for pending tasks. That link does work. Unfortunately it doesn't give a total.

ID: 1098624 · Report as offensive

Message boards : Technical News : The Black Fruit (Apr 07 2011)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.