The Black Fruit (Apr 07 2011)


log in

Advanced search

Message boards : Technical News : The Black Fruit (Apr 07 2011)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 1094515 - Posted: 7 Apr 2011, 22:42:01 UTC

Turns out one of the inputs to our data recorder got messed up. I don't have the details, but this only happened recently - we haven't yet gotten the raw data from Arecibo to split into workunits, etc. And the problem will be fixed soon if not already. The key now is to add some smarts to our scripts to avoid splitting the particular beam on that particular day or so. I guess this is a feature we've been meaning to add, and now we have a reason.

I've been messing a lot with gprof the past couple of days trying to find the subroutines causing the greatest slowdowns in our final analysis code. This actually wasn't getting me very far - I just now had to resort to some fprintf's with microsecond-level timestamps. No big news - we're database i/o constrained but now I have some numbers. We're brainstorming ideas about how to keep only the stuff we need in memory (as opposed to whole tables or indexes). Also I've been getting back to work cleaning up the NTPCkr pages for ultimate public consumption (and ultimately public assistance in helping us find and score detections). Slow, steady progress.

Otherwise all systems are go - bulking up the data pipeline before the weekend, queues are draining, etc. The mysql replica seems to be chugging along just fine. We'll see how it goes through the weekend before we call that project done.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4067
Credit: 32,892,452
RAC: 7,291
United Kingdom
Message 1094523 - Posted: 7 Apr 2011, 23:37:04 UTC - in response to Message 1094515.

Thanks for the update Matt,

Claggy

S@NL - Nick Succorso
Send message
Joined: 24 May 00
Posts: 15
Credit: 23,074,763
RAC: 0
Netherlands
Message 1094636 - Posted: 8 Apr 2011, 9:33:26 UTC

Thanks for all your efforts to keep us up to date, Matt.

However, I have a question that was stated before by someone else, but which most likely disappeared into the bit-bucket. What happened to the pending credits?
I notice the number of them is only getting lower, so my assumption is that you keep the recent pending credits stacked away in some cubbyhole hidden from the public (us, your faithful assistents out there in the cold). Could you give us an idea about the reasons why you keep them hidden and whether they will come back online again? Thanks for your response on this! Cheers, Nick
____________

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8465
Credit: 48,907,187
RAC: 76,419
United Kingdom
Message 1094646 - Posted: 8 Apr 2011, 10:55:06 UTC - in response to Message 1094636.

Thanks for all your efforts to keep us up to date, Matt.

However, I have a question that was stated before by someone else, but which most likely disappeared into the bit-bucket. What happened to the pending credits?
I notice the number of them is only getting lower, so my assumption is that you keep the recent pending credits stacked away in some cubbyhole hidden from the public (us, your faithful assistents out there in the cold). Could you give us an idea about the reasons why you keep them hidden and whether they will come back online again? Thanks for your response on this! Cheers, Nick

That's actually a change implemented by the BOINC developers, not Matt. See this post in Number Crunching for details.

Profile MarkJProject donor
Volunteer tester
Avatar
Send message
Joined: 17 Feb 08
Posts: 937
Credit: 22,510,889
RAC: 91,871
Australia
Message 1094650 - Posted: 8 Apr 2011, 11:39:41 UTC

Hi Matt,

Thanks for the regular updates.

Can you ask Eric to liase with Pappa regarding the next upgrades needed. I'm thinking hard disks are probably the next thing to tackle, but you may have something else needing more urgent attention.

Cheers,
MarkJ

S@NL - Nick Succorso
Send message
Joined: 24 May 00
Posts: 15
Credit: 23,074,763
RAC: 0
Netherlands
Message 1094676 - Posted: 8 Apr 2011, 13:18:35 UTC - in response to Message 1094646.

Thanks for all your efforts to keep us up to date, Matt.

However, I have a question that was stated before by someone else, but which most likely disappeared into the bit-bucket. What happened to the pending credits?
I notice the number of them is only getting lower, so my assumption is that you keep the recent pending credits stacked away in some cubbyhole hidden from the public (us, your faithful assistents out there in the cold). Could you give us an idea about the reasons why you keep them hidden and whether they will come back online again? Thanks for your response on this! Cheers, Nick

That's actually a change implemented by the BOINC developers, not Matt. See this post in Number Crunching for details.

Thanks Richard, I am aware that Matt doesn't know the answers to all questions, but I supposed he knows a lot of them ;)

I did check the thread you pointed out, but that doesn't really give the answer either, does it? It merely states the same question in different words and putting it in the context of the new credit system. A proper explanation is not given by anyone. Well, that's the way it is, obviously.

In any case, visiting the thread also introduced me to the fact of this new credit system being implemented, of which I was not aware. This probably is my own fault, as I am not following this particular forum on a regular basis.

Still, with all repect to the people involved, a little more effort by the Seti lab or Berkeley to communicate these important things to us, would not do any harm, would it? Instead of digging through the long and winding threads on a forum and per chance come upon things that were changed. I know, I know: too few people and too low budgets.
____________

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8465
Credit: 48,907,187
RAC: 76,419
United Kingdom
Message 1094683 - Posted: 8 Apr 2011, 13:33:39 UTC - in response to Message 1094676.
Last modified: 8 Apr 2011, 13:34:05 UTC

Thanks Richard, I am aware that Matt doesn't know the answers to all questions, but I supposed he knows a lot of them ;)

Actually, I think Matt himself has said that he isn't necessarily aware of all the things that change in BOINC - that's a different project from SETI@home, even if they do live on the same campus. My post was at least as much intended to let him know the source of the problem you'd raised, as it was to give you the answer directly. Might save Matt a bit of time searching and asking questions, too!

And also, I wasn't referring either of you to the thread in general, but a particular post by Joe Segur which contains the precise technical answer: changeset [23118].

Profile Chris SProject donor
Volunteer tester
Avatar
Send message
Joined: 19 Nov 00
Posts: 31449
Credit: 12,164,541
RAC: 28,693
United Kingdom
Message 1095659 - Posted: 10 Apr 2011, 14:39:08 UTC

even if they do live on the same campus.


Richard, aren't they all in the same SSL building?

SSL
____________
Damsel Rescuer, Kitty Patron, Uli Devotee, Julie Supporter
ES99 Admirer, Raccoon Friend, Anniet fan, RJ45 rulz OK!


Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1922
Credit: 9,734,758
RAC: 15,782
United States
Message 1095690 - Posted: 10 Apr 2011, 16:24:43 UTC

Mmm, the SSL is two buildings... and even if the BOINC people were in the same room as SETI, Matt is probably so busy keeping S@H running that he doesn't/can't keep up with the BOINC details.
____________
.

Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 1096209 - Posted: 11 Apr 2011, 20:10:52 UTC - in response to Message 1095690.

Mmm, the SSL is two buildings... and even if the BOINC people were in the same room as SETI, Matt is probably so busy keeping S@H running that he doesn't/can't keep up with the BOINC details.


Honestly, Dave's desk is literally right next to mine (albeit we're not facing each other), but (a) our schedules are not in sync so we're only both here a small fraction of the time and (b) we tend to be lost in whatever we're working on to discuss. And if we do discuss anything it's either (a) server related, (b) music related, or (c) hiking related. I kind of don't want to know about BOINC. I try to treat it like an object in the programming sense - keeping all the private stuff SETI@home shouldn't have to care about private. Otherwise I'd be responsible for that stuff too. I got plenty to worry about. Well of course I still end up knowing a lot about BOINC but I may not be up to speed on latest problems/features/advancements/plans so I'd rather just not say anything lest I confuse people.

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Patrick M. ORourke
Volunteer tester
Avatar
Send message
Joined: 21 Aug 02
Posts: 28
Credit: 643,752
RAC: 0
United States
Message 1096507 - Posted: 12 Apr 2011, 14:09:19 UTC - in response to Message 1096209.

When you said the project will be done, is this why I am only getting a few work units on one of my computers?
____________
Patrick M. O'Rourke


msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38920
Credit: 578,406,297
RAC: 516,182
United States
Message 1096719 - Posted: 13 Apr 2011, 6:02:11 UTC - in response to Message 1096507.

When you said the project will be done, is this why I am only getting a few work units on one of my computers?

Post your question in Number Crunching and somebody may be able to help you figure out what your difficulty is.
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1922
Credit: 9,734,758
RAC: 15,782
United States
Message 1098480 - Posted: 17 Apr 2011, 17:45:37 UTC

from the "Client Connection Stats" page:

last updated: October 21 2010 19:25:01

[irony]
Something appears to be broken…
[/irony]
____________
.

John M. Kendall
Avatar
Send message
Joined: 15 May 99
Posts: 24
Credit: 5,740,071
RAC: 1,723
United States
Message 1098569 - Posted: 18 Apr 2011, 1:31:16 UTC

Pending Credits: So it is just me? My number of pending credits has been slipping steadily away since March 8. Back then I had 25 thousand and now it is down to 811!
At first I thought with the new SETI processors and all those folks at home with new computers the pending back log was being processed faster. I take it that is not the case?
____________

msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38920
Credit: 578,406,297
RAC: 516,182
United States
Message 1098572 - Posted: 18 Apr 2011, 1:39:10 UTC - in response to Message 1098569.

Pending Credits: So it is just me? My number of pending credits has been slipping steadily away since March 8. Back then I had 25 thousand and now it is down to 811!
At first I thought with the new SETI processors and all those folks at home with new computers the pending back log was being processed faster. I take it that is not the case?

Nope...
The pending credits page is no longer being supported by Boinc.
What you see there will dwindle as the WUs are returned by other crunchers and are retired.
Pending credits are no longer being added.
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12395
Credit: 6,704,796
RAC: 8,692
United States
Message 1098624 - Posted: 18 Apr 2011, 4:53:32 UTC - in response to Message 1098572.

Pending Credits: So it is just me? My number of pending credits has been slipping steadily away since March 8. Back then I had 25 thousand and now it is down to 811!
At first I thought with the new SETI processors and all those folks at home with new computers the pending back log was being processed faster. I take it that is not the case?

Nope...
The pending credits page is no longer being supported by Boinc.
What you see there will dwindle as the WUs are returned by other crunchers and are retired.
Pending credits are no longer being added.

Kind of.

If you go to your all tasks page, you will find a link for pending tasks. That link does work. Unfortunately it doesn't give a total.

____________

Message boards : Technical News : The Black Fruit (Apr 07 2011)

Copyright © 2014 University of California