Splitting Totals (Sep 24 2008)

Message boards : Technical News : Splitting Totals (Sep 24 2008)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 811652 - Posted: 24 Sep 2008, 21:12:42 UTC

Something we've been lagging on is separating the database count totals on the server status page. Currently we're showing "totals" - for example, the "results ready to send" is a sum of both SETI@home and Astropulse results ready to send. For diagnostic purposes, it would be much better to split these into two separate columns.

However, this isn't so easy, as such queries become suddenly very expensive if we're adding an additional "where appid = N" conditional (AstroPulse and SETI@home are considered two different "applications" in the BOINC realm). I'm talking the difference being a 3 second query versus a 3600 second query. Yup. We've made joint indexes in the past for servers that needed them, but this hasn't been a priority for diagnostic stuff. We also don't really have the memory/resources to keep such extra indexes around. In any case, Bob pointed out that newer versions of mysql are smarter - doing the index joins automatically - so we may push on upgrading mysql sooner than later.

Today I'm actually lost in mundane bureaucracy land. I also should be working on the new software radar blanking embedder code. Sigh.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 811652 · Report as offensive
Profile [KWSN]John Galt 007
Volunteer tester
Avatar

Send message
Joined: 9 Nov 99
Posts: 2444
Credit: 25,086,197
RAC: 0
United States
Message 811670 - Posted: 24 Sep 2008, 22:16:29 UTC

Thanks for the info...moving at the speed of state, eh??
Clk2HlpSetiCty:::PayIt4ward

ID: 811670 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 811685 - Posted: 24 Sep 2008, 23:16:10 UTC


. . . Thanks for the Post Matt - it'll get done when it's done, right ;)


BOINC Wiki . . .

Science Status Page . . .
ID: 811685 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30651
Credit: 53,134,872
RAC: 32
United States
Message 811699 - Posted: 25 Sep 2008, 0:16:44 UTC - in response to Message 811652.  

Something we've been lagging on is separating the database count totals on the server status page. Currently we're showing "totals" - for example, the "results ready to send" is a sum of both SETI@home and Astropulse results ready to send. For diagnostic purposes, it would be much better to split these into two separate columns.

However, this isn't so easy, as such queries become suddenly very expensive if we're adding an additional "where appid = N" conditional (AstroPulse and SETI@home are considered two different "applications" in the BOINC realm). I'm talking the difference being a 3 second query versus a 3600 second query. Yup. We've made joint indexes in the past for servers that needed them, but this hasn't been a priority for diagnostic stuff. We also don't really have the memory/resources to keep such extra indexes around. In any case, Bob pointed out that newer versions of mysql are smarter - doing the index joins automatically - so we may push on upgrading mysql sooner than later.

Today I'm actually lost in mundane bureaucracy land. I also should be working on the new software radar blanking embedder code. Sigh.

- Matt

Might give the science status page a swift kick. Seems to be stuck.

ID: 811699 · Report as offensive
DJStarfox

Send message
Joined: 23 May 01
Posts: 1066
Credit: 1,226,053
RAC: 2
United States
Message 811738 - Posted: 25 Sep 2008, 4:08:34 UTC - in response to Message 811652.  

However, this isn't so easy, as such queries become suddenly very expensive if we're adding an additional "where appid = N" conditional (AstroPulse and SETI@home are considered two different "applications" in the BOINC realm). I'm talking the difference being a 3 second query versus a 3600 second query. Yup.


What about this?
select
count(*) as total_count,
sum(case appid when "N" then 1 else 0) as Astropulse_count
from table

Then do total_count - Astropulse_count = Enhanced_count
ID: 811738 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 811857 - Posted: 25 Sep 2008, 14:42:04 UTC - in response to Message 811699.  

Something we've been lagging on is separating the database count totals on the server status page. Currently we're showing "totals" - for example, the "results ready to send" is a sum of both SETI@home and Astropulse results ready to send. For diagnostic purposes, it would be much better to split these into two separate columns.

However, this isn't so easy, as such queries become suddenly very expensive if we're adding an additional "where appid = N" conditional (AstroPulse and SETI@home are considered two different "applications" in the BOINC realm). I'm talking the difference being a 3 second query versus a 3600 second query. Yup. We've made joint indexes in the past for servers that needed them, but this hasn't been a priority for diagnostic stuff. We also don't really have the memory/resources to keep such extra indexes around. In any case, Bob pointed out that newer versions of mysql are smarter - doing the index joins automatically - so we may push on upgrading mysql sooner than later.

Today I'm actually lost in mundane bureaucracy land. I also should be working on the new software radar blanking embedder code. Sigh.

- Matt

Might give the science status page a swift kick. Seems to be stuck.



Specifically, the "signals already found" stats AND the "overflow rate" have been stuck for over a week!
.

Hello, from Albany, CA!...
ID: 811857 · Report as offensive
Profile speedimic
Volunteer tester
Avatar

Send message
Joined: 28 Sep 02
Posts: 362
Credit: 16,590,653
RAC: 0
Germany
Message 811904 - Posted: 25 Sep 2008, 17:12:51 UTC

Talking about the science status page, there is something strange left of the google skymap:

"SDSS" left of "Google"




mic.


ID: 811904 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 811924 - Posted: 25 Sep 2008, 19:12:47 UTC - in response to Message 811904.  

"SDSS" left of "Google"


Yeah.. that's saying the data is coming from the Sloan Digital Sky Survey. The Google API is kinda mysterious, and the documentation even more so - I have no idea how to remove that string (or even place it in a less clumsy spot).

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 811924 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30651
Credit: 53,134,872
RAC: 32
United States
Message 812524 - Posted: 27 Sep 2008, 1:48:12 UTC - in response to Message 811699.  

Something we've been lagging on is separating the database count totals on the server status page. Currently we're showing "totals" - for example, the "results ready to send" is a sum of both SETI@home and Astropulse results ready to send. For diagnostic purposes, it would be much better to split these into two separate columns.

However, this isn't so easy, as such queries become suddenly very expensive if we're adding an additional "where appid = N" conditional (AstroPulse and SETI@home are considered two different "applications" in the BOINC realm). I'm talking the difference being a 3 second query versus a 3600 second query. Yup. We've made joint indexes in the past for servers that needed them, but this hasn't been a priority for diagnostic stuff. We also don't really have the memory/resources to keep such extra indexes around. In any case, Bob pointed out that newer versions of mysql are smarter - doing the index joins automatically - so we may push on upgrading mysql sooner than later.

Today I'm actually lost in mundane bureaucracy land. I also should be working on the new software radar blanking embedder code. Sigh.

- Matt

Might give the science status page a swift kick. Seems to be stuck.



Seems to be working again, so thanks.


ID: 812524 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 813174 - Posted: 29 Sep 2008, 15:14:16 UTC - in response to Message 812524.  


Might give the science status page a swift kick. Seems to be stuck.



Seems to be working again, so thanks.



The overflow rate is still stuck at 5%, though...
.

Hello, from Albany, CA!...
ID: 813174 · Report as offensive

Message boards : Technical News : Splitting Totals (Sep 24 2008)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.