Marching on... (March 4, 2015)

Message boards : Technical News : Marching on... (March 4, 2015)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3

AuthorMessage
Bill Butler
Avatar

Send message
Joined: 26 Aug 03
Posts: 101
Credit: 4,270,697
RAC: 0
United States
Message 1651464 - Posted: 10 Mar 2015, 22:40:58 UTC - in response to Message 1651273.  

You know how every MB has spikes, triplets, pulses, gaussians, and auto-correlation counts? Every single one of those that gets found in every single WU.. becomes a new row in the respective tables. Based on what I'm seeing on my one cruncher, we've passed 4 billion tasks, and are nearing 1.75 billion WUs. Considering MB has five kinds of signals, and can have 30(?) of each, that adds up to a massive number of rows in the DB.

A technicality about this has bothered me. You remember that it has just been recently that Autocorrelation has been implemented as a parameter and added to the crunching regimen?

I was hoping Autocorr counts would show up in the Master Science Database table display on the Science Status Page. In that table you see the counts of Spikes, Gaussians, Pulses, Triplets. Each count, as noted, presumably a row in the table.

However, ever since the Autocorr parameter has been implemented, that count has never been added to the display.

I assume that the Autocorrelation counts are being added to the MSDb. And that no one on staff has had the time to program a change to add this count to the display. But I have no confirmation. Anyway, it would be nice to see the Autocorr counts compared to the other signal counts.
"It is often darkest just before it turns completely black."
ID: 1651464 · Report as offensive
Grumpy Swede
Volunteer tester
Avatar

Send message
Joined: 1 Nov 08
Posts: 8549
Credit: 49,849,242
RAC: 65
Sweden
Message 1651467 - Posted: 10 Mar 2015, 22:52:51 UTC - in response to Message 1651464.  
Last modified: 10 Mar 2015, 22:53:09 UTC

You know how every MB has spikes, triplets, pulses, gaussians, and auto-correlation counts? Every single one of those that gets found in every single WU.. becomes a new row in the respective tables. Based on what I'm seeing on my one cruncher, we've passed 4 billion tasks, and are nearing 1.75 billion WUs. Considering MB has five kinds of signals, and can have 30(?) of each, that adds up to a massive number of rows in the DB.

A technicality about this has bothered me. You remember that it has just been recently that Autocorrelation has been implemented as a parameter and added to the crunching regimen?

I was hoping Autocorr counts would show up in the Master Science Database table display on the Science Status Page. In that table you see the counts of Spikes, Gaussians, Pulses, Triplets. Each count, as noted, presumably a row in the table.

However, ever since the Autocorr parameter has been implemented, that count has never been added to the display.

I assume that the Autocorrelation counts are being added to the MSDb. And that no one on staff has had the time to program a change to add this count to the display. But I have no confirmation. Anyway, it would be nice to see the Autocorr counts compared to the other signal counts.

The data on the Science Status Page, hasn't updated since: 4 Nov 2014, 23:30:05 UTC
As can be seen under the "Note: This page is still vastly under construction. More features are being developed. Read more about our future plans."
ID: 1651467 · Report as offensive
Profile Julie
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 28 Oct 09
Posts: 33930
Credit: 18,883,157
RAC: 18
Belgium
Message 1651473 - Posted: 10 Mar 2015, 23:01:53 UTC - in response to Message 1651467.  
Last modified: 10 Mar 2015, 23:03:13 UTC

You know how every MB has spikes, triplets, pulses, gaussians, and auto-correlation counts? Every single one of those that gets found in every single WU.. becomes a new row in the respective tables. Based on what I'm seeing on my one cruncher, we've passed 4 billion tasks, and are nearing 1.75 billion WUs. Considering MB has five kinds of signals, and can have 30(?) of each, that adds up to a massive number of rows in the DB.

A technicality about this has bothered me. You remember that it has just been recently that Autocorrelation has been implemented as a parameter and added to the crunching regimen?

I was hoping Autocorr counts would show up in the Master Science Database table display on the Science Status Page. In that table you see the counts of Spikes, Gaussians, Pulses, Triplets. Each count, as noted, presumably a row in the table.

However, ever since the Autocorr parameter has been implemented, that count has never been added to the display.

I assume that the Autocorrelation counts are being added to the MSDb. And that no one on staff has had the time to program a change to add this count to the display. But I have no confirmation. Anyway, it would be nice to see the Autocorr counts compared to the other signal counts.

The data on the Science Status Page, hasn't updated since: 4 Nov 2014, 23:30:05 UTC
As can be seen under the "Note: This page is still vastly under construction. More features are being developed. Read more about our future plans."


Patience is a virtue. Glad we have the updates, myself.
rOZZ
Music
Pictures
ID: 1651473 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 1651549 - Posted: 11 Mar 2015, 5:39:03 UTC - in response to Message 1651373.  

No, there can only be 30 signals total, in any combination of the 5 types. It can be 30 spikes, or 15 spikes and 15 pulses, or 22-2-2-2-2, but not 30-30-30-30-30.

Did not know that one. I always wondered what the actual signal count limit was for MB. Actually, I thought it had to do with the size of the result file (and I feel like I remember the limit being 30KB), not the number of signals found.

Interesting to know, then.

Which just means.. if we've had 1.75 billion WUs, and just for the sake of napkin-math we say that the average is 10 signals per WU, that makes 17.5 billion rows in total.

What I DO know about DBs is that you have to designate the size of each field, which means if it will hold text, then you would need to figure out what the absolute longest string of text will ever be for that field before you even start, or just let it go with the default, which is usually 256 or 512 characters, which also means bytes.

If we assume there are say..5 text fields and 10 numerical fields.. both being 32-bit values, whether float or integer (both are 4 bytes.. but I remember recently some integer values had to be upgraded to 64-bit, so we'll say they're all 8 bytes now), that makes 2640 bytes per row, no matter what data is actually in each field. 2640 bytes per row * 17.5 billion rows = 46.2 TB if it is not compressed.

I don't care what the DB engine is.. having to read through that many rows or bytes is an immense task.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 1651549 · Report as offensive
Darth Beaver Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 20 Aug 99
Posts: 6728
Credit: 21,443,075
RAC: 3
Australia
Message 1651559 - Posted: 11 Mar 2015, 6:50:34 UTC

Thanks Matt for the in depth news .Sounds like I've made the right desition . No seti and Just do Crypto on the GPU . 3 months wow should make a bit in that time so I can donate it to the course , which sounds like your going to need with the gremlins really having a right old feed on all the hardware
ID: 1651559 · Report as offensive
Profile Uli
Volunteer tester
Avatar

Send message
Joined: 6 Feb 00
Posts: 10918
Credit: 5,996,015
RAC: 1
Germany
Message 1651933 - Posted: 12 Mar 2015, 3:28:44 UTC

Thanks for the update Matt.
Pluto will always be a planet to me.

Seti Ambassador
Not to late to order an Anni Shirt
ID: 1651933 · Report as offensive
Jim Herod
Volunteer tester

Send message
Joined: 23 Apr 99
Posts: 21
Credit: 12,409,544
RAC: 4
United States
Message 1656191 - Posted: 24 Mar 2015, 0:52:49 UTC

Regarding disc drives and drive life...

You might want to take a look at these:

https://www.backblaze.com/blog/how-long-do-disk-drives-last/
https://www.backblaze.com/blog/what-hard-drive-should-i-buy/

The second link should be accompanied by:

http://www.theregister.co.uk/2014/02/17/backblaze_how_not_to_evaluate_disk_reliability/

as a caution about doing good statistics.

--Jim
ID: 1656191 · Report as offensive
Previous · 1 · 2 · 3

Message boards : Technical News : Marching on... (March 4, 2015)


 
©2021 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.