Message boards :
SETI@home Science :
Kerned density estimation instead of histograms
Message board moderation
Author | Message |
---|---|
Fabrizio Benedetti Send message Joined: 31 Jul 99 Posts: 4 Credit: 1,892,177 RAC: 0 |
Hello I have a question (or a suggestion) for you. In the window of seti software it is possible to see an histogram. This is for the search of a particular signal with gaussian shape (I suppose...). Why you don't use the Kernel Density Estimation (KDE) to do that? The KDE is a powerful statitical method similar to the histogram but: - it doesn't need the bins and the bin size - it doesn't have the bin position problem (I mean where you have to start your binning) - it converge fastly than a histogram to the real probability density function Tnx for the attention and sorry for the poor english. Bye |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
I can't reply for the project, but your suggestion is like many good ones I've seen on these message boards. It is very likely there aren't enough staff to consider each such suggestion, in light of the other priorities of the project. I wonder, then, wouldn't it be good to have a snapshot of the science database available to us for download from time to time. Then people with sufficient interest could process it any way we like. If something really keen pops up, the results could be given back to the project. Making the snapshots available wouldn't take too much effort, because they are essentially back-up copies I suppose. There would be a bandwidth demand, but very few people might be interested in getting a copy. Even so, downloads could possibly be limited to be during the weekly maintenance period. It might make sense to try something like this for a while to see if something results! A sort of distributed computing at a higher level. |
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
... seti software it is possible to see an histogram. This is for the search of a particular signal with gaussian shape (I suppose...). Why you don't use the Kernel Density Estimation (KDE) to do that? The KDE is a powerful statitical method similar to the histogram but: How is that more useful for the signal analysis than the present code? (The present bins are assumed to be narrow enough to find any wanted gaussians.) Does the KDE discriminate better than fixed histogram bins to more accurately find gaussians that are noisy and still reject random noise that might look like a noisy gaussian? Is the KDE easy to compute? Thanks, could be interesting?... Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Fabrizio Benedetti Send message Joined: 31 Jul 99 Posts: 4 Credit: 1,892,177 RAC: 0 |
Unfortunately I cannot reply to all of your question here, also because it is necessary a lot of math (and some steps are still unclear also for me). What can I do is start to partially reply to your first question, for that I suggest to see the example that is show here: http://school.maths.uwa.edu.au/~duongt/seminars/intro2kde/
Does an histogram do that? In general the calculation of the KDE is more intensive but more accurate and approximate better the true probability density function (look here for more example and discussion: www-hermes.desy.de/notes/pub/TALK/sgliske.tpsh09.pdf ). An interest example wrote in python could be found here: http://jpktd.blogspot.com/2009/03/using-gaussian-kernel-density.html it is really similar to the code that i use to show and fit my data. |
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
The issue is whether a more accurate probability density function gives any advantage for the s@h analysis and search over what is already being done. I suspect that whatever additional accuracy might be gained using KDE, that better accuracy is then wasted when the thresholding is automatically adjusted to allow for the noise floor for each WU. Sorry, but there will have to be some clear and big advantage over the present system for there to be any interest in making a new type of analysis and search. Regardless, it's always good to look at new ideas! Keep searchin', Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.