Message boards :
Technical News :
Shadow (Feb 11 2009)
Message board moderation
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
Before releasing the astropulse application Eric had to add a couple fields to the result tables in the science database that are now necessary. These are large fields, and it's taking informix forever to update the table. The job was started 24 hours ago and is still chugging along. I guess it doesn't help that the assimilator queue is still rather large (though it is draining). So the release is delayed until this job finishes. The radar blanking stuff I was whining about the other day has nothing to do with the astropulse release, in case there was some confusion about that. Josh and I are working on two completely separate and different forms of radar mitigation. Mine is to better clean up data before any splitting/analysis, Josh's is to deal with radar that squeaked through the first pass and made it all the way to the client. The good news is that I made significant progress on mine today. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
perryjay Send message Joined: 20 Aug 02 Posts: 3377 Credit: 20,676,751 RAC: 0 |
Thanks for the update Matt. Gives the optimizing crew a little more time to get ready for the new APs. :) PROUD MEMBER OF Team Starfire World BOINC |
Raistmer Send message Joined: 16 Jun 01 Posts: 6325 Credit: 106,370,077 RAC: 121 |
Before releasing the astropulse application Eric had to add a couple fields to the result tables in the science database that are now necessary. These are large fields, and it's taking informix forever to update the table. The job was started 24 hours ago and is still chugging along. I guess it doesn't help that the assimilator queue is still rather large (though it is draining). So the release is delayed until this job finishes. Do you use any data substitution on server-side data cleanup (I mean, do you insert something in data array instead of removed radar pulses ? ) If yes, is it possible to mark these "blanked" areas in task header so client could know what areas are "true" ones and what contain some substituted stub data? Some list of indexes of blanked regions for example... |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
. . . Congratulations Matt on the 'significant progress' - Good to hear < Accolades to Each of You @ Berkeley - Your Hard Work is Noted . . . BOINC Wiki . . . Science Status Page . . . |
speedimic Send message Joined: 28 Sep 02 Posts: 362 Credit: 16,590,653 RAC: 0 |
...so the client could skip those? mic. |
ML1 Send message Joined: 25 Nov 01 Posts: 20265 Credit: 7,508,002 RAC: 20 |
The radar blanking stuff... What angle are you going for? My scraps of understanding from previous posts are that there is a radar detector on the telescope at Arecibo. The output from that is recorded on a spare data recorder channel in parallel to the 14 streams of radio data. Matt's code then substitutes psuedo-random random noise into the data stream to replace the radar mess whenever the radar has been detected. The psuedo-random noise then goes innocently through the rest of the data processing as though Arecibo hasn't heard anything other than the usual background noise. The radar source is a nearby military radar. The radar is kindly blanked for the arc sweeping Arecibo. However, that doesn't stop reflections that bounce off aircraft (or other local structures?) splattering Arecibo... Perhaps: Also set aside an air exclusion zone around Arecibo? Change the radar frequency to something much higher than the Arecibo receivers? (Or will the radar ERP saturate anything more sensitive than an AC power rectifier diode?!) An interesting question is whether Matt has had to add any cleverness to avoid a signal spike being detected by the FFT for the transition to/from the psuedo-random noise section...? I would expect the FFT to "see" the transition in the noise "texture". Happy crunchin', Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
Can anyone comment on the mathematical correctness of adding pseudorandom noise? It seems to me that doing so would add pseudo-energy to the broadband power spectrum. This has the effect of reducing S/N a bit. For example, if 50% of the time slice had radar noise and it was replaced with random noise, then 50% of the power spectrum, across frequencies, would be artificial. That just doesn't sound right. Wouldn't just replacing the time with a constant, or if processed after the fact replace the radar blank period with the signal average for the time slice, then this would only add noise at the DC level and then would be ignored. I'm sure Matt and others have thought about this, so I'm just looking to understand the analysis a bit. |
Kurt R. Radtke Send message Joined: 30 Mar 00 Posts: 4 Credit: 51,481 RAC: 0 |
Matt, Thanks for the postings here. It's nice to know there is a human or two somewhere in the mix watching over, updating and maintaining things, and it's also nice to have someone crack the door so we can peek in. Thank you. Kurt R. Radtke SETI@Home; Founder of Team Macintosh |
Paul D. Buck Send message Joined: 19 Jul 00 Posts: 3898 Credit: 1,158,042 RAC: 0 |
Change the radar frequency to something much higher than the Arecibo receivers? (Or will the radar ERP saturate anything more sensitive than an AC power rectifier diode?!) If it is a ground based radar it likely will saturate even the power diode. Heck, when I worked on OTH-B ... when we radiated on a frequency no one in the world could use that slot ... the good news is that it was low frequency so most people could not care less ... but it was funny working with the Radar Control guys ... we would suggest a frequency band and they would find a clear spot and then "park" the radar there with short scans before changing the whole system over ... they kinda played nice by reserving the space before stepping on anyone that had had an idea of coming up on that spot ... |
Raistmer Send message Joined: 16 Jun 01 Posts: 6325 Credit: 106,370,077 RAC: 121 |
Well, short question is: Is it possible to include info what areas of data was changed by blanking algorithm into WU header ? |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
I think they archive all raw data. If so the software blanking could in principle be modified in the future and the data re-processed. |
Raistmer Send message Joined: 16 Jun 01 Posts: 6325 Credit: 106,370,077 RAC: 121 |
I think they archive all raw data. If so the software blanking could in principle be modified in the future and the data re-processed. Too much unneeded work to rely on this archived datas. In general, if there is some way not to do smth, better use it as soon as possible ;) |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.