Message boards :
SETI@home Staff Blog :
Eric's Weekly Post #3 (Warning, science enclosed. Do not fold or bend.)
Message board moderation
Author | Message |
---|---|
Eric Korpela Send message Joined: 3 Apr 99 Posts: 1382 Credit: 54,506,847 RAC: 60 |
OK, so I missed a week. Nobody's perfect. This week, I've been busting my brain about some interference we are seeing in the multibeam data recorder. Anyone who runs the beta may have noticed that both Astropulse and SETI@home multibeam work units have been hitting overflow conditions fairly regularly. Regularly enough that a lot of machines were hitting their 100 workunit per day limit. So I set about the task of figuring out what was going on. I'll ask you to forgive me in advance because this might get a bit technical... I grabbed one of the recorder data files that showed this effect and did a little spectral analysis to see if I could figure something out. This is what I saw. This is what we call a waterfall plot of about 0.42 seconds of data from one of the data recorder channels. The vertical axis is frequency difference in MHz from 1.420 GHz. The horizontal axis is time. Generally it looks OK. The darkish stripe at zero frequency is due to a high-pass filter that cuts out any constant voltage offset in the system. The gradual darkening at the top and bottom are the high and low frequency edges of our bandpass filter. You might notice a slightly brighter stripe at about 0.5 MHz that is due to the Hydrogen line that Kevin spends his time making maps of. What I didn't expect to see is the 7 or so vertical lines that cross zero frequency. That's our problem. Pulsed interference of some sort. From the width I can see that the pulses have time scales of 20 to 50 microseconds or so. They aren't entirely regularly spaced, but they are close enough and strong enough to register as pulses in astropulse and as triplets in SETI@home. I looked at all of the receivers and I saw something similar in each. The next thing to check was whether the interference appeared at the same time in all the bands. So I did a complex sum of all of the channels to see if the signals were coherent, meaning they have the same phase and timing in all the channels. Spectral analysis of the result showed this: That convinced me that there is some source of interference that affects all of the channels in nearly the same way. It's also more common than you would think if you just looked at one channel. So I did the obvious thing and subtracted the summed waveform from the channel to see if that would remove the pulses. It did. Perfectly. That is, until I tried to turn the subtracted channel into a work unit. In order to do that, I needed to turn my floating point data back into the 1 bit samples. That restored the pulses. It took a little work to figure out why. I was subtracting a waveform with values between -1 and 1 from values that were either -1 or 1. Then I would take the result and convert it to -1 or 1 depending upon whether it was less than zero. So if I subtracted 1 from 1, I would get zero, which is not less than zero so the result would be 1. 1-1=1. Josh is exploring ways to fix this. The one I'm currently working with is: if the result of the subtraction is between -0.5 and 0.5, then assign a random value. So far it seems to work pretty well, I'll be testing it out in beta next week. Another option would be to use 2 bit samples in the work units. That also appears to work well, but it doubles the size of the workunit files. So we'll try the randomizer first. Next we'll try to figure out where these pulses are coming from. Since they are coherent across channels and polarizations I suspect that that are somewhere in the electronics chain after the mixer. Maybe even in the data recorder itself. We'll be talking with engineers at Arecibo in order to find out if we can get rid of them at the source. Dan suggested this may be the first time that an RFI waveform has been removed from a channel by using multibeam data. I doubt it, but it's certainly the first time we've done it, and that's good enough for me. -- Eric @SETIEric@qoto.org (Mastodon) |
Clyde C. Phillips, III Send message Joined: 2 Aug 00 Posts: 1851 Credit: 5,955,047 RAC: 0 |
It looks like something's happening about 100 times per second, but the frequency varies a total of about 2:1. |
Josef W. Segur Send message Joined: 30 Oct 99 Posts: 4504 Credit: 1,414,761 RAC: 0 |
OK, so I missed a week. Nobody's perfect. 'Tis a gem of a post, well worth waiting an extra week. The second image link has an extra term, this works better. Joe |
ML1 Send message Joined: 25 Nov 01 Posts: 20265 Credit: 7,508,002 RAC: 20 |
OK, so an outsider's completely wild guess: Those FFTs look like the pulses are rounded, and they are symmetrically centred mid-band, so I'd suspect first signal wiring pickup or voltage rail disturbance near/on the ADC(s) in the data recorder, or pickup on the way to the datarecorder. Do those pulses match HDD activity?! A second wild shot is if the datarecorder CPU is overloaded and is missing picking up or responding to a proportion of its interrupts. Thanks for the detail. This looks set to be a good science story. Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Byron Leigh Hatch @ team Carl Sagan Send message Joined: 5 Jul 99 Posts: 4548 Credit: 35,667,570 RAC: 4 |
Hi Eric , thank you very much for your post Best Wishes Byron :-) |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
@ Eric . . . what a Wonderful Portrait of a Day in the Life. > it's really worth being in Science, isn't it Sir? < Thank You Immensely. BOINC Wiki . . . Science Status Page . . . |
Byron Leigh Hatch @ team Carl Sagan Send message Joined: 5 Jul 99 Posts: 4548 Credit: 35,667,570 RAC: 4 |
Eric , the "normal" Multibeam WU in beta testing usually on average has an angle range of somewhere between 3 and 0.03 this __Multibeam - 9 result_overflow WU__ that my computer processed has a true angle range of : 135.086062 ? is there any signifcance re: true angle range of : 135.086062 ? - Byron |
Astro Send message Joined: 16 Apr 02 Posts: 8026 Credit: 600,015 RAC: 0 |
do the signals from each head travel independently to the recorders or are they multiplexed and then separated later? |
Clyde C. Phillips, III Send message Joined: 2 Aug 00 Posts: 1851 Credit: 5,955,047 RAC: 0 |
Byron, I saw someone mentioning one of these extremely long angle range units a year or so ago. On that one it appeared that the recorder was turned off for several hours, the scope then moved most of that angle and then the recorder turned back on. Probably that was the way the cookie crumbled when somebody divided the tape into workunits. There are probably 255 other units with similar ID numbers that have that long angle. |
Odysseus Send message Joined: 26 Jul 99 Posts: 1808 Credit: 6,701,347 RAC: 6 |
Byron, I saw someone mentioning one of these extremely long angle range units a year or so ago. On that one it appeared that the recorder was turned off for several hours, the scope then moved most of that angle and then the recorder turned back on. Probably that was the way the cookie crumbled when somebody divided the tape into workunits. There are probably 255 other units with similar ID numbers that have that long angle. We have been seeing batches of Multibeam WUs with extremely high ARs in Beta, often returning “–9 overflowsâ€Â; the highest I noticed was aroud 75°, with start and end RAs nearly 6 h apart. |
Clyde C. Phillips, III Send message Joined: 2 Aug 00 Posts: 1851 Credit: 5,955,047 RAC: 0 |
Somewhere, records of telescope (receiver) position at time of day were available for SetiClassic. The same might be true for Alfa. Every few seconds the position in declination and right ascension were given. But with the units with long angle there was a big discontinuity of several hours and many tens of degrees, as I remember. The receiver was probably turned off several hours during which the receiver moved that big angle. I'd have my doubts that any of those 256 discontinuous units would be any good. But to search for those long interruptions might be a bigger waste of time than to crunch 256 worthless units. |
Eric Korpela Send message Joined: 3 Apr 99 Posts: 1382 Credit: 54,506,847 RAC: 60 |
Hi Byron, The high angle range workunits are the result of a different problem. Apparently the telescope sends a bad coordinate packet on rare occasion. I'm going to fix that one by checking whether the distance between any two coordinates exceeds the maximum rate the telescope can move in that amount of time. Eric @SETIEric@qoto.org (Mastodon) |
Eric Korpela Send message Joined: 3 Apr 99 Posts: 1382 Credit: 54,506,847 RAC: 60 |
My bet is fluctuations in the ground plane or a loose connection somewhere.
That's another one we can check on. The files are temporarily stored on a RAM disk before being transferred to the HDD. We should be able to run for a period with no HDD activity.
This is probably not the case, but worth checking.... Thanks, Eric @SETIEric@qoto.org (Mastodon) |
Eric Korpela Send message Joined: 3 Apr 99 Posts: 1382 Credit: 54,506,847 RAC: 60 |
do the signals from each head travel independently to the recorders or are they multiplexed and then separated later? Each signal travels separately, although there are plenty of opportunities for crosstalk in electronic systems along the way. It's a long way from the telescope to the recorder. Eric @SETIEric@qoto.org (Mastodon) |
ML1 Send message Joined: 25 Nov 01 Posts: 20265 Credit: 7,508,002 RAC: 20 |
Each signal travels separately, although there are plenty of opportunities for crosstalk in electronic systems along the way. It's a long way from the telescope to the recorder. That and the data recorder itself must make for a good scientific paper... Another thought: Is the interference consistent throughout all the recording period or is it time of day dependant, or runs for short periods? (Mobile phones, portable phones, security radios...?) Good luck for the glitch hunt. Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Michael Send message Joined: 21 Aug 99 Posts: 4608 Credit: 7,427,891 RAC: 18 |
Have you figured out what the pulse repitition interval and the pulse duration is? Personally, my opinion is any pulse repitition less than 100usecs would or should indicate a pulse doppler radar? Or something with a fairly predictable duty cycle...it doesn't even have to be pulse constant... Is there a weather radar near by? Perhaps your picking up sidelobes or back lobes getting into the sidelobes of the receiving antenna? Sidelobe poke through or sibelobe of a separate transmitter getting into the sidelobe of the receiver? I am not actually suggesting radar, but even perhaps unshielded equipment nearby? |
ML1 Send message Joined: 25 Nov 01 Posts: 20265 Credit: 7,508,002 RAC: 20 |
Have you figured out what the pulse repitition interval and the pulse duration is? I half thought that but the spacing is more of the order of 10ms, and variable (100 pulses per second) which is more like the clicks from a mobile phone or HDD activity... There are FFT waterfall plots posted earlier in the thread. Capacitive pickup across cables or ground bounce can be... "difficult"... Happy tracing! Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Misfit Send message Joined: 21 Jun 01 Posts: 21804 Credit: 2,815,091 RAC: 0 |
Is there a weather radar near by? Perhaps your picking up sidelobes or back lobes getting into the sidelobes of the receiving antenna? Sidelobe poke through or sibelobe of a separate transmitter getting into the sidelobe of the receiver? I am not actually suggesting radar, but even perhaps unshielded equipment nearby? Many years ago very quiet night I could pick up the airport radar at Lindberg Field (or possibly the North Island Naval Station) on my CB radio. Gotta watch out for those harmonics. me@rescam.org |
littlegreenmanfrommars Send message Joined: 28 Jan 06 Posts: 1410 Credit: 934,158 RAC: 0 |
Just a thought Eric... There seems to be a "ghost" of the central group of vertical lines at about the 0.12 to 0.17 Mhz range of your graph. To me, that looks possibly like a harmonic frequency, caused usually by a signal diffracting between two objects. If so, you may need to research more than one possible source of the RFI. To illustrate my point, I once experienced the same effect in London, England, while listening to a radio station transmitting on 1.214 MHz. I could receive a weaker, but perfectly clear signal on 310 KHz, a fourth harmonic of the true signal. Of course, I could be well clear of the mark, but I thought it was worth a mention. |
ML1 Send message Joined: 25 Nov 01 Posts: 20265 Credit: 7,508,002 RAC: 20 |
Just a thought Eric... Errrr... I didn't know that "sub"-harmonics existed!... [edit] (Note: They don't! However, you can see an interferometric effect whereby the amplitude of a signal gets modulated at the difference ferquency from the modulating/interfering signal. This is different from generating a (higher frequency) harmonic from non-perfectly-sinusoidal signals or from FFT decomposition of one signal. Corrections welcomed!) [/edit] More seriously, you may well have had that (strong) signal breaking through on the mixer stage alternate frequency of one of the IF stages in your receiver. Was it a non-superhetrodyn receiver with only one (or no) IF stages even? "Ghosting" as commonly seen on TV pictures where you get multipath reflection is from the same frequency signal arriving via multiple paths. You get the same signal but displaced in time. Hence, for an analogue modulated TV picture that is scanned across the screen horizontally, you get multiple identical pictures overlayed but slightly displaced horizontally. If what you notice are indeed "ghosts" but time delayed, then that would suggest multipath pickup. The "multipath" part could be ANY different route, whether external or just different routes through the pickup path in the affected equipment. It could just be that the interference source produces that repeating pattern and that it "just looks like that". Who knows? Until Eric tracks it down. It would be interesting to see what more of the WUs show and how consistent/repeatable it is, and so on... Much better to cure the source rather than to try to remove the symptoms. (Hey, we could make a reality-TV show out of "What next!"... :-) ) Happy hunting! Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.