What to do with known invalid data?

Message boards : Science (non-SETI) : What to do with known invalid data?
Message board moderation

To post messages, you must log in.


Send message
Joined: 5 Mar 02
Posts: 1
Credit: 4,984,552
RAC: 4,514
United States
Message 1759357 - Posted: 26 Jan 2016, 4:43:14 UTC

Hello to all! I have been running a solar battery charging system at my house for several months now. I am using a Raspberry Pi with a 4 channel A/D converter to keep track of my watt hours, amperage and voltage. I am also a ham radio operator and I have discovered that when I am transmitting on certain bands the long wires to the solar panels pick up stray RF. This causes very strange off-scale spikes in my data. I have been working on better RF shielding without much success. My questions is, how best to deal with the known invalid data. Do I delete the data points in question? Do I average my values to smooth it out? Do I interpolate my data based on the data points before and after the spike? I am hoping that someone in the science data field can help as I want as accurate of data as possible. I am also keeping a running total of watt hours and I know that the spikes have falsely added to that total as well. Any suggestions would be greatly appreciated.
ID: 1759357 · Report as offensive
rob smith Special Project $250 donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 16228
Credit: 319,291,378
RAC: 235,638
United Kingdom
Message 1759374 - Posted: 26 Jan 2016, 6:09:17 UTC

It all depends on the amount of RFI getting into your logging system.
If it is just an odd point, then ignoring it is probably the simplest.
If there is a lot then you need to look at "back filling" or averaging. The best solution is to filter out the RF before it gets into the loggers - s very simple low pass filter should work as you are measuring "DC" compared to the Mega Herz of your RF signals.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1759374 · Report as offensive

Message boards : Science (non-SETI) : What to do with known invalid data?

©2018 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.