They are using the SSD as a buffer store before writing to the actual store, this was a little side line in one of Steve's videos

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Money is always important but i can ask Eric if there are special needs in short term.

Yes, IMO what the project needs are more people with green stars

+1

.

I believe the last tidbit I heard was a couple of servers with 24 SSDs. Probably much like what they have shown in that Greenbank video. Which brings that up, all that receiving equipment of that precision is a lot of dollars and probably where a lot of money goes for new installs like that.

What "Greenbank video"? I must have missed that. I'm willing to further donate to the project but it would be nice to know just what hardware they are most in need of to keep the project running with less interruptions or hands-on maintenance that requires scarce personnel intervention.

. . Hi Keith

. . The info/link notice had been plastered on the notices page for a month or more. It was called "How we get our data" or something like that and linked to a video about the new installation of water cooling to their racks of data harvesters at the GBT. Impressive racks ....

. . And yes, hardware updates that reduce the extent of human intervention in the running of the whole shebbang as well as increasing capacity would be the optimal solution. Since, apart from the dollars required (at least 6 figures per person per annum I am guessing), the biggest problem in gaining more staff would be to convince the powers that be that they are needed and justified.

Stephen

:)

I believe the last tidbit I heard was a couple of servers with 24 SSDs.

Yes, there will apparently be a requirement soon for a new storage multi-drive bay + compute capability for new data sources. The specs are still being worked out.

I suspect that this will be required for bringing Parkes online as Breakthrough Listen data source.

. . There you go ...

. . All you Aussies out there, time to break into your piggy banks and help get Parkes online in this shindig.

Stephen

:)

Lots of thoughts/opinions to add here. So many it will take some time to form a proper response. Probably most important in my eyes is that the Universities, as degree mills, are probably the least appropriate venue for anything worthwhile.

. . Actually, Universities have been and probably still are some of the most productive think tanks around. Whatever budgeting/management issues may trouble them that is one of their more important functions. And there is the other side to that issue, if not there then where else?

Stephen

??

I would think it unlikely that us as volunteers could would fund an extra member of Staff. Eric said many years ago that the only realistic way was to set up a rolling Bursary for the person, we were talking a million $ + !!

. . With over 100,000 of us $10 USD per capita would take care of that, but most of us know less than 1% would even consider contributing cash ... :( So if everyone would consider it worthwhile buying Eric a cup of coffee a couple of times a year then the project could have the funds to do something like that. But for those who do/might contribute it would take over $1,000 USD each ... yikes. That is more than I am planning to spend on my new computer.

The answer is for Eric, Matt, and Jeff to give us a shopping list of the most needed stuff over the next 6-12 months and we'll see what we can do.
Just me 2c worth M'lud

. . Yep, having a clearly defined objective can help motivate a lot of people.

Stephen

!!

See that they are using Seagate Red 5TB drives.

I thought it was only the WD NAS drives that had those colour codes... I would think they would use WDs as well as Seagates aren't as reliable.

Seems they are actually using Toshiba drives. http://i.imgur.com/JCdsbE3.png

. . LOL we have a winner ... :)

Stephen

:)

Exactly right. A general fundraiser turns people off because they might think that their $10 goes in overheads or similar. But a list of physical equipment is much more personal. People will put their hand in their pocket for an SSD drive, feeling that "their" drive is maybe helping to crunch "their" data. It's the feel good factor.

This is exactly correct for my donations, but rather my reason is expansion into new areas/data sources or growing the project's capabilities. I'll donate hundreds or more if there is specific hardware required for a given reason, but just the minimum to keep the star on otherwise, ie that drive could be part of an array so that we have a larger work unit pool available or a backup to ensure that our results are not lost, or to store/transport results from a different telescope. Thus why I keep asking for the "shopping list".

---

My green star has expired, and is awaiting a targeted fund raiser for its re-ignition.

---

Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

What SETI really needs to succeed, above and beyond faster hardware, is better algorithms. That's why I posted this https://setiathome.berkeley.edu/forum_thread.php?id=81433. But after lots of views, it's very clear that either or both of the following are true: (1) people realize they have little influence over how SETI designs its pipelines or (2) I've totally failed to get the message across.

I've spent the better part of the past 3 years working on combinatoric methods to accelerate signal detection. These are very sensitive, generic methods which can detect tiny statistical biases at thresholds below those of analog methods such as standard deviation.

Based on quantitative tests with SETI's own data, they appear to be quite effective in practice. It's hard to say exactly how much benefit they could deliver here, due to the complexity of the SETI pipeline, but no one seems to be taking a serious look at this at all.

For one thing, I don't think that we need to remove RFI. That just weakens numerical precision. We can detect anomalous signals with RFI still present. This is demonstrated in the videos, although the SETI problem would probably require a multidimensional bandgap version of that, which is still computationally cheap in relative terms. All that hard analysis should be reserved for the very most unusual signals which actually deserve intense scrutiny.

But look, I do understand this much: If it doesn't work, then all this is just a big distraction. And if it does work, then someone actually needs to code it, which means more man hours that SETI can't easily supply. But the algorithms I'm proposing are all asymptotically O(N). As in, they can run many times faster than Fourier transforms (10X to 100X, typically), while maintaining comparable signal sensitivity in practice.

I'd be happy to provide more details, but all I really want to know at this point is: What am I missing here? How am I failing to communicate?

I'd be happy to provide more details, but all I really want to know at this point is: What am I missing here? How am I failing to communicate?

Your post was day ago and in the area that hardly looked in by staff on daily basis.
If you want to pass message to staff - E-mail/PM them directly.
If you want response from volunteer community - be patient cause volunteer here means quite busy elsewhere to give immediate responses.

---

SETI apps news
We're not gonna fight them. We're gonna transcend them.

What SETI really needs to succeed, above and beyond faster hardware, is better algorithms.

Raistmer is one of the people working on those better algorithms on the client (our) side to improve our number-crunching performance. Petri33 is another one. (Edit: implementation of them at least.)

As far as the actual algorithms used by the project such as the Gaussian fit, chirp, autocorrelation etc. as noted you should contact the SETI@Home team directly. They post in the News threads occasionally but have never seen them participate in the forums elsewhere. They can be found on this page.

---

Raistmer and Mr. Kevvy, for starters thanks for your replies, which I think are reasonable. Let me provide a few more details in that vein.

So first of all, I should emphasize how important the community is, even with the respect to the question of software methods used by SETI. We all know that SETI staff are busting their butts every day to keep all this online. They don't necessarily have time to wander off and investigate every cool new algo that lands on the internet, just as much as they can't be bothered to deal with every GPU driver bug. As you obviously know, the community contains some extraordinarily smart people which SETI can lean on for those purposes. Perhaps most importantly, people are involved because they believe in the SETI enterprise. Even if it all fails, the engineering and algorithmic approaches will create benefits for other areas of science.

To your other point, I first brought this to staff (i.e. not forum) attention last year. The main response out of that was that I should try the methods (entropy transforms) on real SETI data. The staff member kindly provided me with a link to a data store. (Up until that point, I had been exclusively using my own concocted data, so this was an appropriate recommendation.) It took me a month, but I managed to get hold of real data, decode it according to the specification, and run entropy transforms against it. The result was basically as presented in the video: in the worst case, entropy is about a (1/3) as sensitive as optimized Fourier methods. But as I explained, this (1/3) only applies in the region between 800 and 2400 samples, give or take. Furthermore, from my reading of the Nebula documentation, it sounds like 8000 samples is the minimum region of inspection, so in fact entropy transforms would be equally sensitive to Fourier. But actually, entropy would be more sensitive if in fact ET communicates in entropy modulation instead of sinusoid modulation. (That's a theoretical debate, to be sure, but in any event I'd rather be able to "see" entropy modulation signals than not.) I did submit my early results directly to staff, but never heard back. To be fair, it was made very clear to me that this particular staff member had no spare time. Which brings us back to the importance of the community in these matters.

If in the end all my efforts just result in someone else coming up with an even better idea, which speeds up SETI even more, then I'll be very pleased indeed.

Well, cardinal change in algorithm used (and switching off from FFT sounds like such) will definitely require separate application version cause results (even if they would be "better" in sencivity) will not validate versus currently obtaining results.
That is, such change isn't optimization. It's more like new SETI project/search.
Currently we on final stage of finishing this SETI@home experiment. So any changes almost certainly will be postponed for next one.
But don't forget that there are other SETi experiments active as well. Most noticeable and open-minded currently - Breakthrough Listen initiative.
They call for new data processing methods directly. Worth to present your approach there.
Link: https://breakthroughinitiatives.org/News/10

The BSRC team continues to develop more and more sophisticated algorithms, and plans to release updated analyses of Listen data approximately once every six months. The Berkeley team also seeks continued and expanded engagement with signal processing and machine learning experts to help mine the Breakthrough Listen dataset in the search for evidence of signals from extraterrestrial civilizations.

---

SETI apps news
We're not gonna fight them. We're gonna transcend them.

I agree that any new search algorithm would be best implemented on new datasets coming from Green Bank and Parkes. One obvious advantage is the greatly improved local noise environment for observations compared to Arecibo. At least you don't have to deal with all the radar bursts contaminating your data constantly. But if I understand the proposed algorithm adequately, that is not of much concern.

---

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)

@Raistmer

I couldn't agree more. In fact, I actually quoted that exact sentence about "seeking engagement" in an email to them earlier this morning. I wouldn't hesitate to get on a plane and fly out there if I thought it would help.

Secondly, I agree: we don't need to mess up the current SETI experiment, just for the sake of optimizing the tail end of it. So yes, let's think in terms of the next generation.

@Keith Meyers

Well, you raise an interesting point. All else being equal, of course I'd prefer to have no noise at all. (This is why I have also reached out to optical (laser) SETI; they are an ideal case!) But that said, my experiments quantitatively suggest that entropy transforms really shine under crappy SNR conditions. That is, while they do better in absolute terms in a quiet environment, they are more competitive versus conventional approaches when the noise kicks up.

This is the main reason why entropy transforms would be especially helpful if in fact ETs talk to each other using some form of entropy modulation, instead of sinusoid modulation. (To make this crystal clear, when I say "entropy modulation", I'm referring to the method of communicating whereby zero and one are identified by opposing perturbations to the frequency distribution of Gaussian noise. For example, a tall narrow Gaussian means "zero" and a wide low Gaussian means "one". Thus, EM is an analog concept just like sinusoids; bits are then modulated on top of them. In theory, EM should have environmental durability superior to sinusoid modulation, so I would an expect an advanced civilization to have mastered its implementation. Fourier methods would be very hard pressed to detect it.)

@everyone

Psst... see waveatom.org . I have nothing to do with wave atoms, but I know enough to understand that the mathematicians behind them understood enough about physics to know that real world phenomena are often sparser when expressed in wave atoms instead of Fourier sinusoids. This most likely means that you would learn more from wave atom transforms than Fourier transforms. So in my nutty world, SETI 2.0 has an entropy transform frontend which deletes most of the data, and a wave atom backend which gives some serious attention to the oddball signals.

What would be feasible is what the GPU Users Group used to do, as in post a list of hardware and software urgently needed e.g. so many SSD's at so much each, the housing for them etc. People WILL buy a drive. Whether we could raise enough for a new server I don't know, that would be in the $18-20K range for a top spec one. Extra memory for existing servers could be feasible, as could software upgrades.

Our funds transfer bottleneck was just cleared so all is well and we can start new fundraisers. I've let them know so I'm hoping to see a "shopping list" in the near future. We'll probably have to wait on the big drive bay but we can certainly start a few smaller ones ie for replacement drives in the interim.

---

Reliable, steady and increasing funding in support of staff, researchers, as well as hardware maintenance, and upgrades are probably the most important things that SETI and SETI@home need.

As well as continued support from the volunteer community.

Tom

---

A proud member of the OFA (Old Farts Association).

***Your thoughts/opinions on what SETI@home needs to succeed***

A reply from ET would not go a miss...:-)

***Your thoughts/opinions on what SETI@home needs to succeed***

A reply from ET would not go a miss...:-)

*Amen* brother(?)!

---