For the last couple of months Eric and Jeff have been occupied with shrinking our server complex from ~20 machines down to 2, and we haven't made much progress on Nebula or science. The downsizing is now finished, and we're ramping back up on Nebula.

Some items:

• We're working on two papers - one about the front end, one about the back end and science results. I've been doing a lot of work on these, and they're shaping up nicely.
  
• Until recently, all my Nebula runs have used a database dump from 2016. We recently did a final database dump, which of course is a lot bigger since it includes 4 more years of front-end results. I started a Nebula run using this data. Everything takes a lot longer; I need to do some performance tweaks.
  
• One of these tweaks will be using a larger computer in the Atlas cluster. Currently I'm using one with 6 CPUs; I plan to switch to one with 96. Many of the time-consuming parts of Nebula (RFI removal, sorting) are parallel, and will benefit.
  

Good stuff and thanks for the update.

And very good to consolidate from supporting ~20 machines to instead move to a much more maintainable 2! That's an impressively efficient resize :-)


Sorry if far too obvious a question:

Is there good and faster mileage to be had in using a representative subset of the data?

Then quickly test ideas, rather than to lose time trying to optimize ideas that may well later change again?...


Good luck!

Keep searchin',
Martin

---

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)

Thank you for keeping us updated.

---