Results (Dec 29 2010)

Last post of the year!

I understand the SETI@home/BOINC back-end is a bit confusing. I think there are only a handful of people who understand all the relationships between every step of the BOINC finite state machine, the SETI@home data pipeline, and every server in our closet. I can only really scratch the surface of these details during these relatively pithy missives.

So let me try my best to quickly answer the general question: "did those two brand new servers (oscar and carolyn) help?"

First of all, there are about 100 known problems with our systems at any given time. Most of these are low priority and "time out" on their own. There are still a bunch of high priority issues, of which these new servers addressed *some*.

One easy problem to fix was replacing mork (the randomly crashing master mysql database server) with carolyn. This has been great... and I think we just (as of this morning) solved the current batch of disk i/o problems (by properly tweaking the write cache settings). Carolyn also has a lot more disks and memory than mork had, so there's still a lot of room to grow as needed.

The other new server, oscar, is taking care of two major problems. First, the science database on thumper was abysmally slow - so now this is on oscar. That's great, however it's not running as fast as we'd like. We still haven't fully benchmarked this, though. Maybe at worst we'll need to add more disks to the system - we shall see. The second major problem oscar is helping to resolve is ptolemy - our internal administrative file server - which, like mork, also randomly crashes from time to time locking everything up. Now that thumper is off the hook as a science database server, it can take over and easily handle ptolemy's current functionality, and then we can retire ptolemy. There's actually a third minor problem that's also getting fixed: thumper's root partition is on a messed-up software RAID device which kinda works but has been scaring us for way too long. It'll be great to have this server-shuffle opportunity to have a quiet moment to reinstall the OS on thumper and fix that RAID.

But that's pretty much it for now. Other major problems exist with no clear solutions. In fact, many of them are data driven, or network infrastucture driven, and therefore out of our sysadmin hands - no server upgrades will solve them.

That said, I hardly want to sound hopeless (and definitely not ungrateful). We're pros at working through our various struggles, and gaining oscar and carolyn has been the largest improvement in years, with still more benefits to come as we get rolling full bore in the new year and more aggressively shake out the remaining configuration problems. Plus a third new donated server (synergy) is coming down the pike, which we'll use to address other current shortcomings. When oscar, carolyn, and synergy are all being used to their fullest potential (a month or two from now?) let's revisit what our biggest system needs are. It may very well be that these new machines did in fact shake out the bulk of our current issues, and we'll be in good shape for years to come.

Happy new year! May we have actually publishable results in 2011 - positive or negative I don't care - it's science either way. We certainly could stand to get something meaningful in the journals concerning all the data we've been reducing for 11 years.

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Thanks for taking the time Matt...Most of us realize how Hard You All work in the Labs and at Home.
____________

I Desire Peace and Justice, Jim Scott

Thanks for the words. Go Home! Have a happy and prosperous new year.
____________

Thanks for the info Matt. Your taking the time to post these short notes is commendable indeed. These notes are what keep a lot of crunchers here at Seti, believe it or not.

Happy New Year to all the crew at Seti/Boinc. Hope next year we get published also. It would be nice after all these years to see some results in print.

____________
Boinc....Boinc....Boinc....Boinc....

When oscar, carolyn, and synergy are all being used to their fullest potential (a month or two from now?) let's revisit what our biggest system needs are. It may very well be that these new machines did in fact shake out the bulk of our current issues, and we'll be in good shape for years to come.

Thank you for that post, Matt.
Some don't realize that a couple of servers don't make the project.
We shall continue to bolster the hardware for you when and where needed.

Here's to all of the boyz in da lab and a great coming Seti New Year.
I pray you have worked through the worst.

Meow!!!!!!!
____________
******

"Ask not, what your kitty can do for you. Ask what you can do for your kitty."

As it is kitten, so shall it be done.

Thanks for the update Matt. May you and the rest of the Crew have a happy new year.
____________

Most of us understand that 10% of the machines in any system cause 90% of the problems. Fix or replace those, and 90% of your problems go away.

Sounds like that is what you've done. There will still be other issues and random gremlins, now and in the future, but it seems you have wiped out a lot the gremlins with the new servers.

Onwards and Upwards :-)

Ian

Happy new year Matt and the rest of the crew.

Thanks for the hard work.
Its much apreciated.

____________

Thanks for the update on the servers. I always like more insight into how the project is run.
Have a wonderful new years and see you next year.
____________

Thank you for the update, Matt. Happy New Year.
____________

Happy New Year!!!!!
SETI forever. I want to belive..

Thanks Matt, for the clear explanation of the situation in the lab and
ofcoarse a Happy new 2011...


____________


Knight Who Says Ni N!, OUT numbered.................