Last post of the year!

I understand the SETI@home/BOINC back-end is a bit confusing. I think there are only a handful of people who understand all the relationships between every step of the BOINC finite state machine, the SETI@home data pipeline, and every server in our closet. I can only really scratch the surface of these details during these relatively pithy missives.

So let me try my best to quickly answer the general question: "did those two brand new servers (oscar and carolyn) help?"

First of all, there are about 100 known problems with our systems at any given time. Most of these are low priority and "time out" on their own. There are still a bunch of high priority issues, of which these new servers addressed *some*.

One easy problem to fix was replacing mork (the randomly crashing master mysql database server) with carolyn. This has been great... and I think we just (as of this morning) solved the current batch of disk i/o problems (by properly tweaking the write cache settings). Carolyn also has a lot more disks and memory than mork had, so there's still a lot of room to grow as needed.

The other new server, oscar, is taking care of two major problems. First, the science database on thumper was abysmally slow - so now this is on oscar. That's great, however it's not running as fast as we'd like. We still haven't fully benchmarked this, though. Maybe at worst we'll need to add more disks to the system - we shall see. The second major problem oscar is helping to resolve is ptolemy - our internal administrative file server - which, like mork, also randomly crashes from time to time locking everything up. Now that thumper is off the hook as a science database server, it can take over and easily handle ptolemy's current functionality, and then we can retire ptolemy. There's actually a third minor problem that's also getting fixed: thumper's root partition is on a messed-up software RAID device which kinda works but has been scaring us for way too long. It'll be great to have this server-shuffle opportunity to have a quiet moment to reinstall the OS on thumper and fix that RAID.

But that's pretty much it for now. Other major problems exist with no clear solutions. In fact, many of them are data driven, or network infrastucture driven, and therefore out of our sysadmin hands - no server upgrades will solve them.

That said, I hardly want to sound hopeless (and definitely not ungrateful). We're pros at working through our various struggles, and gaining oscar and carolyn has been the largest improvement in years, with still more benefits to come as we get rolling full bore in the new year and more aggressively shake out the remaining configuration problems. Plus a third new donated server (synergy) is coming down the pike, which we'll use to address other current shortcomings. When oscar, carolyn, and synergy are all being used to their fullest potential (a month or two from now?) let's revisit what our biggest system needs are. It may very well be that these new machines did in fact shake out the bulk of our current issues, and we'll be in good shape for years to come.

Happy new year! May we have actually publishable results in 2011 - positive or negative I don't care - it's science either way. We certainly could stand to get something meaningful in the journals concerning all the data we've been reducing for 11 years.

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Thanks for taking the time Matt...Most of us realize how Hard You All work in the Labs and at Home.

---

I Desire Peace and Justice, Jim Scott (Mod-Ret.)

Thank you for the update Matt. May all of you there have a very Happy New Year!!

---

PROUD MEMBER OF Team Starfire World BOINC

Sounds like S@h is doing pretty well on its firm diet of shoe strings. Congrats one and all for all your work

---

In a rich man's house there is no place to spit but his face.
Diogenes Of Sinope

Thanks for the words. Go Home! Have a happy and prosperous new year.

---

Thanks for the info Matt. Your taking the time to post these short notes is commendable indeed. These notes are what keep a lot of crunchers here at Seti, believe it or not.

Happy New Year to all the crew at Seti/Boinc. Hope next year we get published also. It would be nice after all these years to see some results in print.

---

Boinc....Boinc....Boinc....Boinc....

Happy New Year to you and the rest of the Crew!

ront

---

Thank you for that post, Matt.
Some don't realize that a couple of servers don't make the project.
We shall continue to bolster the hardware for you when and where needed.

Here's to all of the boyz in da lab and a great coming Seti New Year.
I pray you have worked through the worst.

Meow!!!!!!!

---

"Time is simply the mechanism that keeps everything from happening all at once."

With all of the good work you have done this year with less than "stellar" equipment, I hope that the project leaders have seen fit to give you a year end performance reward. I know that you are not in this for the $, but with what you have managed to do this year, there should be something for you. Maybe there is some kibble for you somewhere. If not, then you are building up one heck of a positive karma.

thanks for all you have done

Dick

Thanks for the update Matt. May you and the rest of the Crew have a happy new year.

---

[/quote]

Old James

Thanks for posting. And have a Happy New Year....all of you.

---

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours

Most of us understand that 10% of the machines in any system cause 90% of the problems. Fix or replace those, and 90% of your problems go away.

Sounds like that is what you've done. There will still be other issues and random gremlins, now and in the future, but it seems you have wiped out a lot the gremlins with the new servers.

Onwards and Upwards :-)

Ian

Happy new year Matt and the rest of the crew.

Thanks for the hard work.
Its much apreciated.

---

With each crime and every kindness we birth our future.

Just to repeat everyone else, thanks for the post Matt. It's nice knowing what's up behind the scenes when possible. It's a lot of hard pain in the butt work at time and most of us know it, but rarely recognize it. Typical IT department stuff I suppose, always gets taken for granted! Either way nice job fellas, have an enjoyable new years and come in fresh for the 2011 year! It's going to be a hell of a ride.

---

Traveling through space at ~67,000mph!

Thanks for the update on the servers. I always like more insight into how the project is run.
Have a wonderful new years and see you next year.

---

Thank you for the update, Matt. Happy New Year.

---

Happy New Year!!!!!
SETI forever. I want to belive..

Thanks Matt, for the clear explanation of the situation in the lab and
ofcoarse a Happy new 2011...

---

Good stuff! Keep up the excellent work and Happy New Year to all.

---

Nothing cleaver here from me, Matt. Just a complement to you and your team for your perseverence on so many levels last year. Hopefully, there will be a good science publication in 2011, which will make it a very happy new year indeed!