| Author |
Message |
|---|
 Matt LebofskyVolunteer moderator
Project administrator
Project developer
Project scientist
Send message
Joined: 1 Mar 99
Posts: 1379
Credit: 74,079
RAC: 0
|
|
Happy New Year!
We seem to have been running rather smoothly since last I wrote. And the servers were nice enough to wait until we were all back in the lab before going crazy. Well, it's not that bad - just lots of tiny problems. The astropulse science database server got stuck in some deadlock and needed a hard power cycle. Then I tried fixing this nagging db_dump problem (it hasn't regularly generating daily stats dumps) by moving the process from bruno to lando, and lando couldn't handle it - and I ended up needing to hard power cycle lando as well. Having lando reboot caused bruno to freak out a little bit as it was in the middle of a package update when lando disappeared out from underneath it. So I had some rpm database cleanup to deal with to get bruno to start upload results again. Oy!
Meanwhile we were bringing services up and down in a controlled manner to make some more science database tweaks on oscar. We're still deep into trying to figure out how to improve its throughput in general. We're finding the disks are still a bottleneck (even after the RAID restripe), but if we can get informix to cache more efficiently then disk i/o is less important. Or we'll add more disk spindles. In the meantime we have many knobs to turn.
I got the tracking # for new donated server "synergy" - should be here on Friday afternoon, so we'll start playing with it next week!
Regarding the weekly outages: we're sticking to the general idea which I mentioned recently - generally have our standard Tuesday half-day outage, and perhaps other planned one or two day outages as needed (with some effort to provide ample advance warning), but otherwise leave all public facing data services up and running as much as possible. However in the first sign of trouble, these services may be taken down for extended periods. The bottom line is you probably won't notice any difference between current operations and those from six months ago (before the weekly extended outages). The goal is to aim for 24/7 uptime while maintaining staff sanity.
- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
|
|
|
|
|
Happy New Year to you too Matt! And thanks for the update.
The goal is to aim for 24/7 uptime while maintaining staff sanity.
I think we could all raise a glass to that one, sounds like an excellent game plan!
____________
Damsel Rescuer, Kitty Patron, Raccoon Friend, Uli Fan,
Julie Supporter, ES99 Admirer, PETA Member, 1st Childhood
   |
|
|
|
|
|
Happy New Year and thanks for the updates and please keep sane!
____________
  |
|
|
|
|
|
Yes sanity==good and you guys are bending over backwards to keep things going. Hang in there!
____________
 |
|
|
|
|
|
Happy new year to the whole crew!!!!
You are still down for maintenance far less than I am, so its all good here. Science first the rest is just that //the rest//.
Brant
____________
|
|
|
|
|
|
Staff sanity - good luck.
All the best for the New Year.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe? |
|
|
|
|
|
Worth remembering some of that for the subsequent moaners we will encounter as time goes on. |
|
|
|
|
|
Sanity is so overrated!! :-)
____________
PROUD MEMBER OF Team Starfire World BOINC |
|
|
|
|
|
To remind me, what task will Synergy be assigned to? |
|
|
|
|
To remind me, what task will Synergy be assigned to?
Afaik, that is yet to be decided.
____________
/The grumpy old Swede.
"I'm so old, that 98% of all trees in the forest, are younger than I am" |
|
|
|
|
|
Good luck with Sanity and 24/7 up time.
Cheers,
Chris
____________
|
|
|
|
|
|
I think everyone aims for 24/7 up time. Never down time. No matter how hard the team or anyone strides for it, at this point in technological time, it's just going to happen sooner or later, so lots o' luck with keeping everyone sane all the time as well.
Besides. . . All that constant sanity is sooo boring anyway. People should try inserting some unharmful insanity once in a while into their always serious lives. It helps unclog the old brain's drain. ;-)
{Just keep the water balloon & soaker fights confined to safe areas while inside buildings. :-)}
____________
Wishing you continued science "fun" and at least, always valid results,
Laters,
Rick "WHOSIT" W.
Participating in: Einstein/MilkyWay/Rosetta/SETI
 |
|
|
|
|
Besides. . . All that constant sanity is sooo boring anyway. People should try inserting some unharmful insanity once in a while into their always serious lives. It helps unclog the old brain's drain. ;-)
{Just keep the water balloon & soaker fights confined to safe areas while inside buildings. :-)}
...And keep all that water away from the server closet!
____________
.
  |
|
|
|
|
Besides. . . All that constant sanity is sooo boring anyway. People should try inserting some unharmful insanity once in a while into their always serious lives. It helps unclog the old brain's drain. ;-)
{Just keep the water balloon & soaker fights confined to safe areas while inside buildings. :-)}
...And keep all that water away from the server closet!
Oh heck yeah! Exactly the type of area which wouldn't be safe and would most likely cause permanent insanity.
"The horror. The horror." :-)
____________
Wishing you continued science "fun" and at least, always valid results,
Laters,
Rick "WHOSIT" W.
Participating in: Einstein/MilkyWay/Rosetta/SETI
|
|
|
