Back to the Grind (Jan 03 2011)

Message boards : Technical News : Back to the Grind (Jan 03 2011)

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1441
Credit: 213,689
RAC: 0
United States
Message 1063163 - Posted: 3 Jan 2011, 22:12:30 UTC

Happy New Year!

We seem to have been running rather smoothly since last I wrote. And the servers were nice enough to wait until we were all back in the lab before going crazy. Well, it's not that bad - just lots of tiny problems. The astropulse science database server got stuck in some deadlock and needed a hard power cycle. Then I tried fixing this nagging db_dump problem (it hasn't regularly generating daily stats dumps) by moving the process from bruno to lando, and lando couldn't handle it - and I ended up needing to hard power cycle lando as well. Having lando reboot caused bruno to freak out a little bit as it was in the middle of a package update when lando disappeared out from underneath it. So I had some rpm database cleanup to deal with to get bruno to start upload results again. Oy!

Meanwhile we were bringing services up and down in a controlled manner to make some more science database tweaks on oscar. We're still deep into trying to figure out how to improve its throughput in general. We're finding the disks are still a bottleneck (even after the RAID restripe), but if we can get informix to cache more efficiently then disk i/o is less important. Or we'll add more disk spindles. In the meantime we have many knobs to turn.

I got the tracking # for new donated server "synergy" - should be here on Friday afternoon, so we'll start playing with it next week!

Regarding the weekly outages: we're sticking to the general idea which I mentioned recently - generally have our standard Tuesday half-day outage, and perhaps other planned one or two day outages as needed (with some effort to provide ample advance warning), but otherwise leave all public facing data services up and running as much as possible. However in the first sign of trouble, these services may be taken down for extended periods. The bottom line is you probably won't notice any difference between current operations and those from six months ago (before the weekly extended outages). The goal is to aim for 24/7 uptime while maintaining staff sanity.

- Matt


-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

ID: 1063163 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 38181
Credit: 21,347,486
RAC: 27,701
United Kingdom
Message 1063178 - Posted: 3 Jan 2011, 22:35:14 UTC

Happy New Year to you too Matt! And thanks for the update.

The goal is to aim for 24/7 uptime while maintaining staff sanity.


I think we could all raise a glass to that one, sounds like an excellent game plan!

Those are my principles, and if you don't like them ... well, I have others.
Groucho Marx 1895-1977

I also have mine, and if you don't like them ... tough, live with it.
Chris S 2016

Member of UCB Charter Hill Society

ID: 1063178 · Report as offensive
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 18638
Credit: 21,450,039
RAC: 20,142
United States
Message 1063196 - Posted: 3 Jan 2011, 22:56:49 UTC

Happy New Year and thanks for the updates and please keep sane!


ID: 1063196 · Report as offensive
Pete Church
Volunteer tester
Avatar

Send message
Joined: 1 Sep 08
Posts: 7
Credit: 6,827,500
RAC: 0
United States
Message 1063349 - Posted: 4 Jan 2011, 6:18:17 UTC - in response to Message 1063163.  

Yes sanity==good and you guys are bending over backwards to keep things going. Hang in there!


ID: 1063349 · Report as offensive
Profile Brant

Send message
Joined: 10 Dec 99
Posts: 13
Credit: 1,740,794
RAC: 0
United States
Message 1063371 - Posted: 4 Jan 2011, 9:33:52 UTC

Happy new year to the whole crew!!!!

You are still down for maintenance far less than I am, so its all good here. Science first the rest is just that //the rest//.




Brant


ID: 1063371 · Report as offensive
rob smithProject Donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 13335
Credit: 154,651,038
RAC: 116,532
United Kingdom
Message 1063373 - Posted: 4 Jan 2011, 9:45:40 UTC

Staff sanity - good luck.

All the best for the New Year.


Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

ID: 1063373 · Report as offensive
Dave
Avatar

Send message
Joined: 29 Mar 02
Posts: 777
Credit: 25,001,396
RAC: 0
United Kingdom
Message 1063728 - Posted: 5 Jan 2011, 18:53:36 UTC

Worth remembering some of that for the subsequent moaners we will encounter as time goes on.

ID: 1063728 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 1063733 - Posted: 5 Jan 2011, 18:59:42 UTC

Sanity is so overrated!! :-)




PROUD MEMBER OF Team Starfire World BOINC

ID: 1063733 · Report as offensive
Olli

Send message
Joined: 25 Apr 07
Posts: 143
Credit: 2,089,162
RAC: 0
Finland
Message 1063822 - Posted: 6 Jan 2011, 1:34:28 UTC - in response to Message 1063163.  

To remind me, what task will Synergy be assigned to?

ID: 1063822 · Report as offensive
Tutankhamon "Communist"
Volunteer tester
Avatar

Send message
Joined: 1 Nov 08
Posts: 6087
Credit: 37,687,914
RAC: 17,213
Sweden
Message 1063824 - Posted: 6 Jan 2011, 1:46:33 UTC - in response to Message 1063822.  

To remind me, what task will Synergy be assigned to?


Afaik, that is yet to be decided.

This is a test of the Emergency Moron System. Had there been a real moron in the room, there would've been a small mushroom cloud in the place where the idiot had been standing.

ID: 1063824 · Report as offensive
Profile Ozmoses
Volunteer tester
Avatar

Send message
Joined: 9 Jun 03
Posts: 16
Credit: 27,536,204
RAC: 6,647
Australia
Message 1063935 - Posted: 6 Jan 2011, 9:52:35 UTC

Good luck with Sanity and 24/7 up time.

Cheers,

Chris


ID: 1063935 · Report as offensive
Profile WHOSIT
Avatar

Send message
Joined: 12 Nov 10
Posts: 29
Credit: 162,958
RAC: 0
United States
Message 1064815 - Posted: 9 Jan 2011, 0:54:23 UTC

I think everyone aims for 24/7 up time. Never down time. No matter how hard the team or anyone strides for it, at this point in technological time, it's just going to happen sooner or later, so lots o' luck with keeping everyone sane all the time as well.

Besides. . . All that constant sanity is sooo boring anyway. People should try inserting some unharmful insanity once in a while into their always serious lives. It helps unclog the old brain's drain. ;-)
{Just keep the water balloon & soaker fights confined to safe areas while inside buildings. :-)}


Wishing you continued science "fun" and at least, always valid results,
Laters,
Rick "WHOSIT" W.
Participating in: Einstein/MilkyWay/Rosetta/SETI

ID: 1064815 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 2580
Credit: 34,730,541
RAC: 20,012
United States
Message 1065012 - Posted: 9 Jan 2011, 19:02:00 UTC - in response to Message 1064815.  


Besides. . . All that constant sanity is sooo boring anyway. People should try inserting some unharmful insanity once in a while into their always serious lives. It helps unclog the old brain's drain. ;-)
{Just keep the water balloon & soaker fights confined to safe areas while inside buildings. :-)}


...And keep all that water away from the server closet!
.

ID: 1065012 · Report as offensive
Profile WHOSIT
Avatar

Send message
Joined: 12 Nov 10
Posts: 29
Credit: 162,958
RAC: 0
United States
Message 1065201 - Posted: 10 Jan 2011, 8:57:41 UTC - in response to Message 1065012.  


Besides. . . All that constant sanity is sooo boring anyway. People should try inserting some unharmful insanity once in a while into their always serious lives. It helps unclog the old brain's drain. ;-)
{Just keep the water balloon & soaker fights confined to safe areas while inside buildings. :-)}


...And keep all that water away from the server closet!


Oh heck yeah! Exactly the type of area which wouldn't be safe and would most likely cause permanent insanity.

"The horror. The horror." :-)

Wishing you continued science "fun" and at least, always valid results,
Laters,
Rick "WHOSIT" W.
Participating in: Einstein/MilkyWay/Rosetta/SETI

ID: 1065201 · Report as offensive

Message boards : Technical News : Back to the Grind (Jan 03 2011)


 
©2016 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.