Rainy Day (Jan 19 2010)


log in

Advanced search

Message boards : Technical News : Rainy Day (Jan 19 2010)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1390
Credit: 74,079
RAC: 0
United States
Message 964669 - Posted: 19 Jan 2010, 23:13:31 UTC

Long holiday weekend (it was Martin Luther King Day yesterday) during which no major snags, but a couple minor ones. The data pipeline ran dry, but Jeff got on top of that before most people noticed. The mysql replica also lost touch with the master and threw itself offline. This is an old problem that went away for a while, but is apparently back to annoy us. Not a big deal, except the alert e-mails kinda got "lost in the noise" of the holiday weekend and we didn't kick the thing back to life until this morning.

Meanwhile, we're having the usual weekly mysql maintenance outage. The replica caught up a bunch while we were offline today, but still may take a day or two to fully get back in sync. Until then, any queries made to that database will be slightly out of date. Fine.

In better news, this last iteration of the secondary science database recovery project seems to have worked, or at least working. It took 6 days as expected to fully back up and restore the secondary from the primary, which was expected, but this time we had enough logical logs on line so that they didn't "wrap around" during this process and we were forced to try to recover from continuous logical log backups. We tried recovery from the continuous logs last week - The bothersome thing is that should have worked. Anyway... the secondary up and doing the final stages of recovery now.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 16,210,034
RAC: 6,539
United States
Message 964673 - Posted: 19 Jan 2010, 23:30:45 UTC - in response to Message 964669.
Last modified: 19 Jan 2010, 23:31:16 UTC

Hi Matt,
FYI, we noticed but also saw them come back so not many complained. :-) Here's hoping the replica gets caught up soon, a lot of people have been b...complaining about not seeing their stats. Glad to hear about the Sec. Science database Hope you figure out why it didn't work the first time.
____________


PROUD MEMBER OF Team Starfire World BOINC

Profile Neil Blaikie
Volunteer tester
Avatar
Send message
Joined: 17 May 99
Posts: 142
Credit: 6,632,198
RAC: 1,643
Canada
Message 964674 - Posted: 19 Jan 2010, 23:35:08 UTC - in response to Message 964669.
Last modified: 19 Jan 2010, 23:44:10 UTC

That's good news for you guys.

Hopefully you had a relaxing day yesterday, which you so deserve.

Thank you as always for the updates.

As you mentioned in a previous thread, you guys had a tidy up of the lab, any chance of some new pictures of the server closet and your handy work :-)?

This will be my last post from this machine, it is about to move from a E6400 to a Q8200 (Dual to Quad Core), shame the GTX 260 that should have come with it, is not with the order and is on back order for 2 weeks. Nice new 750w Kingwin PSU as well.
____________

Profile FrostKing9
Avatar
Send message
Joined: 20 Oct 01
Posts: 39
Credit: 23,815,960
RAC: 0
United States
Message 964797 - Posted: 20 Jan 2010, 18:59:52 UTC
Last modified: 20 Jan 2010, 19:19:07 UTC

Matt,
You guys do a hell-of-a job, considering how complicated it all is to keep running. SETI is on my gift list for 1-million dollars, after I hit the lotto. Seriously. But in the meantime I'm making a much smaller donation, one that I can afford now. <sigh> Thank you, thank you, thank you.
____________


I DONATE money to SETI@home.... DO YOU?

I'm just slowly BOINC'ing along.

Hey... ET... you have a sister who likes earthlings?

Message boards : Technical News : Rainy Day (Jan 19 2010)

Copyright © 2014 University of California