Fall Cleaning (Oct 25 2007)


log in

Advanced search

Message boards : Technical News : Fall Cleaning (Oct 25 2007)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 666498 - Posted: 25 Oct 2007, 20:30:00 UTC

For some reason I'm in the "deal with boring, nagging sysadmin tasks" zone this week, so that's mostly what I've been working on. Gotta ride the wave when it happens, you know? Nothing really interesting there to report. Writing scripts, updating our UPS plans, cleaning up and improving our internal alert system... stuff like that.

Last night the logical log on our primary science database filled up. This is the log that is used by the secondary to keep in sync with the primary. When the log is full, the primary halts all connections as a protective measure, as the secondary will lose track of future updates. What does all this mean for you? Well, with the primary effectively offline the assimilators and splitters were blocked, and we ran out of work to send this morning. We spotted this quickly enough, but apparently we need better alerts and some automatic logical log rotation system. We're still getting the feel of this informix database replication stuff.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Dr. C.E.T.I.
Avatar
Send message
Joined: 29 Feb 00
Posts: 15993
Credit: 690,597
RAC: 0
United States
Message 666500 - Posted: 25 Oct 2007, 20:35:04 UTC - in response to Message 666498.

For some reason I'm in the "deal with boring, nagging sysadmin tasks" zone this week, so that's mostly what I've been working on. Gotta ride the wave when it happens, you know? Nothing really interesting there to report. Writing scripts, updating our UPS plans, cleaning up and improving our internal alert system... stuff like that.

Last night the logical log on our primary science database filled up. This is the log that is used by the secondary to keep in sync with the primary. When the log is full, the primary halts all connections as a protective measure, as the secondary will lose track of future updates. What does all this mean for you? Well, with the primary effectively offline the assimilators and splitters were blocked, and we ran out of work to send this morning. We spotted this quickly enough, but apparently we need better alerts and some automatic logical log rotation system. We're still getting the feel of this informix database replication stuff.

- Matt


Thank You Matt - and to the Rest of the Berkeley Staff - Thanks


____________
BOINC Wiki . . .

Science Status Page . . .

DJStarfox
Send message
Joined: 23 May 01
Posts: 1040
Credit: 546,997
RAC: 251
United States
Message 666699 - Posted: 26 Oct 2007, 2:29:24 UTC - in response to Message 666498.
Last modified: 26 Oct 2007, 2:30:09 UTC

Any good DBA managing a busy, replicated database is always watching his transaction log files. It's amazing how fast they can fill up; I've seen 8 GB disappear in 5 minutes when a certain user is connected! Unfortunately, I've never worked with Informix, so I couldn't offer any strategies to help.

Profile JLDun
Volunteer tester
Avatar
Send message
Joined: 21 Apr 06
Posts: 307
Credit: 51,623
RAC: 0
United States
Message 666753 - Posted: 26 Oct 2007, 3:46:15 UTC - in response to Message 666699.

Any good DBA managing a busy, replicated database is always watching his transaction log files. It's amazing how fast they can fill up; I've seen 8 GB disappear in 5 minutes when a certain user is connected! Unfortunately, I've never worked with Informix, so I couldn't offer any strategies to help.

Would that be NEZ? ;-p
____________

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8408
Credit: 4,127,320
RAC: 1,338
United Kingdom
Message 666943 - Posted: 26 Oct 2007, 12:26:45 UTC - in response to Message 666699.

Any good DBA managing a busy, replicated database is always watching his transaction log files. ...

An even better one is elsewhere doing useful work whilst a check script (eg logwatch?) is doing the checking, ready to send an alert for anything 'unusual'...

Regards,
Martin

____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 667051 - Posted: 26 Oct 2007, 16:21:12 UTC - in response to Message 666943.

I think the issue is that we are still getting a feel for informix replication, such as it is. Documentation isn't so great. And having done it easily and relatively seamlessly with mysql, there's still an "astonishment" factor with informix replication. So things are going up and down as we test, and logs filling up isn't exactly the main thing on our radar yet.

- Matt

Any good DBA managing a busy, replicated database is always watching his transaction log files. ...

An even better one is elsewhere doing useful work whilst a check script (eg logwatch?) is doing the checking, ready to send an alert for anything 'unusual'...

Regards,
Martin


____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8408
Credit: 4,127,320
RAC: 1,338
United Kingdom
Message 667066 - Posted: 26 Oct 2007, 16:49:28 UTC - in response to Message 667051.
Last modified: 26 Oct 2007, 16:52:10 UTC

I think the issue is that we are still getting a feel for informix replication, such as it is. Documentation isn't so great. And having done it easily and relatively seamlessly with mysql, there's still an "astonishment" factor with informix...

There's always more to learn.

I'm very glad that I'm not the one on the sharp end of a million on-line remote 'experts'!

(Matt alias "Neo"? ;-) )


(And I fully expect a few 'giggles' for myself when I build a mere six server system soonish. A weekend is likely not long enough... And neither will be the patch leads... :-( All good fun! )

Happy crunchin',

Regards,
Martin

____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Message boards : Technical News : Fall Cleaning (Oct 25 2007)

Copyright © 2014 University of California