Balance of the 19 (Aug 04 2011)


log in

Advanced search

Message boards : Technical News : Balance of the 19 (Aug 04 2011)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 1136043 - Posted: 4 Aug 2011, 21:28:25 UTC

Not that it's all bright and shiny, but how about I just report some good news?

Looks like we got beyond the issues with the mysql replica on jocelyn. Basically we swapped in a bunch of different qlogic cards (which we had laying around) and one of them seems to be working. We're also using a new fibre cable (this new card had a different style jack so I was forced to do so). So far, so good - it recovered from the backup dump taken this past Tuesday, and currently as I type this sentence only 21K seconds behind (and still catching up best I can tell). Of course, we need to wait and see - chances are still good it may hiccup like before.

And also finally there's some non-zero hope in the HE connection issues front: one tech there may have a clue about a router configuration we may need to add/update on our end, though I'm still unsure what changed in the world to break this. I sent them some test results, now I'm just waiting to hear back.

You may have noticed some of our backend services going down today. This was planned. The short story is we just plucked 48GB of memory out of synergy (back-end compute server) and added it to oscar (the main science database server). So now oscar has 144GB of RAM to play with - the greater plan being to see if this actually helps informix performance, or are we (a) hopelessly blocked by bad disk i/o, and/or (b) dealing with a database so big that even maxing out memory in oscar at 192GB won't help. In any case, testing on this front moves forward. The more we understand, the more we learn *exactly* what hardware improvements we need.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4139
Credit: 33,442,384
RAC: 20,320
United Kingdom
Message 1136046 - Posted: 4 Aug 2011, 21:32:59 UTC - in response to Message 1136043.

Thanks for the update Matt,

Claggy

mole
Send message
Joined: 19 Jan 02
Posts: 5
Credit: 19,893,772
RAC: 11,992
United States
Message 1136047 - Posted: 4 Aug 2011, 21:33:51 UTC - in response to Message 1136043.

Cool Matt, thanks for the update and no so bad news this time.

Your efforts and those of others on this project are appreciated.
____________

Profile Jim_SProject donor
Avatar
Send message
Joined: 23 Feb 00
Posts: 4526
Credit: 18,792,947
RAC: 8,411
United States
Message 1136049 - Posted: 4 Aug 2011, 21:49:18 UTC

Thanks SO Much for the Good News Update Matt...Also Thanks for the HE update.
We (I) Appreciate the Hard WORK that you guys put into Our mutual Hobby/Work.
And the updates are nice too... ;-))
____________

I Desire Peace and Justice, Jim Scott (Mod-Ret.)

Profile soft^spirit
Avatar
Send message
Joined: 18 May 99
Posts: 6374
Credit: 28,631,148
RAC: 3
United States
Message 1136053 - Posted: 4 Aug 2011, 21:59:39 UTC - in response to Message 1136043.

That is the kind of science I can understand. Regarding Oscars overload, one question comes to mind: Can the database be split in two.. Science East/Science West for example?

May your test work well Matt.
____________

Janice

Profile Byron Leigh Hatch @ team Carl SaganProject donor
Volunteer tester
Avatar
Send message
Joined: 5 Jul 99
Posts: 3619
Credit: 11,906,348
RAC: 1,105
Canada
Message 1136056 - Posted: 4 Aug 2011, 22:23:40 UTC

Thanks for the update Matt and thanks to the SETI@home crew for all your long hours of hard work ... Best Wishes Byron.

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12702
Credit: 7,190,489
RAC: 15,505
United States
Message 1136080 - Posted: 5 Aug 2011, 0:22:20 UTC

Thanks for the update Matt and i hope the tests are successful. Nice to know where the bottleneck is so we can open up the pipe.

____________

Profile Jeff Mercer
Send message
Joined: 14 Aug 08
Posts: 90
Credit: 162,139
RAC: 0
United States
Message 1136106 - Posted: 5 Aug 2011, 2:36:51 UTC

Thanks Matt ! Good news at last !! Glad that things are looking better. All my best to you and the crew !

Profile James Butler
Volunteer tester
Avatar
Send message
Joined: 26 Jul 11
Posts: 143
Credit: 6,266
RAC: 0
United Kingdom
Message 1136742 - Posted: 6 Aug 2011, 14:15:56 UTC

thanks for the good news matt fingers crossed they all go well :D:D
____________

Profile Chris SProject donor
Volunteer tester
Avatar
Send message
Joined: 19 Nov 00
Posts: 32030
Credit: 13,713,256
RAC: 28,157
United Kingdom
Message 1137586 - Posted: 8 Aug 2011, 13:31:40 UTC
Last modified: 8 Aug 2011, 13:32:35 UTC

The short story is we just plucked 48GB of memory out of synergy (back-end compute server) and added it to oscar (the main science database server). So now oscar has 144GB of RAM to play with - the greater plan being to see if this actually helps informix performance, or are we (a) hopelessly blocked by bad disk i/o, and/or (b) dealing with a database so big that even maxing out memory in oscar at 192GB won't help.


You may remember Matt that some time ago, I did query the size of your Informix database, and whether for practical purposes it was managable. It's all very well for Informix to give theoretical maximum sizes. You are finding out first hand what actually works in practice.

Message boards : Technical News : Balance of the 19 (Aug 04 2011)

Copyright © 2014 University of California