Carolyn = Master (Nov 23 2010)


log in

Advanced search

Message boards : Technical News : Carolyn = Master (Nov 23 2010)

1 · 2 · Next
Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1384
Credit: 74,079
RAC: 0
United States
Message 1050627 - Posted: 23 Nov 2010, 20:59:01 UTC

Okay then - after some extreme DBA this morning carolyn is now the master mysql database server and jocelyn is the replica. So that project is officially DONE! Actually, there's a lot of low-priority cleanup to deal with, but all the main plumbing is working and the projects are back up such as they are.

Now all server side focus is on oscar. By far the most important thing to fix during this major long outage was our science database - getting a new mysql database rolling was just icing on the cake. But I guess we still need to finish making the cake. Most of our i/o bottlenecks over the past few years have been somehow linked to thumper (both as a database and file server) so getting this done is essential before we get back on line.

Jeff found a comprehensive list of missing spikes (which I mentioned yesterday) and will begin inserting those. We'll then eat some turkey, then have an all-hands-on-deck week next week to get oscar going. We simply cannot get back on line before then, and so we're still looking at new workunits being generated a couple weeks from today at the earliest. I guess if we're really lucky it'll be sooner, but highly doubtful. I know we're anxious to get rolling again, but remember that when you're dealing with billions of rows of data (in the form of a terabyte of raw files), each step takes many hours no matter how clever you are or how fast you type. It's also easy to get lost in theoretical maximum speeds, which never take into account (a) the dizzying array of initial preparations before even starting, (b) actual speeds, (c) the many extra steps necessary when being careful (like backing up a database one last time before dropping a table containing a billion rows), and (d) unpredictable software/hardware behavior requiring us to go back to N steps in the cookbook and try again.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile SciManStev
Volunteer tester
Avatar
Send message
Joined: 20 Jun 99
Posts: 4672
Credit: 77,437,203
RAC: 43,657
United States
Message 1050631 - Posted: 23 Nov 2010, 21:10:32 UTC

May I congratulate you all on a job well done and on going! We are all proud to have helped you get new servers, and appreciate you giving us these valuable updates. You are all truely experts in your fields, and we will be here when the gates are opened. Thank you very much for your hard work and dedication!

Steve
____________
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 13,743,237
RAC: 12,115
United States
Message 1050632 - Posted: 23 Nov 2010, 21:15:07 UTC - in response to Message 1050627.

Thanks for the update Matt. By all means take all the time you need to get everything right the first time. We aren't going anywhere and as far as I know nobody is dieing from lack of work.

We would like a few more pictures though if you get a chance.
____________


PROUD MEMBER OF Team Starfire World BOINC

Profile soft^spirit
Avatar
Send message
Joined: 18 May 99
Posts: 6374
Credit: 28,216,786
RAC: 128
United States
Message 1050642 - Posted: 23 Nov 2010, 22:02:40 UTC

It will not be until Oscar comes online that we can do a real "let's see what this thing has got" for either. But I saw someone trying to give it a load.. 772 and it did not even blink.
____________

Janice

Robert Ribbeck
Avatar
Send message
Joined: 7 Jun 02
Posts: 644
Credit: 5,283,174
RAC: 0
United States
Message 1050645 - Posted: 23 Nov 2010, 22:28:09 UTC

No good no bad
I'll believe what I see

____________

Profile Gary Charpentier
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 11732
Credit: 5,969,877
RAC: 0
United States
Message 1050659 - Posted: 23 Nov 2010, 23:16:52 UTC

Thanks for the update Matt. Take the time needed to get it right the first time. Maybe then the staff can all take a complete Christmas New Years break and not have to run in one day to reboot.


____________

Profile David Lambert
Send message
Joined: 21 Apr 02
Posts: 4
Credit: 737,751
RAC: 1,118
Canada
Message 1050668 - Posted: 23 Nov 2010, 23:49:50 UTC

OHHH! I can't wait to get crunching again.
____________

zpm
Volunteer tester
Avatar
Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,178,254
RAC: 3,408
United States
Message 1050673 - Posted: 24 Nov 2010, 0:15:11 UTC - in response to Message 1050668.

Keep up the good work, you guys may as well try and launch the shuttle with all the trouble... it must seem like it.
____________

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.

Claggy
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 3965
Credit: 31,928,355
RAC: 12,975
United Kingdom
Message 1050691 - Posted: 24 Nov 2010, 1:22:56 UTC - in response to Message 1050627.
Last modified: 24 Nov 2010, 1:24:34 UTC

Thanks for the update Matt, keep up the good work,

Claggy
____________


Profile [seti.international] Dirk Sadowski
Volunteer tester
Avatar
Send message
Joined: 6 Apr 07
Posts: 6971
Credit: 57,184,250
RAC: 22,072
Germany
Message 1050693 - Posted: 24 Nov 2010, 1:26:53 UTC - in response to Message 1050627.

Matt, thanks for the news!

____________
BR



>Das Deutsche Cafe. The German Cafe.<

Swibby Bear
Send message
Joined: 1 Aug 01
Posts: 236
Credit: 7,084,421
RAC: 1,309
United States
Message 1050696 - Posted: 24 Nov 2010, 1:37:58 UTC

Let me add my congrats and thanks.

However, I operate under the theory that there is no such thing as a stupid question, so here is mine:

What is "extreme DBA"?

Thanks, and Happy Thanksgiving (Thursday, for those outside the USA)

n7rfa
Volunteer tester
Avatar
Send message
Joined: 13 Apr 04
Posts: 370
Credit: 9,058,599
RAC: 0
United States
Message 1050699 - Posted: 24 Nov 2010, 1:45:04 UTC - in response to Message 1050696.

Extreme Database Administration
____________

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 23702
Credit: 498,265
RAC: 531
United States
Message 1050742 - Posted: 24 Nov 2010, 5:02:38 UTC - in response to Message 1050699.

Extreme Database Administration

Right. The question is what is the extreme part.
____________


BOINC WIKI

SMW
Send message
Joined: 16 May 99
Posts: 21
Credit: 10,635,727
RAC: 3,100
United States
Message 1050746 - Posted: 24 Nov 2010, 5:25:16 UTC

Matt,

Thanks for keeping us in the loop. While we all want to be back running the data, we know that working through this is part of it. Thanks again, and have a GREAT T-DAY:)
____________
"It is better to be hated for what you are then to be loved for what you are not"
- Andre Gide (1869-1951)

Swibby Bear
Send message
Joined: 1 Aug 01
Posts: 236
Credit: 7,084,421
RAC: 1,309
United States
Message 1050747 - Posted: 24 Nov 2010, 5:30:51 UTC - in response to Message 1050742.

Extreme Database Administration

Right. The question is what is the extreme part.


Thanks for the responses. (No, my question was actually the DBA part.)

Profile platium
Avatar
Send message
Joined: 5 Jul 10
Posts: 212
Credit: 262,426
RAC: 0
United Kingdom
Message 1050763 - Posted: 24 Nov 2010, 8:40:37 UTC

I did not know EDA either so it was a good question to ask as, I,ve just had a computer a year, thanks to you lot I am learn every day

Profile Mike
Volunteer tester
Avatar
Send message
Joined: 17 Feb 01
Posts: 22446
Credit: 29,462,582
RAC: 25,752
Germany
Message 1050764 - Posted: 24 Nov 2010, 8:40:41 UTC

I´m just wondering.

____________

rob smith
Volunteer moderator
Send message
Joined: 7 Mar 03
Posts: 7694
Credit: 45,185,432
RAC: 79,056
United Kingdom
Message 1050769 - Posted: 24 Nov 2010, 8:56:43 UTC

"Dropping a table with a billion rows" - it really doesn't matter if its a toy US billion, or a real UK billion, that's a big step. Lots of fingers crossed and rabbits' feet grasped, plus another backup.....
When you see these figures it makes one realise why this job has taken so long. Hopefully in a couple of weeks w will see the new server twins in action.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Profile Jack Zhang
Volunteer tester
Avatar
Send message
Joined: 2 Jul 06
Posts: 206
Credit: 5,911,901
RAC: 477
Canada
Message 1050771 - Posted: 24 Nov 2010, 8:59:56 UTC

I'm hoping any ventilation issues would be resolved before the servers come fully online. I'd keep it safe and make both servers very well ventilated and not on top of each other like in the final picture you took a few posts back.
____________
What if Fiction was Fact and Fact was Fiction and vice versa?

Profile Fred J. Verster
Volunteer tester
Avatar
Send message
Joined: 21 Apr 04
Posts: 3232
Credit: 31,585,541
RAC: 0
Netherlands
Message 1050782 - Posted: 24 Nov 2010, 12:19:53 UTC - in response to Message 1050771.

Full AirConditioning is the only way to keep a SERVER RACK cooled.
Especially when it has to be funtional 24 x7, with over a million hosts
connected to them, constantly Reporting and asking for different types of work!

____________


Knight Who Says Ni N!, OUT numbered.................

1 · 2 · Next

Message boards : Technical News : Carolyn = Master (Nov 23 2010)

Copyright © 2014 University of California