Carolyn = Master (Nov 23 2010)

Message boards : Technical News : Carolyn = Master (Nov 23 2010)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1050627 - Posted: 23 Nov 2010, 20:59:01 UTC

Okay then - after some extreme DBA this morning carolyn is now the master mysql database server and jocelyn is the replica. So that project is officially DONE! Actually, there's a lot of low-priority cleanup to deal with, but all the main plumbing is working and the projects are back up such as they are.

Now all server side focus is on oscar. By far the most important thing to fix during this major long outage was our science database - getting a new mysql database rolling was just icing on the cake. But I guess we still need to finish making the cake. Most of our i/o bottlenecks over the past few years have been somehow linked to thumper (both as a database and file server) so getting this done is essential before we get back on line.

Jeff found a comprehensive list of missing spikes (which I mentioned yesterday) and will begin inserting those. We'll then eat some turkey, then have an all-hands-on-deck week next week to get oscar going. We simply cannot get back on line before then, and so we're still looking at new workunits being generated a couple weeks from today at the earliest. I guess if we're really lucky it'll be sooner, but highly doubtful. I know we're anxious to get rolling again, but remember that when you're dealing with billions of rows of data (in the form of a terabyte of raw files), each step takes many hours no matter how clever you are or how fast you type. It's also easy to get lost in theoretical maximum speeds, which never take into account (a) the dizzying array of initial preparations before even starting, (b) actual speeds, (c) the many extra steps necessary when being careful (like backing up a database one last time before dropping a table containing a billion rows), and (d) unpredictable software/hardware behavior requiring us to go back to N steps in the cookbook and try again.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1050627 · Report as offensive
Profile SciManStev Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 20 Jun 99
Posts: 6651
Credit: 121,090,076
RAC: 0
United States
Message 1050631 - Posted: 23 Nov 2010, 21:10:32 UTC

May I congratulate you all on a job well done and on going! We are all proud to have helped you get new servers, and appreciate you giving us these valuable updates. You are all truely experts in your fields, and we will be here when the gates are opened. Thank you very much for your hard work and dedication!

Steve
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website
ID: 1050631 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 1050632 - Posted: 23 Nov 2010, 21:15:07 UTC - in response to Message 1050627.  

Thanks for the update Matt. By all means take all the time you need to get everything right the first time. We aren't going anywhere and as far as I know nobody is dieing from lack of work.

We would like a few more pictures though if you get a chance.


PROUD MEMBER OF Team Starfire World BOINC
ID: 1050632 · Report as offensive
Profile soft^spirit
Avatar

Send message
Joined: 18 May 99
Posts: 6497
Credit: 34,134,168
RAC: 0
United States
Message 1050642 - Posted: 23 Nov 2010, 22:02:40 UTC

It will not be until Oscar comes online that we can do a real "let's see what this thing has got" for either. But I saw someone trying to give it a load.. 772 and it did not even blink.
Janice
ID: 1050642 · Report as offensive
Robert Ribbeck
Avatar

Send message
Joined: 7 Jun 02
Posts: 644
Credit: 5,283,174
RAC: 0
United States
Message 1050645 - Posted: 23 Nov 2010, 22:28:09 UTC

No good no bad
I'll believe what I see

ID: 1050645 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30591
Credit: 53,134,872
RAC: 32
United States
Message 1050659 - Posted: 23 Nov 2010, 23:16:52 UTC

Thanks for the update Matt. Take the time needed to get it right the first time. Maybe then the staff can all take a complete Christmas New Years break and not have to run in one day to reboot.


ID: 1050659 · Report as offensive
Profile David Lambert

Send message
Joined: 21 Apr 02
Posts: 7
Credit: 3,012,995
RAC: 0
Canada
Message 1050668 - Posted: 23 Nov 2010, 23:49:50 UTC

OHHH! I can't wait to get crunching again.
ID: 1050668 · Report as offensive
zpm
Volunteer tester
Avatar

Send message
Joined: 25 Apr 08
Posts: 284
Credit: 1,659,024
RAC: 0
United States
Message 1050673 - Posted: 24 Nov 2010, 0:15:11 UTC - in response to Message 1050668.  

Keep up the good work, you guys may as well try and launch the shuttle with all the trouble... it must seem like it.

I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/
Go Georgia Tech.
ID: 1050673 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1050691 - Posted: 24 Nov 2010, 1:22:56 UTC - in response to Message 1050627.  
Last modified: 24 Nov 2010, 1:24:34 UTC

Thanks for the update Matt, keep up the good work,

Claggy
ID: 1050691 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 1050693 - Posted: 24 Nov 2010, 1:26:53 UTC - in response to Message 1050627.  

Matt, thanks for the news!

ID: 1050693 · Report as offensive
Swibby Bear

Send message
Joined: 1 Aug 01
Posts: 246
Credit: 7,945,093
RAC: 0
United States
Message 1050696 - Posted: 24 Nov 2010, 1:37:58 UTC

Let me add my congrats and thanks.

However, I operate under the theory that there is no such thing as a stupid question, so here is mine:

What is "extreme DBA"?

Thanks, and Happy Thanksgiving (Thursday, for those outside the USA)
ID: 1050696 · Report as offensive
n7rfa
Volunteer tester
Avatar

Send message
Joined: 13 Apr 04
Posts: 370
Credit: 9,058,599
RAC: 0
United States
Message 1050699 - Posted: 24 Nov 2010, 1:45:04 UTC - in response to Message 1050696.  

Extreme Database Administration
ID: 1050699 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 1050742 - Posted: 24 Nov 2010, 5:02:38 UTC - in response to Message 1050699.  

Extreme Database Administration

Right. The question is what is the extreme part.


BOINC WIKI
ID: 1050742 · Report as offensive
Profile SMW

Send message
Joined: 16 May 99
Posts: 22
Credit: 29,285,238
RAC: 16
United States
Message 1050746 - Posted: 24 Nov 2010, 5:25:16 UTC

Matt,

Thanks for keeping us in the loop. While we all want to be back running the data, we know that working through this is part of it. Thanks again, and have a GREAT T-DAY:)
"It is better to be hated for what you are then to be loved for what you are not"
- Andre Gide (1869-1951)
ID: 1050746 · Report as offensive
Swibby Bear

Send message
Joined: 1 Aug 01
Posts: 246
Credit: 7,945,093
RAC: 0
United States
Message 1050747 - Posted: 24 Nov 2010, 5:30:51 UTC - in response to Message 1050742.  

Extreme Database Administration

Right. The question is what is the extreme part.


Thanks for the responses. (No, my question was actually the DBA part.)
ID: 1050747 · Report as offensive
Profile platium
Avatar

Send message
Joined: 5 Jul 10
Posts: 212
Credit: 262,426
RAC: 0
United Kingdom
Message 1050763 - Posted: 24 Nov 2010, 8:40:37 UTC

I did not know EDA either so it was a good question to ask as, I,ve just had a computer a year, thanks to you lot I am learn every day
ID: 1050763 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34249
Credit: 79,922,639
RAC: 80
Germany
Message 1050764 - Posted: 24 Nov 2010, 8:40:41 UTC

I´m just wondering.



With each crime and every kindness we birth our future.
ID: 1050764 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22149
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1050769 - Posted: 24 Nov 2010, 8:56:43 UTC

"Dropping a table with a billion rows" - it really doesn't matter if its a toy US billion, or a real UK billion, that's a big step. Lots of fingers crossed and rabbits' feet grasped, plus another backup.....
When you see these figures it makes one realise why this job has taken so long. Hopefully in a couple of weeks w will see the new server twins in action.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1050769 · Report as offensive
Profile Jack Zhang
Volunteer tester
Avatar

Send message
Joined: 2 Jul 06
Posts: 206
Credit: 6,142,449
RAC: 0
Canada
Message 1050771 - Posted: 24 Nov 2010, 8:59:56 UTC

I'm hoping any ventilation issues would be resolved before the servers come fully online. I'd keep it safe and make both servers very well ventilated and not on top of each other like in the final picture you took a few posts back.
What if Fiction was Fact and Fact was Fiction and vice versa?
ID: 1050771 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 1050782 - Posted: 24 Nov 2010, 12:19:53 UTC - in response to Message 1050771.  

Full AirConditioning is the only way to keep a SERVER RACK cooled.
Especially when it has to be funtional 24 x7, with over a million hosts
connected to them, constantly Reporting and asking for different types of work!

ID: 1050782 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Carolyn = Master (Nov 23 2010)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.