Striped (Dec 22 2010)

Message boards : Technical News : Striped (Dec 22 2010)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1058854 - Posted: 22 Dec 2010, 21:25:56 UTC

Everything on the oscar-raid-restripe project went along swimmingly, and I was able to take care of several steps at home last night so that we could get the whole project back online in the morning today.

Funny aside #1: at the exact moment I was finally comfortable enough to issue the command that blew away the old raid device from my home terminal, the network connectivity in my house randomly disappeared. Needless to say this incited minor panic as I suddenly couldn't reach any SETI servers and thought I must have somehow locked up the whole server closet. Eventually I figured out it was a local problem, my home router rebooted itself and I was back in business. Phew.

Funny aside #2: turns out when I installed the OS on oscar/carolyn I didn't bother removing the NetworkManager package, which gets installed by default for reasons beyond my power of conception. I've ranted about this before but it seems the only reason NetworkManager exists is to generate completely random, unexpected, obfuscated network problems on systems in order to give network administrators (a) something to do to kill time, and (b) ensure job security. In this case, without cause or prior consent it added funny loopback address statements in /etc/hosts which caused remote informix connections to fail. I immediately removed NetworkManager at that point and added a "exclude=NetworkManager*" line to yum.conf (which I should have done before).

Anyway, right off the bat there seems to be at least some improvement with the smaller stripe size on the raid, but we're collecting i/o stats under load and have some tweaks in mind which may continue to help general performance. All told, this ended up just being a one-day outage well worth taking.

Meanwhile, upon turning everything back on mysql started exhibiting some old, bad habits. I haven't seen this behavior in a long while, but sometimes when it gets hit hard enough it says, "Yo! Back off! In fact, I'm going to block all incoming queries and write to disk for 10-20 minutes. I won't tell you why, and there's nothing you can do about it." Fair enough - but if you were having trouble reaching the website an hour ago, that's the reason.

That's pretty much it for this week. It's officially the Xmas holiday starting tomorrow. Whatever you choose to do, enjoy! I'll be checking in from home over the "time off," but we're hoping it's kind of a "set it and forget it" kind of weekend instead of an "oh well everything will be down until we return on Monday" kind of weekend.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1058854 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1058857 - Posted: 22 Dec 2010, 21:32:36 UTC - in response to Message 1058854.  

Thanks for the update Matt, and have a Merry Christmas,

Claggy
ID: 1058857 · Report as offensive
Profile skildude
Avatar

Send message
Joined: 4 Oct 00
Posts: 9541
Credit: 50,759,529
RAC: 60
Yemen
Message 1058858 - Posted: 22 Dec 2010, 21:33:21 UTC - in response to Message 1058854.  

Sounds like you've kept busy. Heck you've earned 4 day weekend.


In a rich man's house there is no place to spit but his face.
Diogenes Of Sinope
ID: 1058858 · Report as offensive
Profile Cliff Harding
Volunteer tester
Avatar

Send message
Joined: 18 Aug 99
Posts: 1432
Credit: 110,967,840
RAC: 67
United States
Message 1058859 - Posted: 22 Dec 2010, 21:34:47 UTC - in response to Message 1058854.  

Matt,

Thank for the update!! Oscar is sure proving his worth with the re-striping and re-loading of science DB. That thing sure is fasssst!!. We might have to replace all of the other servers just to keep up with him and Carolyn. Any idea of an ETA of the third server?

Keep of the good work and have a very joyful Christmas!!
ID: 1058859 · Report as offensive
Profile soft^spirit
Avatar

Send message
Joined: 18 May 99
Posts: 6497
Credit: 34,134,168
RAC: 0
United States
Message 1058860 - Posted: 22 Dec 2010, 21:35:02 UTC

Aww how nice, we got early servers back for Xmas.. And we did not get you anything :(

Enjoy your holiday!
Janice
ID: 1058860 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 9958
Credit: 103,452,613
RAC: 328
United Kingdom
Message 1058875 - Posted: 22 Dec 2010, 21:58:50 UTC

Merry Christmas Matt, we all do really appreciate the work you all do. Despite sometimes being a bit grumpy :-)

Bernie
ID: 1058875 · Report as offensive
Profile Jim_S
Avatar

Send message
Joined: 23 Feb 00
Posts: 4705
Credit: 64,560,357
RAC: 31
United States
Message 1058886 - Posted: 22 Dec 2010, 22:32:26 UTC - in response to Message 1058875.  

Merry Christmas Matt, we all do really appreciate the work you all do. Despite sometimes being a bit grumpy :-)

Bernie

What Bernie said!!! ;-]

I Desire Peace and Justice, Jim Scott (Mod-Ret.)
ID: 1058886 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 31014
Credit: 53,134,872
RAC: 32
United States
Message 1058887 - Posted: 22 Dec 2010, 22:32:55 UTC

Merry Christmas Matt and the rest of the staff.

ID: 1058887 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 1058888 - Posted: 22 Dec 2010, 22:32:59 UTC - in response to Message 1058875.  

Merry Christmas Matt and all the gang there at SETI. Enjoy your holidays and here's hoping we won't have to see you before Monday.


PROUD MEMBER OF Team Starfire World BOINC
ID: 1058888 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 1058895 - Posted: 22 Dec 2010, 22:51:50 UTC
Last modified: 22 Dec 2010, 22:52:27 UTC

Merry Christmas to the
Entire Staff at Boinc/Seti


You earned it and thanks......................from all of us!
Boinc....Boinc....Boinc....Boinc....
ID: 1058895 · Report as offensive
KB7RZF
Volunteer tester
Avatar

Send message
Joined: 15 Aug 99
Posts: 9549
Credit: 3,308,926
RAC: 2
United States
Message 1058898 - Posted: 22 Dec 2010, 22:57:45 UTC

Nice work Matt and folks at the lab. From my family to yours, we wish you a very Merry Christmas!
ID: 1058898 · Report as offensive
Profile Dirk Sadowski
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 1058935 - Posted: 23 Dec 2010, 0:46:25 UTC - in response to Message 1058854.  

Matt, thanks for the news!


Merry Christmas and happy and relaxing holidays to the SETI@home crew, all members, not members and aliens out there!



ID: 1058935 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51478
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1058939 - Posted: 23 Dec 2010, 0:57:25 UTC
Last modified: 23 Dec 2010, 0:57:55 UTC

Simply ROFLMAO....

Matt, I have not read such a hilarious bit in years.
Excepting the part where you thought you had crashed the whole closet...LOL.
Even sys admins have a good day once in a while....

You and the whole Seti staff have a great holiday.
If it breaks itself while you are celebrating, it'll wait.
The birth of Jesus only comes around once a year.

Merry for Chirstmas...from Mark and the kitties.

Meow.
"Time is simply the mechanism that keeps everything from happening all at once."

ID: 1058939 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 1058946 - Posted: 23 Dec 2010, 1:13:36 UTC

Funny aside #1: at the exact moment I was finally comfortable enough to issue the command that blew away the old raid device from my home terminal, the network connectivity in my house randomly disappeared. Needless to say this incited minor panic as I suddenly couldn't reach any SETI servers and thought I must have somehow locked up the whole server closet. Eventually I figured out it was a local problem, my home router rebooted itself and I was back in business. Phew.


This is so cool! Even the best get bitten once in a while.


Boinc....Boinc....Boinc....Boinc....
ID: 1058946 · Report as offensive
Profile Bill G Special Project $75 donor
Avatar

Send message
Joined: 1 Jun 01
Posts: 1282
Credit: 187,688,550
RAC: 182
United States
Message 1058982 - Posted: 23 Dec 2010, 2:58:54 UTC - in response to Message 1058946.  

Thanks for the update and enjoy the holiday.

SETI@home classic workunits 4,019
SETI@home classic CPU time 34,348 hours
ID: 1058982 · Report as offensive
seti@elrcastor.com
Volunteer tester

Send message
Joined: 30 Jan 00
Posts: 35
Credit: 4,879,559
RAC: 0
United States
Message 1059023 - Posted: 23 Dec 2010, 5:56:53 UTC

is the beta project going to be brought back up? it's kicking up a message it's still down for maintenance.
ID: 1059023 · Report as offensive
B-Man
Volunteer tester

Send message
Joined: 11 Feb 01
Posts: 253
Credit: 147,366
RAC: 0
United States
Message 1059030 - Posted: 23 Dec 2010, 7:02:17 UTC

I'll second the question about the Beta project. Any plans to bring it back online.

2nd Hope the staff has a wonderful christmas holiday. May it all run smoothly over the weekend with no staff intervention needed.
ID: 1059030 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34382
Credit: 79,922,639
RAC: 80
Germany
Message 1059034 - Posted: 23 Dec 2010, 7:45:32 UTC


Merry Christmas Matt, Jeff and all others at the lab.



With each crime and every kindness we birth our future.
ID: 1059034 · Report as offensive
tbret
Volunteer tester
Avatar

Send message
Joined: 28 May 99
Posts: 3380
Credit: 296,162,071
RAC: 40
United States
Message 1059056 - Posted: 23 Dec 2010, 9:26:54 UTC - in response to Message 1058854.  

Rest ye merry, gentlemen.
ID: 1059056 · Report as offensive
Profile Helli_retiered
Volunteer tester
Avatar

Send message
Joined: 15 Dec 99
Posts: 707
Credit: 108,785,585
RAC: 0
Germany
Message 1059075 - Posted: 23 Dec 2010, 10:48:25 UTC - in response to Message 1058854.  

...

That's pretty much it for this week. It's officially the Xmas holiday starting tomorrow. Whatever you choose to do, enjoy! I'll be checking in from home over the "time off," but we're hoping it's kind of a "set it and forget it" kind of weekend instead of an "oh well everything will be down until we return on Monday" kind of weekend.

- Matt


LOL

We all hope the second part become not true. :D

However - Merry Christmas to you, your Family and the whole Staff!

Helli
A loooong time ago: First Credits after SETI@home Restart
ID: 1059075 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Striped (Dec 22 2010)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.