Exodus (Dec 20 2011)

Message boards : Technical News : Exodus (Dec 20 2011)

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1441
Credit: 213,689
RAC: 0
United States
Message 1179613 - Posted: 20 Dec 2011, 23:35:51 UTC

Hello, world. Here's another random, non-comprehensive status update regarding our servers, quite possibly the last one before the end of the year.

So what are we dealing with lately. Well, carolyn (the mysql server) seems to have some funky memory. Or maybe it's an overactive watchdog in the kernel. Hard to tell, but the warning messages we're getting aren't given us the warm fuzzies. Operations are more or less normal, so we're just keeping an eye on it for the moment (don't really want to do any surgery before the holidays). Meanwhile, it did have a standard issues CPU lock last week, requiring a hard reset and database recovery. However annoying (and it seems the modern day linux kernels are getting more and more prone to this sort of misbehavior) it's so far easy enough to recover from after hard power cycling the machine. We have to do this a lot on bruno (the upload/compute server) quite often, and oscar (the informix database server) the other day as well. Every time we eventually recover just fine.

Also mysql-wise, we seem to be having performance issues that defy easy understanding and explanation. Maybe this is memory related (hope not), but probably just due to some black-box mysql internal bookkeeping. During some testing/tweaking I turned off the daily stats dump scripts, and (oops) forgot to turn them back on. So there was a period of 5-6 days without stats dumps. Sorry about that.

Another thing we have to keep an eye on is server closet temperature. Seems like (without clear notification) we are already in "holiday energy curtailment" mode. With less people around, lab-wide environmental controls (which assist our server closet cooling) are ramped down to save energy. Makes sense, but that still means temperatures rise in our closet, which isn't happy-making. So far they only went up a degree or two on average. Just one more thing to worry about.

Onto brighter news. The gang over at the GPU Users Group has been incredibly helpful to us thus far. They recently donated a 45 JBOD drive array, and as of today 28 2TB and 6TB drives (for this array and/or data transport to/from Arecibo). We'll use this, along with more drives to come and another whole server, to upgrade various parts of our current server backend in the near future: science database server storage, upload server, download server, and main BOINC admin/compute server... They are still collecting donations over at their site (see our
donation page for a link to their paypal-based donation site) going towards this new hardware.

Happy holidays, safe travels, and all that. See you in the new year...

- Matt


-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

ID: 1179613 · Report as offensive
ClaggyProject Donor
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4622
Credit: 46,337,052
RAC: 2,686
United Kingdom
Message 1179614 - Posted: 20 Dec 2011, 23:42:09 UTC - in response to Message 1179613.

Thanks for the update Matt, and for all the efforts,

Claggy

ID: 1179614 · Report as offensive
Thomas Arnold
Volunteer tester

Send message
Joined: 14 May 99
Posts: 56
Credit: 61,046,144
RAC: 0
United States
Message 1179616 - Posted: 20 Dec 2011, 23:56:34 UTC

+1

Thank you very much for the update. They are always appreciated.

And Happy Holidays, Bah Humbug, and all that too.

And Here's to all the best for our Setiathome community for the coming year. We hope to make a difference.

Tom


ID: 1179616 · Report as offensive
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 18623
Credit: 21,399,296
RAC: 20,131
United States
Message 1179620 - Posted: 21 Dec 2011, 0:01:47 UTC

Thanks for the update.

Merry Christmas and a Happy New Year.


ID: 1179620 · Report as offensive
Bob Giel
Volunteer tester

Send message
Joined: 11 Jan 04
Posts: 74
Credit: 5,282,018
RAC: 281
United States
Message 1179657 - Posted: 21 Dec 2011, 4:10:32 UTC

Thanks for the update Matt.

You and the rest of the crew have a Merry Christmas and Happy New Year!

ID: 1179657 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 38176
Credit: 21,280,884
RAC: 27,967
United Kingdom
Message 1179681 - Posted: 21 Dec 2011, 10:11:38 UTC

Thanks for the update and the kind words for the GPU Group.

ID: 1179681 · Report as offensive
Profile MarkJProject Donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 08
Posts: 978
Credit: 33,082,888
RAC: 9,104
Australia
Message 1179684 - Posted: 21 Dec 2011, 10:46:23 UTC
Last modified: 21 Dec 2011, 10:47:03 UTC

Hi Matt,

Thanks for the updates. Its always interesting to find out what is happening behind the scenes.

Question:
The project is asking for one or more 240Gb (or larger) SSD's. Whatever happened with the Intel X25's? They must have died from too many IOPS by now. What are the new ones likely to be used for?

Cheers,
MarkJ

ID: 1179684 · Report as offensive
David SProject Donor
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 17038
Credit: 20,925,955
RAC: 5,997
United States
Message 1179736 - Posted: 21 Dec 2011, 15:02:40 UTC

Thanks, Matt. We always appreciate updates, the more detailed the better.

And have a happy holiday of your choice.


David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


ID: 1179736 · Report as offensive
Profile Slavac
Volunteer tester
Avatar

Send message
Joined: 27 Apr 11
Posts: 1932
Credit: 17,952,639
RAC: 0
United States
Message 1179810 - Posted: 21 Dec 2011, 20:10:31 UTC - in response to Message 1179684.

Hi Matt,

Thanks for the updates. Its always interesting to find out what is happening behind the scenes.

Question:
The project is asking for one or more 240Gb (or larger) SSD's. Whatever happened with the Intel X25's? They must have died from too many IOPS by now. What are the new ones likely to be used for?

Cheers,
MarkJ


Hey Mark:

I'll make a post over in the NC thread soon about the SSD's.


Executive Director GPU Users Group Inc. -
brad@gpuug.org

ID: 1179810 · Report as offensive
Profile RottenMutt
Avatar

Send message
Joined: 15 Mar 01
Posts: 1011
Credit: 230,274,184
RAC: 0
United States
Message 1181155 - Posted: 28 Dec 2011, 18:23:08 UTC

the project went down for a bit, someone must be lurking.


ID: 1181155 · Report as offensive
Tutankhamon "Communist"
Volunteer tester
Avatar

Send message
Joined: 1 Nov 08
Posts: 6081
Credit: 37,636,059
RAC: 16,157
Sweden
Message 1181158 - Posted: 28 Dec 2011, 18:41:26 UTC - in response to Message 1181155.

the project went down for a bit, someone must be lurking.



Main Page says:

Weekly Outage and Initial Catch Up
Because of the holiday schedule, we are conducting the weekly maintenance outage today (Wednesday, 12/28) 28 Dec 2011 | 15:41:41 UTC
This is a test of the Emergency Moron System. Had there been a real moron in the room, there would've been a small mushroom cloud in the place where the idiot had been standing.

ID: 1181158 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 2578
Credit: 34,680,169
RAC: 19,928
United States
Message 1181372 - Posted: 29 Dec 2011, 15:56:49 UTC

Umm, unless there's a good reason not to, could someone bring beta back up?


.

ID: 1181372 · Report as offensive
ClaggyProject Donor
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4622
Credit: 46,337,052
RAC: 2,686
United Kingdom
Message 1181422 - Posted: 29 Dec 2011, 18:02:21 UTC - in response to Message 1181372.

Umm, unless there's a good reason not to, could someone bring beta back up?

+1

Claggy

ID: 1181422 · Report as offensive
Profile PERPLEXER ~ Thomas Huettinger
Volunteer tester
Avatar

Send message
Joined: 25 Jan 05
Posts: 10
Credit: 329,602,987
RAC: 162,165
Germany
Message 1181538 - Posted: 30 Dec 2011, 3:46:24 UTC

oh oh oh

ID: 1181538 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 2578
Credit: 34,680,169
RAC: 19,928
United States
Message 1181615 - Posted: 30 Dec 2011, 17:22:25 UTC
Last modified: 30 Dec 2011, 17:23:09 UTC

still waiting on Beta — dot, dot, dot...


.

ID: 1181615 · Report as offensive
ClaggyProject Donor
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4622
Credit: 46,337,052
RAC: 2,686
United Kingdom
Message 1181619 - Posted: 30 Dec 2011, 17:49:40 UTC - in response to Message 1181615.
Last modified: 30 Dec 2011, 17:50:00 UTC

still waiting on Beta — dot, dot, dot...

I was hoping they'd left Beta down while they built v7 6.98 apps, But there's no sign of them yet, :-(

Claggy

ID: 1181619 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 2578
Credit: 34,680,169
RAC: 19,928
United States
Message 1181620 - Posted: 30 Dec 2011, 18:00:12 UTC - in response to Message 1181619.


I was hoping they'd left Beta down while they built v7 6.98 apps, But there's no sign of them yet, :-(

Claggy


last I have are 6.97 apps - 14 of which I'm trying to report!
.

ID: 1181620 · Report as offensive
Profile PERPLEXER ~ Thomas Huettinger
Volunteer tester
Avatar

Send message
Joined: 25 Jan 05
Posts: 10
Credit: 329,602,987
RAC: 162,165
Germany
Message 1182041 - Posted: 1 Jan 2012, 1:26:14 UTC - in response to Message 1179613.

me fall down...but me not want... HAPPY NEW YEAR !!!

ID: 1182041 · Report as offensive
musicplayer

Send message
Joined: 17 May 10
Posts: 1785
Credit: 842,842
RAC: 0
Message 1182050 - Posted: 1 Jan 2012, 1:57:45 UTC
Last modified: 1 Jan 2012, 1:58:16 UTC

Happy New Year!

Time is now 02:54 here. I had a bottle of vine last evening, so I decided to stay away from it tonight.

Apparently an accident with the public fireworks downtown. Fortunately no-one was injured.

And apparently some Chinese lights in the sky as well. Never mind.

ID: 1182050 · Report as offensive
Profile Ed. Krushenski (Mr. K.)

Send message
Joined: 30 May 99
Posts: 1
Credit: 8,065,988
RAC: 0
United States
Message 1182577 - Posted: 3 Jan 2012, 11:14:59 UTC - in response to Message 1179613.

Hope you all had a happy holiday.

Here's to the hope for MANY candidates this year. Also hope for solving of the wierd issues you have been having.

Perhaps its voodoo, or Gremlins ????

Seriously, Happy New Year to all you folks.

ID: 1182577 · Report as offensive

Message boards : Technical News : Exodus (Dec 20 2011)


 
©2016 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.