Borderland (Nov 23 2011)


log in

Advanced search

Message boards : Technical News : Borderland (Nov 23 2011)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 1173377 - Posted: 23 Nov 2011, 23:59:58 UTC

Before we disappear for the long Thanksgiving holiday weekend I figure I'd catch you up on a couple things.

Keeping up with good security practices I'm in OS upgrade mode around here. So far so good getting our machines up to the latest rev of Fedora (FC16) but I hit a couple snags with vader yesterday. Vader is one of the two download servers, as well as a general BOINC backend server - you may have noticed I moved some assimilators/splitters/etc. off of it yesterday before the upgrade.

Anyway, there was a bunch of tiny annoyances during the whole process that ate up my whole day. Things like messed of network configurations and such. I'm kind of peeved how much Fedora and linux has changed over the past couple of years. I don't need job security in the form of relearning fundamental changes to OSes that worked just fine a month ago. Long story short it seemed like the only way I could truly configure the network was to yum in an old version of the network configuration GUI and use that to create the proper startup scripts.

I had to reboot vader a lot during all this, and some more again this morning, but downloads are now back to normal (albeit dropping packets as usual since we're constantly maxed out). I also got some of the BOINC backend processes running on it again.

Meanwhile after the last couple of Tuesday outages we've had a hard time recovering in general. Those watching the traffic graphs may have noticed how depressed they were upon coming back on line. This was mostly mysql's fault. It's doing some kind of mysterious i/o and/or internal bookkeeping causing queries to take forever after our weekly outages. My suspicion is that we just need to reboot the mysql server to clear some pipes. It's been a while. We'll do that next Tuesday.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile SciManStevProject donor
Volunteer tester
Avatar
Send message
Joined: 20 Jun 99
Posts: 4826
Credit: 80,968,052
RAC: 34,021
United States
Message 1173378 - Posted: 24 Nov 2011, 0:25:23 UTC

Thank you so much for the update! These little tidbits of information mean the world to us. Have a very Happy Thanksgiving!

Steve
____________
Warning, addicted to SETI crunching!
Crunching as a member of GPU Users Group.
GPUUG Website

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4067
Credit: 32,904,494
RAC: 7,883
United Kingdom
Message 1173384 - Posted: 24 Nov 2011, 1:08:03 UTC - in response to Message 1173377.

Thanks for the update Matt, have a good Thanksgivings,

Claggy

msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38923
Credit: 578,798,699
RAC: 514,254
United States
Message 1173426 - Posted: 24 Nov 2011, 6:56:04 UTC

As always Matt, thanks for the background insights.

Have yourself a fine holiday...

And all the rest of the Seti staff as well.

Meow!
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Profile MikeProject donor
Volunteer tester
Avatar
Send message
Joined: 17 Feb 01
Posts: 23806
Credit: 32,629,143
RAC: 23,597
Germany
Message 1173437 - Posted: 24 Nov 2011, 8:44:34 UTC

Thanks for the update Matt.

Happy Thanksgiving.

____________

KB7RZF
Volunteer tester
Avatar
Send message
Joined: 15 Aug 99
Posts: 9463
Credit: 3,109,969
RAC: 2,116
United States
Message 1173448 - Posted: 24 Nov 2011, 9:47:48 UTC

Thanks for the news Matt, its greatly appreciated! I hope all of you guys in the lab have a wonderful holiday!
____________

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8465
Credit: 48,957,559
RAC: 75,411
United Kingdom
Message 1173477 - Posted: 24 Nov 2011, 15:26:57 UTC

@ Matt,

Interesting post by Oliver Bock - project administrator at Einstein, managing their (very busy) GPU download server:

A few months ago we noticed that Apache wasn't able to handle the BRP/FGRP download requests anymore and switched to lighttpd which turned to be more suitable for our specific setup, data type and access pattern. The load increased even further and we seem to have crossed a crucial threshold last week such that lighttpd also wasn't up to the task anymore. Various filesystem/network/daemon tests have revealed that the web server was in fact the bottleneck and we now moved to nginx, the very efficient web server that powers Facebook, WordPress, SourceForge and GitHub for instance (third, almost second, most popular web server).

msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38923
Credit: 578,798,699
RAC: 514,254
United States
Message 1173478 - Posted: 24 Nov 2011, 15:41:36 UTC - in response to Message 1173477.
Last modified: 24 Nov 2011, 15:47:45 UTC

@ Matt,

Interesting post by Oliver Bock - project administrator at Einstein, managing their (very busy) GPU download server:

A few months ago we noticed that Apache wasn't able to handle the BRP/FGRP download requests anymore and switched to lighttpd which turned to be more suitable for our specific setup, data type and access pattern. The load increased even further and we seem to have crossed a crucial threshold last week such that lighttpd also wasn't up to the task anymore. Various filesystem/network/daemon tests have revealed that the web server was in fact the bottleneck and we now moved to nginx, the very efficient web server that powers Facebook, WordPress, SourceForge and GitHub for instance (third, almost second, most popular web server).

It would be interesting if you could convey this information directly to Matt.
And even more interesting to see what his response would be.....

EDIT...
And I wonder what the price tag on the software is.
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1923
Credit: 9,756,466
RAC: 16,705
United States
Message 1173481 - Posted: 24 Nov 2011, 15:53:10 UTC
Last modified: 24 Nov 2011, 16:10:14 UTC

I realize that it's low priority, but... are the client connection stats ever going to get back online? it's been a year and a month (plus...) since they went off-line!

And (while I'm writing...) how about getting the "Multi-Beam Data Recorder Status" back online? - that's been off since May, IIRC...
____________
.

Profile jason_geeProject donor
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 24 Nov 06
Posts: 4964
Credit: 73,106,831
RAC: 15,270
Australia
Message 1173483 - Posted: 24 Nov 2011, 15:56:43 UTC - in response to Message 1173478.

EDIT...
And I wonder what the price tag on the software is.


Nginx is free & open source

Jason
____________
"It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change."
Charles Darwin

msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38923
Credit: 578,798,699
RAC: 514,254
United States
Message 1173484 - Posted: 24 Nov 2011, 16:09:14 UTC - in response to Message 1173483.

EDIT...
And I wonder what the price tag on the software is.


Nginx is free & open source

Jason

Cool beans....
Now if Matt would just consider it. See if it is compatible with the Seti server setup. And be willing to endure just a little more hair pulling to do a conversion if it would be a good fit.
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Profile kaseychief
Avatar
Send message
Joined: 3 Dec 07
Posts: 1643
Credit: 416,781
RAC: 0
United States
Message 1173486 - Posted: 24 Nov 2011, 16:15:07 UTC

Matt, thanks for all the hard work you do to
keep things runnin smoothly, It's time now
for you to go home & enjoy your turkey day.

Best Regards

Tom & Jo H.
____________

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8465
Credit: 48,957,559
RAC: 75,411
United Kingdom
Message 1173672 - Posted: 25 Nov 2011, 17:09:03 UTC - in response to Message 1173484.

EDIT...
And I wonder what the price tag on the software is.


Nginx is free & open source

Jason

Cool beans....
Now if Matt would just consider it. See if it is compatible with the Seti server setup. And be willing to endure just a little more hair pulling to do a conversion if it would be a good fit.

Oliver Bock has just posted an expanded version of his message board answer, to the "BOINC Projects" mailing list which is used for announcements to project administrators (not that they all read it). Hopefully, that might prompt some feedback/experiences from other BOINC server-minders.

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1923
Credit: 9,756,466
RAC: 16,705
United States
Message 1173791 - Posted: 26 Nov 2011, 2:49:34 UTC

Irregardless of what the "server status" page reads, I can't report... but I can upload, though...
____________
.

Profile arkaynProject donor
Volunteer tester
Avatar
Send message
Joined: 14 May 99
Posts: 3623
Credit: 48,548,989
RAC: 29,500
United States
Message 1173796 - Posted: 26 Nov 2011, 3:17:57 UTC - in response to Message 1173484.

EDIT...
And I wonder what the price tag on the software is.


Nginx is free & open source

Jason

Cool beans....
Now if Matt would just consider it. See if it is compatible with the Seti server setup. And be willing to endure just a little more hair pulling to do a conversion if it would be a good fit.


I know another forum that I visit also just changed over to it, loading much faster now.
____________

Neil L. CarterProject donor
Volunteer tester
Send message
Joined: 6 Dec 99
Posts: 53
Credit: 4,066,503
RAC: 5,402
United States
Message 1174004 - Posted: 27 Nov 2011, 3:58:45 UTC

Interesting that you're having the same experience with Fedora that caused me to basically abandon it, too many changes too fast.

You might want to consider Oracle Enterprise Linux (OEL), which is what I've moved to in my server room.

FAQ for Oracle Enterprise Linux
Main Web Site

It's completely free unless you need to buy a support contract. Since you've been using Fedora, that doesn't sound like a problem.

OEL is basically a clone of Red Hat Enterprise Linux, tweaked by Oracle. (see the URLs)

Hope this helps with your OS woes.

Happy Holidays!

Neil

N9JFE David SProject donor
Volunteer tester
Avatar
Send message
Joined: 4 Oct 99
Posts: 11162
Credit: 13,956,442
RAC: 12,400
United States
Message 1174273 - Posted: 28 Nov 2011, 14:46:25 UTC - in response to Message 1173378.

Thank you so much for the update! These little tidbits of information mean the world to us. Have a very Happy Thanksgiving!

Steve is absolutely right.

____________
David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


Message boards : Technical News : Borderland (Nov 23 2011)

Copyright © 2014 University of California