Wird (Dec 29 2008)


log in

Advanced search

Message boards : Technical News : Wird (Dec 29 2008)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1384
Credit: 74,079
RAC: 0
United States
Message 846566 - Posted: 29 Dec 2008, 23:56:24 UTC

One short holiday week is behind us, now here comes another one.

We did fairly well over the weekend, considering we were pretty much maxed out the whole time. The assimilator queue finally drained, thanks to splitters starting to chew on raw data files physically located on the new raw data storage server (as opposed to located on the same server as the science database), but also thanks to the validator queue falling behind.

In times of low resources we do have some knobs to turn to help squeeze more juice out of our embattled servers. Sometimes you have to roll up your sleeves (or, in this case, pull out a calculator) and determine what processes needs what resource, and which are claiming too much. After some investigation it was clear this time around we were giving httpd too much - and this is a tunable we have to adjust every so often, depending on how many people are connecting at any given time, and for how long - otherwise you have too many httpd listeners hanging out doing nothing eating up valuable memory/cpu. Anyway, long story short I reduced the number of validators from 6 to 4, moved the validator logs to a different filesystem (reduce i/o contention), and vastly reduced the number of httpd listeners. So far so good - that queue is draining (and therefore the assimilator queue is inflating again).

We will have the usual outage drill tomorrow, followed by another set of "days off."

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Alinator
Volunteer tester
Send message
Joined: 19 Apr 05
Posts: 4178
Credit: 4,647,982
RAC: 6
United States
Message 846582 - Posted: 30 Dec 2008, 0:26:22 UTC
Last modified: 30 Dec 2008, 0:27:01 UTC

LOL...

Well, from where I'm sitting, this Xmas/News Years period has been distressingly boring so far (knock on wood).

If it wasn't for the CUDA rollout and a couple of other items the boards would be deadly dull. ;-)

Alinator

Profile Virtual Boss*
Volunteer tester
Avatar
Send message
Joined: 4 May 08
Posts: 417
Credit: 6,108,448
RAC: 1,054
Australia
Message 846721 - Posted: 30 Dec 2008, 3:18:46 UTC

Seems to me that the extra 1TB of WU storage may have prevented more serious problems over the last few days.

May tomorrows outage go smoothly.

Profile Dr. C.E.T.I.
Avatar
Send message
Joined: 29 Feb 00
Posts: 15988
Credit: 683,400
RAC: 100
United States
Message 846783 - Posted: 30 Dec 2008, 6:16:04 UTC

. . . Thanks for the Post Update Matt

and, May You & Each of You @ Berkeley have an Excellent Weekend

and A Happy New Year . . .

Thanks for All that You have Each done in the Past Year [and All other Years as Well] -

it has been Appreciated . . .


Respectfully,
joanne & richard



____________
BOINC Wiki . . .

Science Status Page . . .

Lars Vindal
Send message
Joined: 19 Jun 03
Posts: 3
Credit: 955,837
RAC: 1,821
Norway
Message 847931 - Posted: 2 Jan 2009, 0:40:34 UTC

Nice to see you are getting better equipped to meet future demands. I hope this will help reducing some of the bottlenecks that have been popped up sometimes. :)

All good wishes for the new year to staff and crunchers alike!

Message boards : Technical News : Wird (Dec 29 2008)

Copyright © 2014 University of California