Message boards :
Technical News :
Wird (Dec 29 2008)
Message board moderation
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
One short holiday week is behind us, now here comes another one. We did fairly well over the weekend, considering we were pretty much maxed out the whole time. The assimilator queue finally drained, thanks to splitters starting to chew on raw data files physically located on the new raw data storage server (as opposed to located on the same server as the science database), but also thanks to the validator queue falling behind. In times of low resources we do have some knobs to turn to help squeeze more juice out of our embattled servers. Sometimes you have to roll up your sleeves (or, in this case, pull out a calculator) and determine what processes needs what resource, and which are claiming too much. After some investigation it was clear this time around we were giving httpd too much - and this is a tunable we have to adjust every so often, depending on how many people are connecting at any given time, and for how long - otherwise you have too many httpd listeners hanging out doing nothing eating up valuable memory/cpu. Anyway, long story short I reduced the number of validators from 6 to 4, moved the validator logs to a different filesystem (reduce i/o contention), and vastly reduced the number of httpd listeners. So far so good - that queue is draining (and therefore the assimilator queue is inflating again). We will have the usual outage drill tomorrow, followed by another set of "days off." - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Alinator Send message Joined: 19 Apr 05 Posts: 4178 Credit: 4,647,982 RAC: 0 |
LOL... Well, from where I'm sitting, this Xmas/News Years period has been distressingly boring so far (knock on wood). If it wasn't for the CUDA rollout and a couple of other items the boards would be deadly dull. ;-) Alinator |
Virtual Boss* Send message Joined: 4 May 08 Posts: 417 Credit: 6,440,287 RAC: 0 |
Seems to me that the extra 1TB of WU storage may have prevented more serious problems over the last few days. May tomorrows outage go smoothly. |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
. . . Thanks for the Post Update Matt and, May You & Each of You @ Berkeley have an Excellent Weekend and A Happy New Year . . . Thanks for All that You have Each done in the Past Year [and All other Years as Well] - it has been Appreciated . . . Respectfully, joanne & richard BOINC Wiki . . . Science Status Page . . . |
Lars Vindal Send message Joined: 19 Jun 03 Posts: 7 Credit: 1,849,539 RAC: 0 |
Nice to see you are getting better equipped to meet future demands. I hope this will help reducing some of the bottlenecks that have been popped up sometimes. :) All good wishes for the new year to staff and crunchers alike! |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.