In and Out of Sorts (Jan 28 2008)

Message boards : Technical News : In and Out of Sorts (Jan 28 2008)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 705388 - Posted: 28 Jan 2008, 21:28:05 UTC

Things are running more or less smoothly. The workunit/result traffic was fairly high over the weekend, but consistent and below our current cap, so no major faults there. Our active user count is still slowly climbing but the acceleration of growth is negative (at least until we have another press releases or "reminder" e-mails are sent out). Since various index builds (and removals of seemingly unused indexes) the MySQL database is masterfully handling everything we give it. The router upgrade is still in limbo.

One odd thing was our "feeder" polarity problem reared its ugly head again. Reminder: we have two scheduling/upload servers (bruno and ptolemy) each given a separate queue of work to send to our participants. If all is well, they should send out work at the same rate. However, in the past this wasn't always the case. DNS favoritism was causing one queue to run out faster than the other, causing errant "no work from project" messages given to half the clients. This was fixed with software load balancing on top of DNS. However, this time around it seems the increased traffic tickled an actual, particular disparity between the two. That is, bruno writes uploaded result files to directly attached RAID storage, while ptolemy writes to bruno's storage over NFS. We seemed to hit a "too many files open" limit on bruno, and therefore bumped up the maximum on that. We'll see if that helps.

In case you haven't noticed, I un-DNS-aliased one of the three setiathome.berkeley.edu webservers last week, and another this morning. All public web traffic is theoretically aimed solely at our new 1U dual opteron system, and it's doing great. However, DNS rollout takes forever (even with time-to-live set for 5 minutes) - it will take a week or so for those old aliases to disappear. The old web servers (kosh and penguin) were wonderful sparc/solaris systems but are approaching 8 years old and therefore are relatively physically big and slow. We'll pull them out of the closet to make way for more modern systems - like bruno. Yeah, bruno is still sitting in our secondary lab, connected to the systems in our closet via some funky switching around the building. It will be great to it on the same single switch as everything else.

Other plans for the week: We're upgrading the fedora core levels on several systems, including our science database systems. We have already tested similar upgrades on our more-expendable desktops with little trouble. However, we will proceed with great caution given many terabytes of data are involved on the database servers - full recovery would be painful, to put it mildly.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 705388 · Report as offensive
Nick Fox

Send message
Joined: 5 Jan 04
Posts: 46
Credit: 2,834,922
RAC: 0
United Kingdom
Message 705604 - Posted: 29 Jan 2008, 17:22:51 UTC

Thanks for the update Matt - keep up the good work!
ID: 705604 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 705646 - Posted: 30 Jan 2008, 0:30:42 UTC

Thanks for the Post Matt . . . it's much appreciated Sir!
BOINC Wiki . . .

Science Status Page . . .
ID: 705646 · Report as offensive

Message boards : Technical News : In and Out of Sorts (Jan 28 2008)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.