Message boards :
Technical News :
Loss Leader (Dec 16 2008)
Message board moderation
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
First and foremost, it's snowing outside. This doesn't happen very often around here. So today was an outage day - with one unexpected surprise: a visit from Court, systems administrator extraordinaire here in our lab a couple years back. Nice to see him again and catch up. The standard outage stuff was, well, standard. Allow me to remind our new readers: Weekly we "compress" the mysql databases (which bloat from continual inserts/deletes all week, much like disk fragmentation) and back them up. These databases contain all the user/host/team info, and who is working on which workunits - basically all the generic volunteer computing stuff. The science is all kept in a separate database (using an Informix engine) on a different server altogether. The latter doesn't suffer from the same bloat, so we can do simple no-frills backups to disk while the database is live, without much ado. In theory we could do the mysql dumps live as well, but we choose to take things down to ensure the master/replica databases are in sync, and allow us some regular downtime to take care of pending server tasks. For example... Today we finally turned off the old Network Appliance - a NAS server which worked fast and wonderfully, but (a) was only 3 Terabytes raw storage, (b) took up one third of our server closet, and (c) the individual disks have been failing at an increasing rate. We moved all of its functionality elsewhere already, so it was time to say goodbye. Jeff and I tore it apart shelf by shelf. Any sadness was lost in the joy of now having a completely empty rack full of completely useful shelves (we've had ridiculous problems finding racks/shelves that matched in the past). It's kind of funny the most useful part of that system at this point was its racks/shelves. We put all the recently donated Overland Storage servers into this now-empty rack (containing 10 Terabytes worth of storage), as well as anakin (the scheduling server), and there's still room for a lot more stuff. We still have to configure/employ all this new storage, but it's all plugged in and on line at least. Recovery from the outage is usually painful. Today seems a little worse. Part of that is our work-to-send queue is at zero and the splitters are waiting for some space to free up before creating new work. I also think server "bruno" is having result storage issues slowing things down (people are connecting okay, which they can push through the usual traffic jam). We might need to reconfigure/rebuild that RAID array sooner than later. I brought the mini video camera to make a quick video tour of our server closet, but the noise of all the fans is so loud it's basically worthless. I did take some low-quality still photos though - I'll get those up on the web someday. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Gary Charpentier Send message Joined: 25 Dec 00 Posts: 30651 Credit: 53,134,872 RAC: 32 |
Keep up the good work. I may be seeing that snow in a few hours down here in LA. |
Fred J. Verster Send message Joined: 21 Apr 04 Posts: 3252 Credit: 31,903,643 RAC: 0 |
|
Neil Blaikie Send message Joined: 17 May 99 Posts: 143 Credit: 6,652,341 RAC: 0 |
Good work as always and Thank you for at least spending some of your time recording the video / images even though the video is worthless. Saw on the news that you guys were getting snow, probably goes with the very very cold temperatures with windchill of -50 that the western Canadian provinces are currently getting. Here on the East coast of Canada, Montreal area, we are just waiting on 15+ cm's of snow coming tomorrow. Thank you for your information and glad you managed to get a lot done today. |
Jon (nanoreid) Send message Joined: 16 Aug 07 Posts: 643 Credit: 583,870 RAC: 0 |
Any idea when we will start seeing work packages again? My poor little laptop has nothing to do while I asleep now. Thanks Hopefully the cosmos is not trying to reverse the charges. Moderation in all things. |
Allie in Vancouver Send message Joined: 16 Mar 07 Posts: 3949 Credit: 1,604,668 RAC: 0 |
Thanks for the quick background info on what happens on outage day. I had known that things ‘went down’ for a few hours on every Tuesday but never really understood why. (I probably still have no real idea, but your disk-defrag simile is something that even us less technical crunchers can understand.) Also, on behalf of many others, we appreciate your occasional posting of pictures in the about/picture albums section. I check it out from time to time and find that it helps put a human face on things. :) Pure mathematics is, in its way, the poetry of logical ideas. Albert Einstein |
Zebra3 Send message Joined: 22 Oct 01 Posts: 186 Credit: 13,658,148 RAC: 0 |
I don't mean to burst your bubble Neil but the last time I looked Quebec was part of Upper Canada..as much as it chooses not to be and in no way could it be classed as the East Coast of Canada...Leave that up to us Bluenosers and the Newfies..I apoligize if that being too harsh, |
Neil Blaikie Send message Joined: 17 May 99 Posts: 143 Credit: 6,652,341 RAC: 0 |
True, my apology. Still it is going to snow and much needed to help the huge ice rink I spent a good part of today making. |
Gary Charpentier Send message Joined: 25 Dec 00 Posts: 30651 Credit: 53,134,872 RAC: 32 |
I brought the mini video camera to make a quick video tour of our server closet, but the noise of all the fans is so loud it's basically worthless. Make a call over to your TV people on campus and ask them if you can borrow a "Lav" short for lavaliere. It is the kind of mike that you clip on your collar or necktie. It should reduce that fan noise a bit and bring up your voice talking. If that isn't available then cheat and just shoot your talking part in a nice quiet room without echos and replace the spoiled audio. As long as your lips aren't on camera who is to know? |
JLDun Send message Joined: 21 Apr 06 Posts: 573 Credit: 196,101 RAC: 0 |
I brought the mini video camera to make a quick video tour of our server closet, but the noise of all the fans is so loud it's basically worthless.As long as your lips aren't on camera who is to know? Is it necessary for you to be on camera, or can a Commentary Track be added on later? |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
. . . Thanks for the Updates Matt - and Accolades to Each of You @ Berkeley [just edit the sound out - and, like others mention - Overdub where necessary] BOINC Wiki . . . Science Status Page . . . |
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
Make a call over to your TV people on campus and ask them if you can borrow a "Lav" short for lavaliere. In my other life I have 24 hour access to a full-fledged recording studio with tons of gear, and do lots of movie editing/production/scoring as well as regular ol' music recording etc. However, whether or not SETI will get any time on that work queue (to do sound overdubs) remains to be seen. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Dr.Argentum Send message Joined: 24 Nov 99 Posts: 6 Credit: 6,977,431 RAC: 0 |
Yikes! I can't let that one pass. Montreal was part of LOWER Canada, sited on the lower part of the St Lawrence River. I can't be too harsh as I could never remember who were the herring chokers... |
Neil Blaikie Send message Joined: 17 May 99 Posts: 143 Credit: 6,652,341 RAC: 0 |
I am British with Canadian Permanent Residency so technical it doesn't really matter which part of Canada it is in :-) Only thing that matters, is that Quebec is part of Canada (albeit for the foreseeable future at least). Really is quite annoying though doing computing jobs and having to switch languages all the time, you get used to it though over time. I have one client who insists on sending messages to me and my colleagues in french, when it quite clearly states that all correspondents to the server admins be in ENGLISH. He pays well though so can't complain. Someone attempting to run a DB with no knowledge of IBM Dynamic Servers (Informix based) is really annoying, worst of all the guy gets paid to do it and yet sub-contracts the work out! |
The Watery Tart Send message Joined: 24 Aug 00 Posts: 1 Credit: 197,230 RAC: 0 |
It's snowing in Vegas, and sticking! Global weirding, here we come... |
JLDun Send message Joined: 21 Apr 06 Posts: 573 Credit: 196,101 RAC: 0 |
It's snowing in Vegas, and sticking! Global weirding, here we come... Had snow here- and slipped INSIDE the house. (Just a few bumps...) |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
. . . while it's snowin' where it usually doesn't - we are gettin' Sping-Like Weather here in the Carolina's ladeedaladeeda ;) @ Matt - remember to provide a link to the Photographs of the 'closet' - OR more appropriately put - let us know 'when' You do ;) Thanks mate . . . added notes: 'most' of us left Quebec - when 'le separatist' movement took over Quebec [middle-late 70's] - and the majority of us ended up in California > my friends had just begun filmin' 'Scanners' @ that time ;) BOINC Wiki . . . Science Status Page . . . |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.