Another Day (Feb 27 2007)


log in

Advanced search

Message boards : Technical News : Another Day (Feb 27 2007)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 524252 - Posted: 27 Feb 2007, 21:31:09 UTC

Someday I hope another person from our project will start a thread on this forum. Until then, here's the next installment written by me. I just don't want to give people the impression that I run this show, or that I know everything, or that these messages offer a comprehensive vision of what goes on behind the scenes. I tend to leave stuff out that other people are working on.

The big task for today was upgrading isaac (the boinc.berkeley.edu web server among other things). We tried this last week but hit a roadblock when we discovered the internal drives (previously completely hidden behind hardware RAID) were half the size we hoped. We got new drives, and started the whole drill over again today.

And all was well until we configured the RAID using the new drives. I estimated the initial RAID configuration would only take 30 minutes, so I planned for an hour. Based on my software RAID experience, this seemed fair. I was wrong. The whole process ended up taking almost two hours. So be it.

But then we hit a couple snags trying to install the new OS. The optical drive on the system was broken (it won't eject the disc) so I used a USB-connected DVD drive. The installer booted and about halfway through complained it couldn't find the media. This was odd, as it was used the media to get this far. Basically, at this point during the install it was expecting a disk in the internal drive and refused to accept the USB drive. Even more mysterious is that I used this method to install the OS on another system without incident.

Sigh. Fine. I broke out my trusted paper clip and forced open the system drive and put the installer in there. It refused to boot. After Jeff and I scratched our heads for a minute I realized the stupid drive doesn't read DVDs - only CDs. The system isn't that old, so the fact it didn't have DVD-reading capability was startling, but we are seasoned professionals and learned long ago to expect the unexpected. Or at least accept the unexpected.

Our only option at this point was to install over the net, which is perfectly okay to do but slooooow. I was hoping to have the OS installed by now as I write this, but we'll be lucky to have it done within the next two hours. I got here early today in the hopes that we'd finish the whole project by the afternoon. Now we're going to have to let it sleep overnight and finish it tomorrow. No big deal - we have a stub page on a temporary server in its stead, but I just want to get this done already.

Meanwhile, we had the regular outage. No big news there, except a couple more steps were enacted for us to start replication. I'll let you know when that's in full swing. I also rebooted our Network Appliance file server. It hosts most of our home accounts, some data, our cvs repositories, and more. It's been a wonderful, robust server for many many years, but now I guess it's getting old and cranky. There were error messages clogging the displays and a power cycle seemed to clear that right up.

Oh yeah - Daylight Savings Time is going to change. What a hassle. I'm going to go around making sure ntp is working on all our systems. Not sure what is going to happen with all of our appliances that aren't under service, but the fine people at Snap Appliance hooked me up with free patches to take care of that particular file server (which hosts all the workunit downloads, as well as many of our data backups).

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Jan Schotsmans
Avatar
Send message
Joined: 27 Oct 00
Posts: 98
Credit: 92,693
RAC: 0
Belgium
Message 524286 - Posted: 27 Feb 2007, 22:53:36 UTC - in response to Message 524252.

From my experience with Software raid the change from 30 minutes to 2 hours isn't that surprising.

I was building a system a while back that I was setting up with 4 arrays of Soft RAID5.

All 16 drives were the exact same (they actualy had near seqential serial numbers :p, they came from 1 big box with 40 drives in it, Maxtor SATA's) 2 arrays finished building in about an hour and the last 2 took nearly 2 hours.

All arrays were build on the same system with the same chipsets for all the SATA controllers, there was only 1 thing I could identify in any of the chains that could explain the 2 last arrays to build slower and benchmarks afterwards showed nearly the exact same performance for all 4 arrays, so it wasn't like there was a problem with some of the drives.

The only thing I could identify for the difference in performance between the building and the actual benchmarks is that I rebooted the system after the arrays were built to check that they were correctly mounted on bootup.

So maybe before the reboot there was some IRQ related thing causing the slower performance or something not booting right in the kernel, but that was all difference I could find.

Profile Dr. C.E.T.I.
Avatar
Send message
Joined: 29 Feb 00
Posts: 15993
Credit: 690,597
RAC: 0
United States
Message 524298 - Posted: 27 Feb 2007, 23:07:54 UTC - in response to Message 524252.

Someday I hope another person from our project will start a thread on this forum. Until then, here's the next installment written by me. I just don't want to give people the impression that I run this show, or that I know everything, or that these messages offer a comprehensive vision of what goes on behind the scenes. I tend to leave stuff out that other people are working on.

The big task for today was upgrading isaac (the boinc.berkeley.edu web server among other things). We tried this last week but hit a roadblock when we discovered the internal drives (previously completely hidden behind hardware RAID) were half the size we hoped. We got new drives, and started the whole drill over again today.

And all was well until we configured the RAID using the new drives. I estimated the initial RAID configuration would only take 30 minutes, so I planned for an hour. Based on my software RAID experience, this seemed fair. I was wrong. The whole process ended up taking almost two hours. So be it.

But then we hit a couple snags trying to install the new OS. The optical drive on the system was broken (it won't eject the disc) so I used a USB-connected DVD drive. The installer booted and about halfway through complained it couldn't find the media. This was odd, as it was used the media to get this far. Basically, at this point during the install it was expecting a disk in the internal drive and refused to accept the USB drive. Even more mysterious is that I used this method to install the OS on another system without incident.

Sigh. Fine. I broke out my trusted paper clip and forced open the system drive and put the installer in there. It refused to boot. After Jeff and I scratched our heads for a minute I realized the stupid drive doesn't read DVDs - only CDs. The system isn't that old, so the fact it didn't have DVD-reading capability was startling, but we are seasoned professionals and learned long ago to expect the unexpected. Or at least accept the unexpected.

Our only option at this point was to install over the net, which is perfectly okay to do but slooooow. I was hoping to have the OS installed by now as I write this, but we'll be lucky to have it done within the next two hours. I got here early today in the hopes that we'd finish the whole project by the afternoon. Now we're going to have to let it sleep overnight and finish it tomorrow. No big deal - we have a stub page on a temporary server in its stead, but I just want to get this done already.

Meanwhile, we had the regular outage. No big news there, except a couple more steps were enacted for us to start replication. I'll let you know when that's in full swing. I also rebooted our Network Appliance file server. It hosts most of our home accounts, some data, our cvs repositories, and more. It's been a wonderful, robust server for many many years, but now I guess it's getting old and cranky. There were error messages clogging the displays and a power cycle seemed to clear that right up.

Oh yeah - Daylight Savings Time is going to change. What a hassle. I'm going to go around making sure ntp is working on all our systems. Not sure what is going to happen with all of our appliances that aren't under service, but the fine people at Snap Appliance hooked me up with free patches to take care of that particular file server (which hosts all the workunit downloads, as well as many of our data backups).

- Matt


@ Matt - phewww . . . got your hands full eh. Well, here's a Word of Praise for that which you well deserve Sir . . . get some rest and look @ it all with another day . . . Best to You (and the Others - Eric's been Posting a little along with Kevin . . . Keep up the good work Berkeley.

Sincerely, richard

Profile littlegreenmanfrommars
Volunteer tester
Avatar
Send message
Joined: 28 Jan 06
Posts: 1410
Credit: 934,158
RAC: 0
Australia
Message 525348 - Posted: 2 Mar 2007, 12:35:09 UTC

What nobody said... :)
____________

seti@elrcastor.com
Volunteer tester
Send message
Joined: 30 Jan 00
Posts: 35
Credit: 4,868,442
RAC: 0
United States
Message 525382 - Posted: 2 Mar 2007, 15:16:40 UTC - in response to Message 525348.

Have a cookie Matt!!
____________

Message boards : Technical News : Another Day (Feb 27 2007)

Copyright © 2014 University of California