Sorry about the extended two-day website brown-out just now. The mysql database server crashed during the "re-org," so that had to be restarted, then it crashed *again*. We didn't get a successful backup out of the thing until last night. That's a little bit annoying, and a little bit worrisome.

Let's see.. it's been a while since I put forth a litany of server issues. Except for the a/c debacle last week everything has been more or less status quo, but this week there was extra shuffling. Allow me to elaborate:

There have been some interesting unexpected consequences due to these extended weekly outages. For example, the amount of results hanging out in the mysql database has pretty much doubled (growing slowly but consistently over the past two months), which is causing minor indigestion: the database backups and re-orgs take much longer, and workunits and results are hanging out on disk much longer (and filling up their respective disks). But also some power users are trying to return hundreds, perhaps thousands, of results in a single scheduler request. This last thing was an issue because these requests were failing due to an apache request-limit-size bottleneck, and then the scheduler itself would barf on it. Well, the thing is, up until this week the scheduler had been running on anakin - one of the last few 32-bit machines in our closet. A new scheduler was built and tested to work on 64-bit systems. Long story short, this week we moved the scheduler onto bane, which was an under-utilized 64-bit machine just handling one half of the workunit downloads. And moved bane's downloads onto anakin. This was done via ip address swapping, so no worries about DNS rollout. We'll try this out either today, or when we open the floodgates tomorrow. By the way, we're looking into the "ghost" issue. That might explain the aforementioned "result indigestion" or at least part of it.

Also the boinc.berkeley.edu server has been suffering from OS rot, getting hit by several simultaneous web spiders, and just plain getting outdated and outgrown. It has served us well, but we finally bit the bullet and moved all that functionality to a newer, faster, better system and so far so good.

Fairly soon I'm going to blow away the current filesystems on bambi now that marvin is the trusted Astropulse database server. This should be quick, though I expect some snags (we had trouble before on this system having the BIOS recognize the 3ware RAID volumes as bootable drives). Once that's done we'll start moving all of bruno's functionality to bambi, and finally retire bruno (another flailing, troublesome 32-bit machine).

We're still trying to nail down the exact specs of the new science database server - Jeff has been doing some additional research regarding CPU upgrades - but that'll get purchased really really soon I swear.

- Matt

---

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Thanks for the news Matt. Sounds like good things on the way.

---

Thank you for the updated info, Matt.
As is probably usual, more going on behind the scenes than some may realize.
Hope all the shuffling around of hardware proves to be the answer to some problems.

Anxiously awaiting new news on Oscar.

Meow meow.

---

"Time is simply the mechanism that keeps everything from happening all at once."

Looks like you will have to update the server status page again as well.

Thanks for the update Matt.

---

May all the transitions be smooth and painless and may Oscar be the bestest server ever!! Hope we got you all you need to cover Oscar, I'm not sure Kittyman could live through another fund drive! :-)

---

PROUD MEMBER OF Team Starfire World BOINC

Thanks for the update Matt,

Can you, along with Eric, Jeff and David, work out how get an Astropulse_v505 switch implemented, along with changing the scheduler messages to not mention Astropulse_v5,
Eric posted he had'nt realised the Astropulse_v505 switch hadn't been implemented in this post last September, thanks.

Claggy

Matt, thanks for the news!

---

Thanks for the update Matt much appreciated!

---

Thanks for the update.

---

Many thanks!

---

Join the PACK!

Thanks Matt

---

It's 100% certain. There is no intelligent life in this universe.

That's great news! I know I'm looking forward to see the upgraded hardware smooth things out in the system. Good work there.

thanks for the update.

---

SETI@home crew, it's look like something is wrong with the scheduler..

Number crunching : 'Let the games begin 9-24'

---

The Hard Disk Drive is full again, Can't upload or report.

---

Savoir-Faire is everywhere!
The T1 Trust, T1 Class 4-4-4-4 #5550, America's First HST

Matt:

Just curious-- as if you don't have enough to do putting out server fires-- how is it possible to have successfully downloaded a "master file" when the status returns indicate the servers are done? Are these files stored elsewhere in the system, or is my success just due to sporadic server operation?

Have you ever considered separate servers for dishing out unprocessed packets versus those for receiving processed packets? At least when the receiving servers failed, we'd still be able to retrieve new packets. At the moment, I am at a CPU standstill.

Keep up the good work, though.

WATERHOLE

---

Team Waterhole Administrator

Matt:

Just curious-- as if you don't have enough to do putting out server fires-- how is it possible to have successfully downloaded a "master file" when the status returns indicate the servers are done? Are these files stored elsewhere in the system, or is my success just due to sporadic server operation?

Have you ever considered separate servers for dishing out unprocessed packets versus those for receiving processed packets? At least when the receiving servers failed, we'd still be able to retrieve new packets. At the moment, I am at a CPU standstill.

Keep up the good work, though.

WATERHOLE

The "master file" is on the web server, so while these web pages can be accessed so can the "master file".

Have you ever considered separate servers for dishing out unprocessed packets versus those for receiving processed packets? At least when the receiving servers failed, we'd still be able to retrieve new packets.

Uploading of completed Tasks and downloading of new unprocessed Tasks ARE done by separate servers. The problem is in the Scheduling Server and Scheduling Process, which receive Reports of completed Tasks and control the assignment of new work.

This past week they swapped server functions, and somehow something got bollixed so that the Scheduler is not responding to requests. Until that gets sorted out, we can upload completed Tasks, but cannot Report them, or get new Tasks.

You can read more about the server swaps in the 1st message of this thread, and more about the problems everyone is having, in threads in the Number Crunching section.

At the moment, I am at a CPU standstill.

Keep up the good work, though.

WATERHOLE

You are in good company. I run only CPU tasks on two old Mac G4s, and both are dry, waiting to report and get new Tasks. Many of us are dry or about to be. If you are not running another BOINC project as backup, this weekend might be a good time to shut down and do some seasonal cleaning of your rigs.

Around Seti@Home, patience is not just a virtue, it is a requirement.

---

Donald
Infernal Optimist / Submariner, retired

As far as I am aware the people who look after Seti@Home are volunteers. I am amazed at what they manage to do in coping with and solving what must be highly stessful situations when there are problems. They all deserve some good recognition.

---

As far as I am aware the people who look after Seti@Home are volunteers. I am amazed at what they manage to do in coping with and solving what must be highly stessful situations when there are problems. They all deserve some good recognition.

No, all of the key people (Eric, Jeff, Matt, even Dr. Anderson) are employees of the UC Space Sciences Lab, and divide their time between S@H and other projects. They do spend a lot of their time keeping S@H running, including coming in on weekends and holidays to deal with casualties.

They ARE much appreciated by most of us, and that needs to be said more often.

---

Donald
Infernal Optimist / Submariner, retired