The Return of Bruno (Sep 30 2008)

Message boards : Technical News : The Return of Bruno (Sep 30 2008)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 813523 - Posted: 30 Sep 2008, 23:28:47 UTC

We had an extended outage today (more than the regular 3-4 hour database maintenance outage) to finally upgrade one of our core servers, bruno. Usually the OS upgrades are trivial, however this particular machine required a little extra TLC, due to its functional importance, as well as its unique (but admittedly not that unusual) hardware configuration. In regards to the latter, we basically put off upgrading this system until a modern day OS would automatically support its fibre channel card (as opposed to us having to compile drivers into the kernel, etc... blech...).

Anywho... there were no major failures during the long procedure (which included backing everything up, reconfiguring root RAID devices (while trying not to destroy others), then resetting all the network/RAID/apache/etc. services). It still took longer than it should due to a steady stream of minor annoyances (installer crash on first attempt, missing sym links that had to be discovered/recreated, missing packages to be installed, having to recompile every BOINC service due to standard library changes). Doesn't matter - it's done. Or at least done enough - there are still some screws to tighten which I'll tackle later.

So, we'll be catching up for a while. If at first you don't connect, let your client try again later.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 813523 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 813525 - Posted: 30 Sep 2008, 23:32:46 UTC - in response to Message 813523.  

Thanks for the update, as always.

Claggy
ID: 813525 · Report as offensive
DJStarfox

Send message
Joined: 23 May 01
Posts: 1066
Credit: 1,226,053
RAC: 2
United States
Message 813538 - Posted: 1 Oct 2008, 0:14:21 UTC - in response to Message 813523.  

If you're getting a fibre channel card with some storage behind it, bruno should scream with speed!
ID: 813538 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 813697 - Posted: 1 Oct 2008, 16:46:43 UTC

Matt,

Early warning! I think the .xml stats upload is on the fritz again... or the figures in it are frozen.
.

Hello, from Albany, CA!...
ID: 813697 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 813715 - Posted: 1 Oct 2008, 18:06:06 UTC - in response to Message 813697.  

Early warning! I think the .xml stats upload is on the fritz again... or the figures in it are frozen.


Oops - you're right. I just recompiled/installed that, too - should kick in sometime today.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 813715 · Report as offensive
Arthur L. Smith

Send message
Joined: 17 Apr 02
Posts: 28
Credit: 244,050,922
RAC: 9
United States
Message 814019 - Posted: 2 Oct 2008, 14:45:55 UTC - in response to Message 813715.  

Matt,

Ever since the upgrade of Bruno on Tuesday I am having problems with all of my machines (50 or 60 of them). I get either "HTTP error" when reporting or "HTTP Internal Server" errors when trying to report. And none of them want to pull down any new work. They get a "reached daily quota of 200 results" error even though some have not had any work in a few days. I don't know if Bruno has anything to do with it but it all seemed to start happening after the outage on Tuesday. Any ideas?

Thanks

ID: 814019 · Report as offensive
Profile the silver surfer
Avatar

Send message
Joined: 24 Feb 01
Posts: 131
Credit: 3,739,307
RAC: 0
Austria
Message 814046 - Posted: 2 Oct 2008, 16:21:03 UTC - in response to Message 814019.  

Matt,

Same problem as above since Tuesday with HTTP errors when uploading, downloading new work isn`t affected.

Regards,

Kurt

ID: 814046 · Report as offensive
PhonAcq

Send message
Joined: 14 Apr 01
Posts: 1656
Credit: 30,658,217
RAC: 1
United States
Message 814337 - Posted: 3 Oct 2008, 9:50:44 UTC

ditto.

also, the server status page is now readable from 10K feet (i.e. the default font has changed making the page cosmetically challenged)
ID: 814337 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 814341 - Posted: 3 Oct 2008, 10:17:49 UTC


No problems here.
NB- keep in mind that approx every 2 hours (give or take an hour) there is a big surge in network traffic- at these times uploads, downloads & scheduler requests will often result in an error or timeout. Once the traffic is back to normal levels, there are no communication problems & all is good.
Grant
Darwin NT
ID: 814341 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 814517 - Posted: 3 Oct 2008, 19:48:52 UTC - in response to Message 814337.  

also, the server status page is now readable from 10K feet (i.e. the default font has changed making the page cosmetically challenged)


Yeah - the server status page is generated separately from the rest of the web site, and CSS stuff has changed recently, so I need to sync all that up at some point. It's on my list.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 814517 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 814731 - Posted: 4 Oct 2008, 14:24:25 UTC - in response to Message 814517.  

also, the server status page is now readable from 10K feet (i.e. the default font has changed making the page cosmetically challenged)


Yeah - the server status page is generated separately from the rest of the web site, and CSS stuff has changed recently, so I need to sync all that up at some point. It's on my list.

- Matt

Must be a long list......LOL.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 814731 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 814815 - Posted: 4 Oct 2008, 19:46:18 UTC - in response to Message 814731.  

also, the server status page is now readable from 10K feet (i.e. the default font has changed making the page cosmetically challenged)


Yeah - the server status page is generated separately from the rest of the web site, and CSS stuff has changed recently, so I need to sync all that up at some point. It's on my list.

- Matt

Must be a long list......LOL.

Does it ever get shorter?


BOINC WIKI
ID: 814815 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20289
Credit: 7,508,002
RAC: 20
United Kingdom
Message 815033 - Posted: 5 Oct 2008, 12:25:48 UTC - in response to Message 814815.  
Last modified: 5 Oct 2008, 12:29:43 UTC

Must be a long list......LOL.

Does it ever get shorter?

Does your list ever get shorter?



+
+
+
+
+
+
+
+
+
Big Stats!
+
+
+
+
+
+
+
+
+



Happy crunchin',

(And trimmer sig lines? Just a summary perhaps?)

Martin

("Visual real-estate challenged..." :-( )
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 815033 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 815235 - Posted: 6 Oct 2008, 0:05:12 UTC - in response to Message 815033.  

Must be a long list......LOL.

Does it ever get shorter?

Does your list ever get shorter?



+
+
+
+
+
+
+
+
+
Big Stats!
+
+
+
+
+
+
+
+
+



Happy crunchin',

(And trimmer sig lines? Just a summary perhaps?)

Martin

("Visual real-estate challenged..." :-( )

It fluctuate in size as the RAC of the slowest projects rise and fall above a RAC of 1.


BOINC WIKI
ID: 815235 · Report as offensive

Message boards : Technical News : The Return of Bruno (Sep 30 2008)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.