Major Power Outage at SSL

Message boards : News : Major Power Outage at SSL

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 9 · Next

AuthorMessage
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45918
Credit: 815,249,639
RAC: 124,952
United States
Message 1233192 - Posted: 19 May 2012, 2:33:09 UTC - in response to Message 1233186.  
Last modified: 19 May 2012, 2:33:26 UTC


Side note: I see the AP data is showing pretty much non-existant, is this a residual problem or just a side effect of Jocelyn apparently taking some time off?

I think it's because the AP data is shown is still for the little bit of v505 still in the wild.
It does not reflect the v6 info yet.
Cats.....what more does one need?

Have made friends in this life.
Most were cats.

ID: 1233192 · Report as offensive
Profile celttooth
Avatar

Send message
Joined: 21 Nov 99
Posts: 26503
Credit: 27,673,529
RAC: 6,546
Canada
Message 1233194 - Posted: 19 May 2012, 2:47:09 UTC

Someone must be working like a Trojan to get everything back on, good work and thanks for the effort. My boxes still have lots of work unites to slave over.

ID: 1233194 · Report as offensive
Profile Boomer
Volunteer tester

Send message
Joined: 1 Jun 09
Posts: 41
Credit: 20,357,203
RAC: 9,796
United States
Message 1233229 - Posted: 19 May 2012, 4:46:09 UTC

Hey Matt,

I was worried for a while there that someone pulled the plug on this project. Not that anyone would ever do that; right?

I am really happy to see SETI@HOME come back online. Thanks for doing a great job and for keeping us informed. Most technical folks do not communicate very well. But, you have great technical skills AND communication skills which is a very rare combination.

Kudos and kudos.

Regards, Boomer


ID: 1233229 · Report as offensive
Profile ivan
Volunteer tester
Avatar

Send message
Joined: 5 Mar 01
Posts: 780
Credit: 232,648,560
RAC: 81,793
United Kingdom
Message 1233342 - Posted: 19 May 2012, 10:09:56 UTC - in response to Message 1233065.  

They do have UPSes and remote control power switches (or those are part of a UPS or two.. it allows them to SSH in and power cycle a machine). I remember reading a few years ago that all the UPSes were basically just power strips at one point as the battery capacity in them was basically zero.

Even with a large 3000VA UPS, having 4-6 servers on it, some of which have 30+ HDDs, you're talking 5 minutes on brand new batteries for a graceful shutdown, and you can't just tell them all to shut down all at once. Some have to go down before others.
I know that this was kind of a freak occurrence, but it does appear to show a weakness in the systems. Would it possibly make sense to direct some of our fund raising contributions towards an even more robust UPS setup? Most UPS's of this size allow you to daisy chain multiple batteries together to allow more time to shut things down before they run out of juice. This makes sense to me, what do you guys there in the thick of it think?

Actually, my thoughts wandered to providing a backup diesel generator! :-) Probably too expensive I know... In fact, their being on the wet coast of the US leads to other problems -- I heard recently about such a backup system kicking in after an earthquake (might have been last year's Japanese disaster) and maintaining systems perfectly for a half-hour or so, until the years of sludge at the bottom of the fuel tanks, stirred up by the shaking, clogged the fuel filters!

ID: 1233342 · Report as offensive
Terror Australis
Volunteer tester

Send message
Joined: 14 Feb 04
Posts: 1790
Credit: 225,320,154
RAC: 10,479
Australia
Message 1233484 - Posted: 19 May 2012, 16:12:41 UTC - in response to Message 1233088.  

Run time at full load went from a few minutes to about 6 hours.

Six hours? you must be on the Casuarina sub-station ;-)

T.A.

ID: 1233484 · Report as offensive
Larry Coolidge
Avatar

Send message
Joined: 16 May 99
Posts: 19
Credit: 30,396,273
RAC: 21,221
United States
Message 1233603 - Posted: 19 May 2012, 21:14:28 UTC - in response to Message 1233342.  

A gas turbine would resolve that issue and you have less polution generated as well [/quote]


ID: 1233603 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 7486
Credit: 91,108,265
RAC: 46,447
Australia
Message 1233643 - Posted: 19 May 2012, 22:00:51 UTC - in response to Message 1233484.  

Six hours? you must be on the Casuarina sub-station ;-)

Nah, but even without those episodes the power goes of more frequently & for longer periods of time than it used to.
Best way to save money according to the bean counters- cut back on maintenance. If it's working why spend money on it?

Grant
Darwin NT

ID: 1233643 · Report as offensive
Nick
Avatar

Send message
Joined: 11 Oct 11
Posts: 4344
Credit: 2,621,009
RAC: 548
United Kingdom
Message 1233683 - Posted: 19 May 2012, 23:21:22 UTC - in response to Message 1232936.  

Thanks guys for all the work in getting it back up and running again. However, it is a shame the Berkley Uni web site could not find space in it's news to say what had happened to seti@home!
Does not boad well for inter-stellar contact if we can't get the communications right here does it?


It did report the power fault, Rt'; you just had to fish around to find it
though.

The Kite Fliers

--------------------
Kite fliers: An imaginary club of solo members, those who don't yet
belong to a formal team so "fly their own kites" - as the saying goes.

ID: 1233683 · Report as offensive
Profile Nomad

Send message
Joined: 24 Apr 12
Posts: 2
Credit: 279,123
RAC: 0
United States
Message 1234970 - Posted: 22 May 2012, 9:56:37 UTC

When will the Astro Pulse Tasks be ready to download? Have not received any since the outage.


NOMAD

ID: 1234970 · Report as offensive
David SProject Donor
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 17042
Credit: 20,948,557
RAC: 6,204
United States
Message 1235038 - Posted: 22 May 2012, 13:17:05 UTC - in response to Message 1234970.  

When will the Astro Pulse Tasks be ready to download? Have not received any since the outage.

It looks like there are some now.

(Currently, you can't go by what it says about AP tasks in the field, etc., on the Server Status Page. But the page also shows some of the AP splitters running, which is good info.)

By the way, you would do better to ask questions such as this in the Number Crunching forum.

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.


ID: 1235038 · Report as offensive
Profile ShadowStalker

Send message
Joined: 22 Dec 99
Posts: 4
Credit: 70,334
RAC: 0
United States
Message 1238961 - Posted: 31 May 2012, 17:04:01 UTC - in response to Message 1232771.  

need to have a independent website outside of seti for when power goes out again where can get updates of whats going on or send out mass email


ID: 1238961 · Report as offensive
Dave
Avatar

Send message
Joined: 29 Mar 02
Posts: 777
Credit: 25,001,396
RAC: 0
United Kingdom
Message 1238985 - Posted: 31 May 2012, 17:45:50 UTC

Updates were posted in the Technical News section & the front page.

ID: 1238985 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 45918
Credit: 815,249,639
RAC: 124,952
United States
Message 1238988 - Posted: 31 May 2012, 17:49:05 UTC - in response to Message 1238985.  

Updates were posted in the Technical News section & the front page.

I can see his point....
If one has not happened to have read the news before the outage.
Once things are down, there is no source of news direct from the Seti project with all the main servers offline.
Cats.....what more does one need?

Have made friends in this life.
Most were cats.

ID: 1238988 · Report as offensive
Profile Chris SCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 19 Nov 00
Posts: 38183
Credit: 21,373,373
RAC: 27,760
United Kingdom
Message 1239023 - Posted: 31 May 2012, 18:33:57 UTC

I would suggest that Boincstats could be used for this, they have a project news update section. Alternatively messages could be posted on other Boinc projects, and the word spread that way.

We need to keep a sensible balance here. It is NOT the end of the world if Seti goes down without prior knowledge for a couple of days. To those few who feel that it is, there are other Boinc projects to support in the meantime. There will be a good reason for the downtime and it will be back ASAP.


Those are my principles, and if you don't like them ... well, I have others.
Groucho Marx 1895-1977

I also have mine, and if you don't like them ... tough, live with it.
Chris S 2016

Member of UCB Charter Hill Society

ID: 1239023 · Report as offensive
Profile Cliff Harding
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 18 Aug 99
Posts: 1265
Credit: 73,356,632
RAC: 19,855
United States
Message 1239031 - Posted: 31 May 2012, 18:43:36 UTC - in response to Message 1239023.  
Last modified: 31 May 2012, 18:46:22 UTC

I saw via the Cricket Graphs page that SETI was back up after the normal scheduled outage on Tuesday afternoon. When I checked the progress on my task reporting a few minutes later, it had taken a sudden nose dive. I COULD NOT GET TO THE HOME PAGE. I had to check the Lunatics site to get any information on this outage. If this was a scheduled outage there must be a better way of notifying people, how about a broadcast notice that will appear on the NOTICE tab.




I don't buy computers, I build them!!

ID: 1239031 · Report as offensive
LetiCern

Send message
Joined: 3 Oct 99
Posts: 15
Credit: 11,362,831
RAC: 2,240
United States
Message 1239036 - Posted: 31 May 2012, 18:46:34 UTC

My 2 cents: Seti and Bionic apps are hands-off on my systems so I hardly ever look at 'em. I may check Bionic twice a month. So when I saw uploads stacking up I could not locate any news. At first I checked my ISP connection finally I checked connection to berkeley.edu so when that failed I stopped checking.

It would have been nice to have BIONIC post a notice 'bout outage.

Thanx guys for all your work (never mind the little SNAFU).


ID: 1239036 · Report as offensive
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 18642
Credit: 21,467,225
RAC: 20,028
United States
Message 1239053 - Posted: 31 May 2012, 19:19:03 UTC - in response to Message 1239031.  

I saw via the Cricket Graphs page that SETI was back up after the normal scheduled outage on Tuesday afternoon. When I checked the progress on my task reporting a few minutes later, it had taken a sudden nose dive. I COULD NOT GET TO THE HOME PAGE. I had to check the Lunatics site to get any information on this outage. If this was a scheduled outage there must be a better way of notifying people, how about a broadcast notice that will appear on the NOTICE tab.

Perhaps, but the notice about the planned outage was, and still is with time stamp, on the home page before the normal maintenance outage.

ID: 1239053 · Report as offensive
Profile Enzeder
Avatar

Send message
Joined: 30 Jan 01
Posts: 6
Credit: 49,310,487
RAC: 6,456
New Zealand
Message 1239055 - Posted: 31 May 2012, 19:20:58 UTC - in response to Message 1238985.  

Updates were posted in the Technical News section & the front page.


Can you point us rookies to the updates you're referring to? I don't see any updates since the power went out (29th), and most of the time I tried to look, the website seemed to be down.

It would be nice if SETI/BOINC could simply mirror their homepage to another location. Those of us that missed that 90 min window could then satisfy our curiosity and know the problem isn't on our end. Perhaps another UC Berkeley server (on the other side of campus?), but in an ideal world something offsite would be nice.

I'm a little surprised that there weren't even any announcements on the news section of the UC Berkeley homepage (that I could find).

ID: 1239055 · Report as offensive
Profile Ex
Volunteer tester
Avatar

Send message
Joined: 12 Mar 12
Posts: 2895
Credit: 2,079,557
RAC: 408
United States
Message 1239072 - Posted: 31 May 2012, 19:43:50 UTC
Last modified: 31 May 2012, 19:48:27 UTC

David Anderson made a post in the Boinc boards, but not until a day before the outage. I too had no idea we were to be down for a couple days. Luckily I asked on the Boinc boards what was going on, just before they too went lights-out.

Some more notice here would have been cool I think.

(And, I still don't quite understand why the forum's servers have anything to do with the scientific servers....... I think it's kind of- erm - dinosaurish that the forums have to go down for scientific maintenance)


-Dave #2

3.2.0-33

ID: 1239072 · Report as offensive
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 18642
Credit: 21,467,225
RAC: 20,028
United States
Message 1239075 - Posted: 31 May 2012, 19:58:36 UTC - in response to Message 1239055.  

Updates were posted in the Technical News section & the front page.


Can you point us rookies to the updates you're referring to? I don't see any updates since the power went out (29th), and most of the time I tried to look, the website seemed to be down.

It would be nice if SETI/BOINC could simply mirror their homepage to another location. Those of us that missed that 90 min window could then satisfy our curiosity and know the problem isn't on our end. Perhaps another UC Berkeley server (on the other side of campus?), but in an ideal world something offsite would be nice.

I'm a little surprised that there weren't even any announcements on the news section of the UC Berkeley homepage (that I could find).

See the word "HOME" in the upper left. Press it. It will take you to the notice posted at 17:11:12 UTC that there was an outage in the works. That notice was up from then until the system was taken down. And it was up all the time the maintenance outage was going on.

I suspect the lab didn't get much more notice than this.

ID: 1239075 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · 6 . . . 9 · Next

Message boards : News : Major Power Outage at SSL


 
©2016 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.