Major Power Outage at SSL


log in

Advanced search

Message boards : News : Major Power Outage at SSL

1 · 2 · 3 · 4 . . . 9 · Next
Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1391
Credit: 74,079
RAC: 10
United States
Message 1232771 - Posted: 18 May 2012, 15:57:18 UTC

There was a major power outage on Tuesday evening that affected several buildings here on campus including the entire Space Sciences Laboratory. Power has been restored this morning, and we are slowly getting the project back on line.
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1391
Credit: 74,079
RAC: 10
United States
Message 1232776 - Posted: 18 May 2012, 15:59:18 UTC

More detail (from Jon - another systems administrator here at the lab):

Power failed when there was a short in the power distribution lines that feed the hill buildings [including Space Sciences Laboratory, Math Science Research Institute, Lawrence Hall of Science]. The short occurred because of deterioration of the insulation on these power lines, which are buried in a conduit in the ground. The power distribution lines are thousands of feet long, originating in a substation at LBL below LHS. Most of this conduit only contain one power line, but we were lucky in that the section that failed had two power lines in it, so the electricians could just utilize the second unused line to restore electrical service. It was an arduous process finding just where the short was, since they had to disconnect one segment at a time and test it. This is why it took so long to find and repair.

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Ronald R CODNEY
Avatar
Send message
Joined: 19 Nov 11
Posts: 87
Credit: 420,920
RAC: 32
United States
Message 1232792 - Posted: 18 May 2012, 16:10:04 UTC

Heh Matt:
Is there gonna b an ISP change for uploads/downloads, or a reset requirement by the masses?

Thanks for filling us in also.

Profile Chris SProject donor
Volunteer tester
Avatar
Send message
Joined: 19 Nov 00
Posts: 32636
Credit: 14,523,101
RAC: 14,119
United Kingdom
Message 1232797 - Posted: 18 May 2012, 16:12:56 UTC

To get it all up and back on line from scratch in two hours was a major feat of teamwork, I take my hat off to you and the lads, well done!

Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1391
Credit: 74,079
RAC: 10
United States
Message 1232810 - Posted: 18 May 2012, 16:24:08 UTC - in response to Message 1232797.

To get it all up and back on line from scratch in two hours was a major feat of teamwork, I take my hat off to you and the lads, well done!


Thanks. A mixture of "all hands on deck" and incredible luck that nothing really got corrupted/fried when the power suddenly disappeared. There are some RAID resyncs happening at the moment, but looking good thus far...

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

rob smithProject donor
Volunteer tester
Send message
Joined: 7 Mar 03
Posts: 8815
Credit: 63,075,463
RAC: 78,833
United Kingdom
Message 1232815 - Posted: 18 May 2012, 16:34:33 UTC

Glad to hear nothing got fried.
Frying leading to corruption is one of the biggest hazards of a big mains cable shorting, especially one that is of any decent length.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Profile Keith T.
Volunteer tester
Avatar
Send message
Joined: 23 Aug 99
Posts: 740
Credit: 233,186
RAC: 6
United Kingdom
Message 1232817 - Posted: 18 May 2012, 16:36:27 UTC

Well done to all.

Uploads, reporting and Downloads are all working here. Even SETI Beta is working !

I noticed that SSL is over 50 years old now, are the cables a similar age ?

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 16,383,264
RAC: 9,364
United States
Message 1232818 - Posted: 18 May 2012, 16:36:41 UTC - in response to Message 1232810.

Glad to have you guys back up and running. Thanks for all you do.
____________


PROUD MEMBER OF Team Starfire World BOINC

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4247
Credit: 34,966,002
RAC: 22,194
United Kingdom
Message 1232819 - Posted: 18 May 2012, 16:36:44 UTC - in response to Message 1232810.

To get it all up and back on line from scratch in two hours was a major feat of teamwork, I take my hat off to you and the lads, well done!


Thanks. A mixture of "all hands on deck" and incredible luck that nothing really got corrupted/fried when the power suddenly disappeared. There are some RAID resyncs happening at the moment, but looking good thus far...

- Matt

Thanks for the update, and well done for getting everything up so quick too,

Claggy

Lynn Curtis
Send message
Joined: 20 Feb 12
Posts: 1
Credit: 5,353
RAC: 0
United States
Message 1232825 - Posted: 18 May 2012, 16:45:31 UTC - in response to Message 1232771.

Haven't been able to Upload to SETI via BOINC. Are there still issues with connecting to the server?

Steven Proveaux
Send message
Joined: 10 Jun 99
Posts: 1
Credit: 732,399
RAC: 295
United States
Message 1232829 - Posted: 18 May 2012, 16:50:22 UTC

I just uploaded the latest batch of work units since the power has been restored (Friday morning) but there has been no new downloads. has anyone had a similar problem?
____________

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4247
Credit: 34,966,002
RAC: 22,194
United Kingdom
Message 1232833 - Posted: 18 May 2012, 16:52:42 UTC - in response to Message 1232829.

I just uploaded the latest batch of work units since the power has been restored (Friday morning) but there has been no new downloads. has anyone had a similar problem?

See msattler's reply in his last post.

Claggy

Profile Jaye Ellen
Send message
Joined: 29 Nov 08
Posts: 24
Credit: 5,784,138
RAC: 5,558
United States
Message 1232834 - Posted: 18 May 2012, 16:53:39 UTC - in response to Message 1232825.

Hi all,
I also have been unable to send or rcv stuff from BOINC (probly from your power problems) --- I'm hoping the comm restarts soon !!!

Jaye Ellen

Profile perryjay
Volunteer tester
Avatar
Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 16,383,264
RAC: 9,364
United States
Message 1232846 - Posted: 18 May 2012, 17:11:55 UTC

I just managed to report 181 tasks and actually got 1 (one) downloaded in return. That's okay though, I'm just happy things are moving.
____________


PROUD MEMBER OF Team Starfire World BOINC

Profile Ex
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 12 Mar 12
Posts: 2895
Credit: 1,802,523
RAC: 274
United States
Message 1232850 - Posted: 18 May 2012, 17:15:47 UTC
Last modified: 18 May 2012, 17:17:39 UTC

Thanks to the scientists for a great job! +10,000!

and thanks for the update, I think some of us thought the world was ending. :-)

Glad to see things coming online, and people be patient; it just came back online after days of being down, I bet it'll be 24 hours before everyone's machines report and start crunching (getting work) again.
____________
-Dave #2

3.2.0-33

Profile dg6580004
Avatar
Send message
Joined: 14 Mar 12
Posts: 109
Credit: 3,681,089
RAC: 15,247
United States
Message 1232858 - Posted: 18 May 2012, 17:28:18 UTC - in response to Message 1232850.

Nice job on the restore, "everything takes longer than it takes".........keep up the great support.

Until I saw the power outage news posted on the Net, I thought maybe the signal had been found...........Oh well, back to work

1 · 2 · 3 · 4 . . . 9 · Next

Message boards : News : Major Power Outage at SSL

Copyright © 2014 University of California