Weekend Update (Mar 20 2015)

Message boards : Technical News : Weekend Update (Mar 20 2015)

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1441
Credit: 216,982
RAC: 2
United States
Message 1655100 - Posted: 20 Mar 2015, 19:37:21 UTC

Another weekend approaches. Perfect time for an update.

Master science database (informix/paddym): Due to transient disk issues last weekend the database crashed, but we quickly recovered. However this caused a big assimilator queue backlog, which we are just now about to clear out. I'll wait till Monday before I start up the result table merges (we hit extent limits on the old table, so I'm shoveling all those into a new larger table with more extents).

Random crashes: yesterday one of our file servers choked up for no obvious reason (it's a lustre system, of which I'm not an expert, so nothing that happens on that is obvious to me). I don't think this had any public effects, but was hanging a bunch of our servers up a bit. I actually spent all morning getting that in working order again. This morning synergy (the scheduling server) had some automounter freakout so I just reboot it to clear some pipes. All is well now.

Oh yeah the results-to-send queue got kinda low as part of that synergy freakout, but also due to hitting a bunch of tapes with data the splitter is deeming unworthy of workunit creation. So some CPU and I/O time is being wasted as it goes through those. It's best to let this just push through on its on. I did just add a bunch more files to the blanking/splitter queue for processing over the weekend (they'll show up within in the next 24 hours or so).

Unless there's weirdness before the end of the day I won't do anything crazy that'll mess things up for the weekend. I've been otherwise working on some GBT (Green Bank Telescope) code in advance of getting SERENDIP VI hardware working there.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1655100 · Report as offensive
Profile Gary CharpentierCrowdfunding Project Donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 19919
Credit: 25,977,413
RAC: 31,401
United States
Message 1655118 - Posted: 20 Mar 2015, 20:35:52 UTC

Updates are always appreciated.

Thanks
ID: 1655118 · Report as offensive
Profile MikeProject Donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 30105
Credit: 52,632,918
RAC: 21,197
Germany
Message 1655163 - Posted: 20 Mar 2015, 22:58:26 UTC

Thanks Matt for the update.
With each crime and every kindness we birth our future.
ID: 1655163 · Report as offensive
kittymanProject Donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 47650
Credit: 848,301,283
RAC: 130,740
United States
Message 1655205 - Posted: 21 Mar 2015, 1:01:28 UTC

Thank you for the much appreciated updates, Matt.
It means a lot to so many to get your input from the lab.
It really does.

Meow!
Always remember.....kitties are all Angels with fur.
'Cat lives matter.'

Have made friends in this life.
Most were cats.
ID: 1655205 · Report as offensive
Cheopis

Send message
Joined: 17 Sep 00
Posts: 156
Credit: 17,851,694
RAC: 1,997
United States
Message 1655472 - Posted: 21 Mar 2015, 18:21:20 UTC
Last modified: 21 Mar 2015, 18:22:12 UTC

Matt,

You mentioned the other day that one of the power sources in the racking that the servers was located in was the actual cause of what you thought were power supply failures on a couple machines.

Where there is one power system failure, there are frequently others. Have you had the rest of your power sources (not power supplies, sources) tested out? Power fluctuations could be causing all sorts of weird issues. I don't think it would be too much to ask that the datacenter check the others, since there was already one problem found.

Is there anything connected to power which then connects to the servers, but has no power-monitoring capacity? Perhaps an old router or switch or even a monitor? The fact that you had at least one bad power source makes me a bit suspicious of all the weird issues recently maybe having something to do with power.
ID: 1655472 · Report as offensive
Profile UliProject Donor
Volunteer tester
Avatar

Send message
Joined: 6 Feb 00
Posts: 10846
Credit: 5,742,720
RAC: 59
Germany
Message 1655919 - Posted: 23 Mar 2015, 4:17:17 UTC

Thank you for the update Matt.
Pluto will always be a planet to me.

Seti Ambassador
ID: 1655919 · Report as offensive
Mithotar
Avatar

Send message
Joined: 11 Apr 01
Posts: 71
Credit: 36,354,860
RAC: 32,208
United States
Message 1656181 - Posted: 24 Mar 2015, 0:10:46 UTC

I'll add my thanks to this chain........it doesn't take alot - even a once
a week note like this would do wonders to keep the natives less restless.
ID: 1656181 · Report as offensive

Message boards : Technical News : Weekend Update (Mar 20 2015)


 
©2017 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.