Ship of Fools (Jan 12 2012)

Message boards : Technical News : Ship of Fools (Jan 12 2012)
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1184819 - Posted: 12 Jan 2012, 21:01:42 UTC

Hello people. I'm actually about to head out again shortly (once again for a whole month) so let me get y'all caught up before I disappear.

Let's see. There's been a lot of the usual hiccups over the past couple of weeks. Overloaded servers locking up and requiring a hard restart, drives failing and being replaced, bringing machines down on purpose to upgrade the OS, etc. No singular event was tragic or noteworthy, but the quantity of such events has been slightly higher than normal.

Meanwhile various projects have been pushing along. After enough analysis, database tweaking, and data dumping/reloading, we finally created some test "small signal tables" containing the top 1% signals on which to do our final analysis. Turns out doing the same on the 100% full (and constantly growing) tables was a performance disaster. Basically we're now determining what our i/o needs and parameters are with much smaller cases, and then going from there. Right now the signal tables entirely fit in memory, but part of this equation is adding more spindles to the science database array to improve disk i/o as well. This is where the GPU User's Group-donated JBOD comes in. More on that below.

Another project I've been working on is to get the splitters (the programs that make workunits out of raw data) to become sensitive to VGC (voltage gain control) values available in the raw data headers so that we can avoid splitting areas with low VGC values (and therefore loud noise). In layman's terms: we're trying to set everything up to automatically reject noisy workunits before sending them out. We know one or two beams (out of fourteen) are sometimes flaky, and keeping those workunits out of the pipeline will help reduce network competition for downloads.

This should have been fairly straightforward, however during the course of testing we're finding more than one or two beams with various problems. More like 5 or 6. This may be for several different reasons, including bogus or misreported VGC values. This is on a front burner, with several parties involved here and at Arecibo.

Speaking of network competition - yes, we're away that we are dropping all kinds of connections during uploads/downloads. This isn't because of our router (which was definitely the problem over the summer before we added RAM to it), but somewhere else further up the pipeline. Still figuring this out, but it's certainly load related.

Hardware wise, we took an archive server out of the closet to make way for the JBOD mentioned above. The archive server will move into our secondary lab down the hall (where other servers currently reside). We were going to install the JBOD on Tuesday but the hole in the rack made for it isn't big enough to let us mess with internal cabling. Given that we hope to hook this up to at least two separate servers, we'll likely need to mess with internal cabling. So we're going to try to do our best with that while the JBOD is still on the table in our lab.

Oh yeah.. our web server crashed due to overloading last Friday, likely due to an article Andrew published about recent Kepler analysis results. He didn't clearly enough state that these plots were radio frequency interference, and thus we got clobbered due to confused news reports that we found ET. The usual drill, basically. The text of the article was cleaned up. Eric suggested we put disclaimers on the top of every web page on our site that says, "everything we find is Radio Frequency Interference unless we specifically tell you otherwise."

Okay. I should wrap this up. As a parting gift here are a couple random recent photos:

Here's the new JBOD, as seen from behind, sitting on our lab table. There's only 21 (currently empty) drive bays back here, but on the front there are 24 full ones in front.

Here's the current state of our server closet across the hall. Note the hole in the middle rack - that's where the JBOD is going.

And for fun, last week I shot this photo, which is the entire Bay Area consumed in fog, which we at the lab (over 1000 above sea level) are enjoying lovely weather over said fog.

Wow those pictures are blurry. Well, it's from my iPhone 3GS. Not exactly state of the art.

So! I'm now official on the road. I'll be playing with my band MoeTar in Whittier, California on Saturday (opening up for the Allan Holdsworth Band), then I drive up to Seattle to meet some of the guys in Secret Chiefs 3, and then we all drive in the tour van to Denver, where we meet to remaining guys (flying in from NYC and Sydney, Australia). We'll rehearse two days, then tour for a few weeks all over the western US (with one stop in Vancouver), co-headlining with the awesome band Dengue Fever. Should be fun!

Cheers,

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1184819 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1184837 - Posted: 12 Jan 2012, 23:03:51 UTC - in response to Message 1184819.  

Thanks for the update Matt, have an awesome time touring,

Claggy
ID: 1184837 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30593
Credit: 53,134,872
RAC: 32
United States
Message 1184844 - Posted: 12 Jan 2012, 23:42:51 UTC - in response to Message 1184819.  

When you say further up the pipeline, you mean farther from Berkeley and closer to the user, correct?

Thanks for the update.

ID: 1184844 · Report as offensive
B-Man
Volunteer tester

Send message
Joined: 11 Feb 01
Posts: 253
Credit: 147,366
RAC: 0
United States
Message 1184847 - Posted: 12 Jan 2012, 23:48:37 UTC

Thanks for the update Matt. Bet of luck with the tour. Enjoy visiting Australia I want to go visit there so badly myself you lucky dog. I look forward to the next status update in roughly a month.
ID: 1184847 · Report as offensive
Profile PERPLEXER ~ Thomas Huettinger
Volunteer tester
Avatar

Send message
Joined: 25 Jan 05
Posts: 11
Credit: 395,156,213
RAC: 371
Germany
Message 1184896 - Posted: 13 Jan 2012, 4:58:08 UTC

ship of fools ?
ID: 1184896 · Report as offensive
Profile Slavac
Volunteer tester
Avatar

Send message
Joined: 27 Apr 11
Posts: 1932
Credit: 17,952,639
RAC: 0
United States
Message 1184910 - Posted: 13 Jan 2012, 8:15:33 UTC - in response to Message 1184896.  

Thanks for the update Matt, that JBOD is a big fella.


Executive Director GPU Users Group Inc. -
brad@gpuug.org
ID: 1184910 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15184
Credit: 4,362,181
RAC: 3
Netherlands
Message 1184913 - Posted: 13 Jan 2012, 8:53:20 UTC - in response to Message 1184896.  

ship of fools ?

Great soft-rock song by Robert Plant. :-)
ID: 1184913 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 1184919 - Posted: 13 Jan 2012, 9:26:31 UTC - in response to Message 1184913.  

ship of fools ?

Great soft-rock song by Robert Plant. :-)

I was thinking World Party.
Grant
Darwin NT
ID: 1184919 · Report as offensive
Profile PERPLEXER ~ Thomas Huettinger
Volunteer tester
Avatar

Send message
Joined: 25 Jan 05
Posts: 11
Credit: 395,156,213
RAC: 371
Germany
Message 1184931 - Posted: 13 Jan 2012, 11:10:07 UTC - in response to Message 1184919.  

Ship of fools? What does it mean in that case of tech news?
ID: 1184931 · Report as offensive
musicplayer

Send message
Joined: 17 May 10
Posts: 2430
Credit: 926,046
RAC: 0
Message 1184944 - Posted: 13 Jan 2012, 12:35:34 UTC
Last modified: 13 Jan 2012, 12:37:37 UTC

My guess is that someone is thinking that there may be some "dumb users" around.

Possible credit not included.

You may have noticed Matt starting up a thread here a little earlier. He makes a point of possible extraterrestrials as being "technological civilizations".

Meaning that such civilizations very well could be very advanced and highly developed.

This is why they possibly are not being detected. Is there any assumption that a category 3 civilization (totally confined to space) would bother about transmitting their information by means which would be detectable by our own receiving equipment (Arecibo).

Lightning and thunder as well as earthquakes are naturally occurring here on Earth.

Similarly, we have novae, supernovae (rare events) and other outburst from stars leading to creation of planetary nebulae and the like. Nothing intelligent is ever thought being behind such natural occurring events, though. We still rely on what could be going on in the small frequency band we are scanning through by our clients in order to try detecting something of particular interest.
ID: 1184944 · Report as offensive
Profile Khangollo
Avatar

Send message
Joined: 1 Aug 00
Posts: 245
Credit: 36,410,524
RAC: 0
Slovenia
Message 1184976 - Posted: 13 Jan 2012, 15:12:41 UTC - in response to Message 1184819.  
Last modified: 13 Jan 2012, 15:17:57 UTC

Thanks for the update, Matt.

Speaking of network competition - yes, we're away that we are dropping all kinds of connections during uploads/downloads. This isn't because of our router (which was definitely the problem over the summer before we added RAM to it), but somewhere else further up the pipeline. Still figuring this out, but it's certainly load related.

Maybe you should try to swap Apache for nginx ?
Guys at Einstein@home had problems with overcommitted dl servers and solved them by using nginx.
Apache just goes berserk with memory and CPU under heavy load.
ID: 1184976 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1184987 - Posted: 13 Jan 2012, 16:54:19 UTC - in response to Message 1184913.  
Last modified: 13 Jan 2012, 16:58:13 UTC

RE: Ship of Fools

Also a movie from 1965, and the novel it was adapted from by Katherine Ann Porter. The movie was Vivien Leigh's (Gone with the Wind} last appearance on film. The movie won two Oscars®.
.

Hello, from Albany, CA!...
ID: 1184987 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 1185036 - Posted: 13 Jan 2012, 21:41:41 UTC - in response to Message 1184931.  

Ship of fools? What does it mean in that case of tech news?

It's the name of a song & Matt often uses song names for Tech News thread names.
Grant
Darwin NT
ID: 1185036 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 1186038 - Posted: 17 Jan 2012, 23:06:24 UTC

Yeah I've been titling threads for the past few months using nothing but names of Secret Chiefs 3 songs (the band I'm touring with lately) - trying to make them apropos if possible. I was thinking "Ship of Fools" was perfect for describing me about to leave on tour, then driving around in circles around the western half of North America in all kinds of crazy winter weather...

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 1186038 · Report as offensive
Profile PERPLEXER ~ Thomas Huettinger
Volunteer tester
Avatar

Send message
Joined: 25 Jan 05
Posts: 11
Credit: 395,156,213
RAC: 371
Germany
Message 1186060 - Posted: 18 Jan 2012, 1:45:59 UTC - in response to Message 1186038.  

ok thx
play it loud for getting warm fingers
ID: 1186060 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22149
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1186115 - Posted: 18 Jan 2012, 8:17:52 UTC

Thanks Matt.
Travel safe, play well, and have a ball.






(Aside, how's your small fluffy assistant doing - got him/her centre stage yet?)
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1186115 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1188095 - Posted: 24 Jan 2012, 21:28:02 UTC

Just noticed today (Tuesday 1/24) that the usual outage notice still says that the SETI computers will be down three days... why was this never changed when you went back to the Tuesday-only schedule? And could someone change it to an appropriate message soon?
.

Hello, from Albany, CA!...
ID: 1188095 · Report as offensive
Profile john3760
Avatar

Send message
Joined: 9 Feb 11
Posts: 334
Credit: 3,400,979
RAC: 0
United Kingdom
Message 1188110 - Posted: 24 Jan 2012, 22:42:32 UTC

good luck matt

ship of fools is one of my favorite

World Party songs.

It's amazing how so many songs have the same titles

(not including cover versions)

I myself once wrote a song which I called Warpigs

Imagine my sorrow when I realised Black Sabbath had

already released a song of the same name.

I had to almost completely rewright the lyrics

and I called it californication.

luckily I have never heard of another song with the

same name (phew!)

john3760

PM or email me the next time you are touring england

I was touring myself the last time you played here

(with Take That) and missed it.

I would lovre to see your band live.


ID: 1188110 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1188290 - Posted: 25 Jan 2012, 14:42:17 UTC - in response to Message 1188110.  

I myself once wrote a song which I called Warpigs

Imagine my sorrow when I realised Black Sabbath had

already released a song of the same name.

I had to almost completely rewright the lyrics

and I called it californication.

luckily I have never heard of another song with the

same name (phew!)

john3760

You are aware, aren't you, that there's a TV show called Californication? It's on Showtime and features lots of sex and nudity and other stuff you can't get on US TV other than on the premium cable channels. I don't know if it has a theme song. Seti topicality: it stars David Duchovny, previously the star of The X Files, which was about UFOs.

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1188290 · Report as offensive
Profile Es99
Volunteer tester
Avatar

Send message
Joined: 23 Aug 05
Posts: 10874
Credit: 350,402
RAC: 0
Canada
Message 1194241 - Posted: 12 Feb 2012, 8:16:45 UTC - in response to Message 1186038.  

Yeah I've been titling threads for the past few months using nothing but names of Secret Chiefs 3 songs (the band I'm touring with lately) - trying to make them apropos if possible. I was thinking "Ship of Fools" was perfect for describing me about to leave on tour, then driving around in circles around the western half of North America in all kinds of crazy winter weather...

- Matt

I was hoping to come and see your Vancouver gig, but life, family drama and a Saturday morning job interview got in the way.

Hope you enjoyed Vancouver and the gig was a success.
Reality Internet Personality
ID: 1194241 · Report as offensive
1 · 2 · Next

Message boards : Technical News : Ship of Fools (Jan 12 2012)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.