Goblin Valley (Jan 29 2009)

Message boards : Technical News : Goblin Valley (Jan 29 2009)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 859476 - Posted: 29 Jan 2009, 23:25:26 UTC

The replica mysql database on sidious recovered more or less just fine. It may be ever so slightly out of sync with the master database. This means we'll probably rebuild it during the next weekly outage just to be sure.

The scheduling server was up and down yesterday afternoon and this morning. The scheduler CGIs have been segfaulting and adding core dumps caused the system to grind to a halt, needing a reboot. Turns out the problem wasn't in the CGI, but in apache itself (or the fastcgi module). This has been a problem in the past. We seem to have to tweak various apache parameters at random times, based on a chaotic, unpredictable equation involving current resources/demands, mysql health, network health, system health, various queue sizes, etc. Simply reducing the MaxClients to a much lower number caused the segfaults to disappear while still servicing all incoming requests.

We're running low on data to send out, and we're in a murky period where the weekend is rapidly approaching and we are still awaiting the latest shipment of raw data drives from Arecibo. We could pull up as-yet-unanalysed data from our archives, but the offsite storage archive (HPSS) is undergoing several upgrades and have been offline for days. We'll see how this all pans out...

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 859476 · Report as offensive
Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 26 May 99
Posts: 9954
Credit: 103,452,613
RAC: 328
United Kingdom
Message 859503 - Posted: 30 Jan 2009, 0:42:36 UTC

Thanks Matt as always for the update.

Seems a little quiet round here these days. Only 1 post yesterday!

I hope work doesn't run out. Still that will give em something to moan about over in NC :-)
ID: 859503 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 859527 - Posted: 30 Jan 2009, 1:28:08 UTC

Thanks for the info Matt.

Any part of Southern Utah you have not been to???

And one more question..........is AstroPulse a dead project now???

Boinc....Boinc....Boinc....Boinc....
ID: 859527 · Report as offensive
Swibby Bear

Send message
Joined: 1 Aug 01
Posts: 246
Credit: 7,945,093
RAC: 0
United States
Message 859570 - Posted: 30 Jan 2009, 3:11:19 UTC
Last modified: 30 Jan 2009, 3:12:30 UTC

No, I am sure that AstroPulse is not dead. As Matt explained below, ALL the data for AP has been split and sent out. New data from Arecibo is expected soon.

If the hard drives with new data doesn't arrive on Friday, there is a good possibility that MultiBeam WUs will also be exhausted over the weekend, as well as AstroPulse.

In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.

Whit
ID: 859570 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 859586 - Posted: 30 Jan 2009, 4:08:28 UTC - in response to Message 859570.  


In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.


I hope new work arrives tomorrow, if not I think this will be the first time I have ever seen Seti run out of work. I will not be one to moan because I think the whole crew does a great job at the lab. Sometimes you can't help delays with the mail and courier services as annoying as it is.
ID: 859586 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 859595 - Posted: 30 Jan 2009, 4:34:46 UTC - in response to Message 859586.  


In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.


I hope new work arrives tomorrow, if not I think this will be the first time I have ever seen Seti run out of work. I will not be one to moan because I think the whole crew does a great job at the lab. Sometimes you can't help delays with the mail and courier services as annoying as it is.

This is when a 4+ day cache comes in handy. I should be fine until about Wednesday, even though it's only a 4-day cache.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 859595 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 18996
Credit: 40,757,560
RAC: 67
United Kingdom
Message 859599 - Posted: 30 Jan 2009, 4:58:02 UTC - in response to Message 859527.  

Thanks for the info Matt.

Any part of Southern Utah you have not been to???

And one more question..........is AstroPulse a dead project now???

AP is not dead, on Beta V5.01 has just started testing.
ID: 859599 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 860039 - Posted: 31 Jan 2009, 4:47:14 UTC - in response to Message 859595.  


In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.


I hope new work arrives tomorrow, if not I think this will be the first time I have ever seen Seti run out of work. I will not be one to moan because I think the whole crew does a great job at the lab. Sometimes you can't help delays with the mail and courier services as annoying as it is.

This is when a 4+ day cache comes in handy. I should be fine until about Wednesday, even though it's only a 4-day cache.

I run multiple projects on each computer, and it would take about 15 specific projects down at the same time to turn off work to the least capable of my computers.


BOINC WIKI
ID: 860039 · Report as offensive
Profile Misfit
Volunteer tester
Avatar

Send message
Joined: 21 Jun 01
Posts: 21804
Credit: 2,815,091
RAC: 0
United States
Message 860568 - Posted: 1 Feb 2009, 5:52:26 UTC - in response to Message 859476.  

We could pull up as-yet-unanalysed data from our archives,

Wouldn't that be everything from day 1 to now?
me@rescam.org
ID: 860568 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 860574 - Posted: 1 Feb 2009, 6:14:53 UTC - in response to Message 860568.  

Wouldn't that be everything from day 1 to now?

It could be but I would think it will just be the data that hasn't been analyzed that is in the archives. I have raised the question before. Why don't we process all the old data first? I think the main reason is because to date we simply don't have enough computing power to keep up with the data volume. If there is another reason please correct me.
ID: 860574 · Report as offensive
michael mcbride

Send message
Joined: 28 Jan 09
Posts: 3
Credit: 482
RAC: 0
Portugal
Message 860608 - Posted: 1 Feb 2009, 10:39:50 UTC - in response to Message 859476.  

Thanks so much for the update, thus i could understand what happened here with my Computer, with downloaded none during yesterday... I was thinking something was wrong here!
ID: 860608 · Report as offensive
Profile Virtual Boss*
Volunteer tester
Avatar

Send message
Joined: 4 May 08
Posts: 417
Credit: 6,440,287
RAC: 0
Australia
Message 860660 - Posted: 1 Feb 2009, 13:03:30 UTC - in response to Message 860608.  

Welcome to the message boards michael.

And yes it can be confusing trying to work out where the problem is sometimes, but there usually will be posts in the number crunching forum when there is problems with the Seti servers.

Also most of us have bookmarked the Server Status page, and have a look at it whenever problems arise.
ID: 860660 · Report as offensive
Profile Neil Blaikie
Volunteer tester
Avatar

Send message
Joined: 17 May 99
Posts: 143
Credit: 6,652,341
RAC: 0
Canada
Message 860739 - Posted: 1 Feb 2009, 16:55:10 UTC

Seems like for the first time in a very long time, I am going to run out of work, should be in about 5 hours time.

Ah well, will give the computer a nice rest as it has been crunching 24/7 for the last 2 years solid! Give or take the inevitable occasional windows reboot required.

Will be patient until it can send some work, seems to have gone through the 4 day cache fast though.

Happy SuperBowl day :-)
ID: 860739 · Report as offensive
Profile Neil Blaikie
Volunteer tester
Avatar

Send message
Joined: 17 May 99
Posts: 143
Credit: 6,652,341
RAC: 0
Canada
Message 860772 - Posted: 1 Feb 2009, 18:03:29 UTC - in response to Message 860739.  

Seems like there is a small amount of work filtering through, since my last post I have got a few tasks come in.
ID: 860772 · Report as offensive
Profile Pooh Bear 27
Volunteer tester
Avatar

Send message
Joined: 14 Jul 03
Posts: 3224
Credit: 4,603,826
RAC: 0
United States
Message 860786 - Posted: 1 Feb 2009, 18:28:34 UTC - in response to Message 860772.  

Seems like there is a small amount of work filtering through, since my last post I have got a few tasks come in.

If work expires, it will be resent. So there will be small amounts.


My movie https://vimeo.com/manage/videos/502242
ID: 860786 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 860832 - Posted: 1 Feb 2009, 20:24:31 UTC
Last modified: 1 Feb 2009, 20:29:10 UTC

Server Status as of 1 Feb 20:00:10 as of * 47 minutes says there are 33,474 Results ready to send. Spliters are spliting the last 5 channels at present, tape 18dc08aa is upto chanel 13. Cricket Graph says as of Feb 1 11:52:22

    Average bits in Cur: 28.43 Mbits/sec
    Average bits out Curr: 14.45 Mbits/sec


This is my 2 cents worth for the crunchers that like technical information
7 new data tapes have been added while I've been typing this post. A big thank you to the person who has come to the lab to add the work to queue on a Sunday morning. I didn't think we will run out of work now. Thank you again to the staff member that came into the lab


ID: 860832 · Report as offensive
Profile Misfit
Volunteer tester
Avatar

Send message
Joined: 21 Jun 01
Posts: 21804
Credit: 2,815,091
RAC: 0
United States
Message 860969 - Posted: 2 Feb 2009, 3:25:59 UTC - in response to Message 860574.  

I think the main reason is because to date we simply don't have enough computing power to keep up with the data volume. If there is another reason please correct me.

I'd have to disagree with that. I remember a post by Matt a long time ago asking for assistance at Beta when developing the enhanced workunits or SETI would run out of work. That's why I asked NEZ to attach to Beta and he graciously complied.
me@rescam.org
ID: 860969 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 860997 - Posted: 2 Feb 2009, 5:22:21 UTC - in response to Message 860969.  


I'd have to disagree with that. I remember a post by Matt a long time ago asking for assistance at Beta when developing the enhanced workunits or SETI would run out of work. That's why I asked NEZ to attach to Beta and he graciously complied.


I think the main reason is because to date we simply don't have enough computing power to keep up with the data volume.

ok heres the thread i based the information on that i gave you.

ID: 860997 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14645
Credit: 200,643,578
RAC: 874
United Kingdom
Message 861039 - Posted: 2 Feb 2009, 9:57:55 UTC - in response to Message 860832.  

7 new data tapes have been added while I've been typing this post. A big thank you to the person who has come to the lab to add the work to queue on a Sunday morning. I didn't think we will run out of work now. Thank you again to the staff member that came into the lab

Adding my thanks for the extra work added to the queue on a Sunday.

We're almost at the end of the 18dec 'shorty' (G-ALFA Continuum Transit Survey) recordings. Let's hope the 'tapes' brought back out of storage contain meatier fare, to damp down the bandwidth demand a bit. Maybe even some VLARs, to really damp down the CUDA demand ;-) ?
ID: 861039 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14645
Credit: 200,643,578
RAC: 874
United Kingdom
Message 861251 - Posted: 2 Feb 2009, 20:54:57 UTC

Do the staff have access to an internal graph like Scarecrow's?



See how the clear trendline jumps on 17 December (CUDA v6.05 day), declines again as the bugs come to light, then starts to climb strongly on 21 January (CUDA v6.08 day).

We're now returning consistently double the number of results we were before CUDA.

Matt, soon after the first CUDA launch, you did a quick snapshot and came up with a figure of 3% of results returned having been processed by a CUDA application. Would you have a moment to re-run that query now we seem to be in full production?
ID: 861251 · Report as offensive

Message boards : Technical News : Goblin Valley (Jan 29 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.