Goblin Valley (Jan 29 2009)


log in

Advanced search

Message boards : Technical News : Goblin Valley (Jan 29 2009)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 859476 - Posted: 29 Jan 2009, 23:25:26 UTC

The replica mysql database on sidious recovered more or less just fine. It may be ever so slightly out of sync with the master database. This means we'll probably rebuild it during the next weekly outage just to be sure.

The scheduling server was up and down yesterday afternoon and this morning. The scheduler CGIs have been segfaulting and adding core dumps caused the system to grind to a halt, needing a reboot. Turns out the problem wasn't in the CGI, but in apache itself (or the fastcgi module). This has been a problem in the past. We seem to have to tweak various apache parameters at random times, based on a chaotic, unpredictable equation involving current resources/demands, mysql health, network health, system health, various queue sizes, etc. Simply reducing the MaxClients to a much lower number caused the segfaults to disappear while still servicing all incoming requests.

We're running low on data to send out, and we're in a murky period where the weekend is rapidly approaching and we are still awaiting the latest shipment of raw data drives from Arecibo. We could pull up as-yet-unanalysed data from our archives, but the offsite storage archive (HPSS) is undergoing several upgrades and have been offline for days. We'll see how this all pans out...

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Bernie Vine
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 26 May 99
Posts: 7084
Credit: 27,556,120
RAC: 36,121
United Kingdom
Message 859503 - Posted: 30 Jan 2009, 0:42:36 UTC

Thanks Matt as always for the update.

Seems a little quiet round here these days. Only 1 post yesterday!

I hope work doesn't run out. Still that will give em something to moan about over in NC :-)
____________


Today is life, the only life we're sure of. Make the most of today.

Profile Geek@PlayProject donor
Volunteer tester
Avatar
Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,144,272
RAC: 826
United States
Message 859527 - Posted: 30 Jan 2009, 1:28:08 UTC

Thanks for the info Matt.

Any part of Southern Utah you have not been to???

And one more question..........is AstroPulse a dead project now???

____________
Boinc....Boinc....Boinc....Boinc....

Swibby Bear
Send message
Joined: 1 Aug 01
Posts: 236
Credit: 7,276,504
RAC: 3
United States
Message 859570 - Posted: 30 Jan 2009, 3:11:19 UTC
Last modified: 30 Jan 2009, 3:12:30 UTC

No, I am sure that AstroPulse is not dead. As Matt explained below, ALL the data for AP has been split and sent out. New data from Arecibo is expected soon.

If the hard drives with new data doesn't arrive on Friday, there is a good possibility that MultiBeam WUs will also be exhausted over the weekend, as well as AstroPulse.

In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.

Whit

Speedy
Volunteer tester
Avatar
Send message
Joined: 26 Jun 04
Posts: 679
Credit: 5,931,710
RAC: 3,863
New Zealand
Message 859586 - Posted: 30 Jan 2009, 4:08:28 UTC - in response to Message 859570.


In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.


I hope new work arrives tomorrow, if not I think this will be the first time I have ever seen Seti run out of work. I will not be one to moan because I think the whole crew does a great job at the lab. Sometimes you can't help delays with the mail and courier services as annoying as it is.
____________

Live in NZ y not join Smile City?

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2292
Credit: 8,815,877
RAC: 4,052
United States
Message 859595 - Posted: 30 Jan 2009, 4:34:46 UTC - in response to Message 859586.


In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.


I hope new work arrives tomorrow, if not I think this will be the first time I have ever seen Seti run out of work. I will not be one to moan because I think the whole crew does a great job at the lab. Sometimes you can't help delays with the mail and courier services as annoying as it is.

This is when a 4+ day cache comes in handy. I should be fine until about Wednesday, even though it's only a 4-day cache.
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

WinterKnight
Volunteer tester
Send message
Joined: 18 May 99
Posts: 8687
Credit: 25,049,386
RAC: 30,165
United Kingdom
Message 859599 - Posted: 30 Jan 2009, 4:58:02 UTC - in response to Message 859527.

Thanks for the info Matt.

Any part of Southern Utah you have not been to???

And one more question..........is AstroPulse a dead project now???

AP is not dead, on Beta V5.01 has just started testing.

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24694
Credit: 522,659
RAC: 19
United States
Message 860039 - Posted: 31 Jan 2009, 4:47:14 UTC - in response to Message 859595.


In other words, SETI may have absolutely no work to send out over part of the weekend and possibly even into next week.


I hope new work arrives tomorrow, if not I think this will be the first time I have ever seen Seti run out of work. I will not be one to moan because I think the whole crew does a great job at the lab. Sometimes you can't help delays with the mail and courier services as annoying as it is.

This is when a 4+ day cache comes in handy. I should be fine until about Wednesday, even though it's only a 4-day cache.

I run multiple projects on each computer, and it would take about 15 specific projects down at the same time to turn off work to the least capable of my computers.
____________


BOINC WIKI

Profile Misfit
Volunteer tester
Avatar
Send message
Joined: 21 Jun 01
Posts: 21790
Credit: 2,510,901
RAC: 0
United States
Message 860568 - Posted: 1 Feb 2009, 5:52:26 UTC - in response to Message 859476.

We could pull up as-yet-unanalysed data from our archives,

Wouldn't that be everything from day 1 to now?
____________

Join BOINC Synergy!

Speedy
Volunteer tester
Avatar
Send message
Joined: 26 Jun 04
Posts: 679
Credit: 5,931,710
RAC: 3,863
New Zealand
Message 860574 - Posted: 1 Feb 2009, 6:14:53 UTC - in response to Message 860568.

Wouldn't that be everything from day 1 to now?

It could be but I would think it will just be the data that hasn't been analyzed that is in the archives. I have raised the question before. Why don't we process all the old data first? I think the main reason is because to date we simply don't have enough computing power to keep up with the data volume. If there is another reason please correct me.
____________

Live in NZ y not join Smile City?

michael mcbride
Send message
Joined: 28 Jan 09
Posts: 3
Credit: 482
RAC: 0
Portugal
Message 860608 - Posted: 1 Feb 2009, 10:39:50 UTC - in response to Message 859476.

Thanks so much for the update, thus i could understand what happened here with my Computer, with downloaded none during yesterday... I was thinking something was wrong here!
____________

Profile Virtual Boss*
Volunteer tester
Avatar
Send message
Joined: 4 May 08
Posts: 417
Credit: 6,199,642
RAC: 428
Australia
Message 860660 - Posted: 1 Feb 2009, 13:03:30 UTC - in response to Message 860608.

Welcome to the message boards michael.

And yes it can be confusing trying to work out where the problem is sometimes, but there usually will be posts in the number crunching forum when there is problems with the Seti servers.

Also most of us have bookmarked the Server Status page, and have a look at it whenever problems arise.

Profile Neil Blaikie
Volunteer tester
Avatar
Send message
Joined: 17 May 99
Posts: 142
Credit: 6,600,932
RAC: 7,490
Canada
Message 860739 - Posted: 1 Feb 2009, 16:55:10 UTC

Seems like for the first time in a very long time, I am going to run out of work, should be in about 5 hours time.

Ah well, will give the computer a nice rest as it has been crunching 24/7 for the last 2 years solid! Give or take the inevitable occasional windows reboot required.

Will be patient until it can send some work, seems to have gone through the 4 day cache fast though.

Happy SuperBowl day :-)
____________

Profile Neil Blaikie
Volunteer tester
Avatar
Send message
Joined: 17 May 99
Posts: 142
Credit: 6,600,932
RAC: 7,490
Canada
Message 860772 - Posted: 1 Feb 2009, 18:03:29 UTC - in response to Message 860739.

Seems like there is a small amount of work filtering through, since my last post I have got a few tasks come in.
____________

Profile Pooh Bear 27
Volunteer tester
Avatar
Send message
Joined: 14 Jul 03
Posts: 3221
Credit: 2,640,394
RAC: 1,530
United States
Message 860786 - Posted: 1 Feb 2009, 18:28:34 UTC - in response to Message 860772.

Seems like there is a small amount of work filtering through, since my last post I have got a few tasks come in.

If work expires, it will be resent. So there will be small amounts.

____________

Speedy
Volunteer tester
Avatar
Send message
Joined: 26 Jun 04
Posts: 679
Credit: 5,931,710
RAC: 3,863
New Zealand
Message 860832 - Posted: 1 Feb 2009, 20:24:31 UTC
Last modified: 1 Feb 2009, 20:29:10 UTC

Server Status as of 1 Feb 20:00:10 as of * 47 minutes says there are 33,474 Results ready to send. Spliters are spliting the last 5 channels at present, tape 18dc08aa is upto chanel 13. Cricket Graph says as of Feb 1 11:52:22


    Average bits in Cur: 28.43 Mbits/sec
    Average bits out Curr: 14.45 Mbits/sec


This is my 2 cents worth for the crunchers that like technical information
7 new data tapes have been added while I've been typing this post. A big thank you to the person who has come to the lab to add the work to queue on a Sunday morning. I didn't think we will run out of work now. Thank you again to the staff member that came into the lab
____________

Live in NZ y not join Smile City?

Profile Misfit
Volunteer tester
Avatar
Send message
Joined: 21 Jun 01
Posts: 21790
Credit: 2,510,901
RAC: 0
United States
Message 860969 - Posted: 2 Feb 2009, 3:25:59 UTC - in response to Message 860574.

I think the main reason is because to date we simply don't have enough computing power to keep up with the data volume. If there is another reason please correct me.

I'd have to disagree with that. I remember a post by Matt a long time ago asking for assistance at Beta when developing the enhanced workunits or SETI would run out of work. That's why I asked NEZ to attach to Beta and he graciously complied.
____________

Join BOINC Synergy!

Speedy
Volunteer tester
Avatar
Send message
Joined: 26 Jun 04
Posts: 679
Credit: 5,931,710
RAC: 3,863
New Zealand
Message 860997 - Posted: 2 Feb 2009, 5:22:21 UTC - in response to Message 860969.


I'd have to disagree with that. I remember a post by Matt a long time ago asking for assistance at Beta when developing the enhanced workunits or SETI would run out of work. That's why I asked NEZ to attach to Beta and he graciously complied.


I think the main reason is because to date we simply don't have enough computing power to keep up with the data volume.

ok heres the thread i based the information on that i gave you.

____________

Live in NZ y not join Smile City?

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8634
Credit: 51,628,780
RAC: 49,026
United Kingdom
Message 861039 - Posted: 2 Feb 2009, 9:57:55 UTC - in response to Message 860832.

7 new data tapes have been added while I've been typing this post. A big thank you to the person who has come to the lab to add the work to queue on a Sunday morning. I didn't think we will run out of work now. Thank you again to the staff member that came into the lab

Adding my thanks for the extra work added to the queue on a Sunday.

We're almost at the end of the 18dec 'shorty' (G-ALFA Continuum Transit Survey) recordings. Let's hope the 'tapes' brought back out of storage contain meatier fare, to damp down the bandwidth demand a bit. Maybe even some VLARs, to really damp down the CUDA demand ;-) ?

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8634
Credit: 51,628,780
RAC: 49,026
United Kingdom
Message 861251 - Posted: 2 Feb 2009, 20:54:57 UTC

Do the staff have access to an internal graph like Scarecrow's?



See how the clear trendline jumps on 17 December (CUDA v6.05 day), declines again as the bugs come to light, then starts to climb strongly on 21 January (CUDA v6.08 day).

We're now returning consistently double the number of results we were before CUDA.

Matt, soon after the first CUDA launch, you did a quick snapshot and came up with a figure of 3% of results returned having been processed by a CUDA application. Would you have a moment to re-run that query now we seem to be in full production?

Message boards : Technical News : Goblin Valley (Jan 29 2009)

Copyright © 2014 University of California