Increasing Numbers (Jul 01 2009)


log in

Advanced search

Message boards : Technical News : Increasing Numbers (Jul 01 2009)

1 · 2 · 3 · Next
Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 912982 - Posted: 1 Jul 2009, 19:38:40 UTC

Sorry about the forums (and other web site features) being shut off for over a day. These Tuesday outages are really taking forever. I guess we've been really busy, which means our tables get ridiculously fragmented throughout the week. Plus I noticed our database is easily 50% larger than it has been about 2 months ago. And the replica lost a couple of its CPUs recently (it's a used/donated system and the CPUs were known to be flakey from the start). Anyway, since the normal recovery procedure was so painful last week I opted to keep all web page database lookups offline until the replica was caught up. Once again, I'm sorry for the inconvenience.

To make up for that, how about some videos from the SETI@home 10 year anniversary? I'll link these to the home page soon enough. Consider this a sneak preview for those who read these threads. Let me know if there are problems downloading/viewing these mpegs.

Data recorder-wise... After all the effort to work with what we got, we're finally throwing in the towel on the current set of data drive enclosures. We have a plan B and plan C already in place - just a matter of deciding which one to enact. Meanwhile, I'm pulling old data off the archives at a pretty good clip - hopefully fast enough to keep up with demand.

Otherwise, I'm still working on NTPCkr and radar stuff. And I adjusted the stats scripts that generate the numbers on the server status page. The Astropulse numbers up until this morning reflected version 5 workunits/results, now they reflect version 5.05.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

B-Man
Volunteer tester
Send message
Joined: 11 Feb 01
Posts: 253
Credit: 147,366
RAC: 0
United States
Message 912985 - Posted: 1 Jul 2009, 19:50:02 UTC

Nit-picky comment. It is not June 1 but July 1. I know I want the last month as a do over to.

____________

Sten-Arne
Volunteer tester
Send message
Joined: 1 Nov 08
Posts: 3510
Credit: 20,648,422
RAC: 22,107
Sweden
Message 912989 - Posted: 1 Jul 2009, 20:04:05 UTC - in response to Message 912982.

....I opted to keep all web page database lookups offline until the replica was caught up. Once again, I'm sorry for the inconvenience.

- Matt


Thanks for the update.

However the replica is far from up to date. It's almost 19 hours behind, and falling more an more behind all the time. So keeping web page db lookups offline, doesn't seem to have helped much...

Sten-Arne

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13625
Credit: 30,970,107
RAC: 20,711
United States
Message 912997 - Posted: 1 Jul 2009, 20:16:51 UTC - in response to Message 912982.

The Astropulse numbers up until this morning reflected version 5 workunits/results, now they reflect version 5.05.


Will a new option be added to the preferences page for v5.05 for opt-in and opt-out?
____________

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12702
Credit: 7,190,935
RAC: 15,523
United States
Message 913023 - Posted: 1 Jul 2009, 21:09:45 UTC

Thanks for the update and the videos!

Q: Can we expect more day long data driven web page outages?

____________

Invisible Man
Send message
Joined: 24 Jun 01
Posts: 22
Credit: 1,129,336
RAC: 0
United Kingdom
Message 913053 - Posted: 1 Jul 2009, 22:16:55 UTC

Thanks for update Matt.

Quote "01/07/2009 23:00:37 SETI@home Message from server: (Project has no jobs available).

Server status - "Results ready to send 1,568,646".

Any problems here?
____________

JohnDKProject donor
Volunteer tester
Avatar
Send message
Joined: 28 May 00
Posts: 862
Credit: 46,465,391
RAC: 71,962
Denmark
Message 913056 - Posted: 1 Jul 2009, 22:21:22 UTC - in response to Message 913053.
Last modified: 1 Jul 2009, 22:21:43 UTC

Quote "01/07/2009 23:00:37 SETI@home Message from server: (Project has no jobs available).

Server status - "Results ready to send 1,568,646".

Any problems here?

Something about these numbers, they're supposed to be a lot higher if all was working optimal:
Current result creation rate 0.3913/sec 0.7114/sec 5m

HAL
Send message
Joined: 28 Mar 03
Posts: 704
Credit: 870,617
RAC: 0
United States
Message 913060 - Posted: 1 Jul 2009, 22:29:02 UTC - in response to Message 913056.

And out of all that work available I get a message that the project has no jobs available.
Curiouser and Cusriouser! :)
____________

Classic WU= 7,237 Classic Hours= 42,079

Profile Mike O
Avatar
Send message
Joined: 1 Sep 07
Posts: 428
Credit: 6,670,998
RAC: 0
United States
Message 913085 - Posted: 1 Jul 2009, 23:17:16 UTC

Matt...
You are Da Man!
I always read your thread each day.. I dont always post as..my mouth's not required.
I have talked about this before.. I really think you need to cut your losses and move some of the data from MySQL over to archives.
Your poor servers are trying so hard to keep up but unless you replace them with something that can handle it, its only going to get worse as the tables grow to 100's of gigs.
Your a genious.. Write some code and move the table entries based on date(or something) over to a big ol' hard drive and put it away somewhere safe. Myabe even replicate it just incase.
There is so much data just from the old forums here, it would probably choke my server's 480 gig just trying to deal with that let'alone the results in/out and storage. Good Grief! I can already smell the cpus smoldering.
I understand the need to keep the Results Data.. I have read your posts about this before and it's all clear to me. Breaking that up would make it a MF to put back together so .. best to leave that one alone.

I never did ask.. what do you play? Guitar? Drums?.. Tuba? <-- kidding ;)
I have been in many bands and miss those days at times. (I don't miss the work of hauling the gear to a gig tho!)
Maybe.. one of these days, i'll upload some pics from my 80's haircut bands :)

Anyway... When are you guys going to 'bite the bullet' and get a the power for the server you need? I posted a link a while abck to a killer 8x dual core xeon (16 cores) IBM server with heaps of memory and 3 new raid drives that was listed on Ebay.
This thing was at $999.00 and no one bid on it. You could a had it for 999.10 USD. Sad thing.. it weighed almost 200 pounds! Shipping would cost a chunk!
I wish I could help some way.. It must be a rush when things are going wrong and even more of a rush when things are going 'just right'
Good luck and...
If you took a poll, I think you would find that most here would rather the servers be off for 5 days then take a chance on losing any of the results data.
10 years is an long a## time of collection. I dont think anyone can even touch it. I know a lot are looking forward with huge.. 'on the edge of the chair' eggerness to see this data crunched down. Besides ET.. who knows what this will produce? There may be things in there that no one expected!
crap.. gotta go
--Mike
____________
Not Ready Reading BRAIN. Abort/Retry/Fail?

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12702
Credit: 7,190,935
RAC: 15,523
United States
Message 913091 - Posted: 1 Jul 2009, 23:44:40 UTC - in response to Message 913060.

And out of all that work available I get a message that the project has no jobs available.
Curiouser and Cusriouser! :)

Nothing funny at all.

There is this job to hand out work and it has a queue. Here is how it works.
A guy behind a window (the job) gets a request for a work unit and he grabs one off the queue and hands it to you. He keep doing this until the queue is empty. When it is empty he hangs up a sign No Jobs Available and swivels his chair around to the database and grabs 100 work units to fill his queue up. When he is done and his queue full again he turns back around and takes the sign down.

When the load is light, the man is facing you most of the time and you get work most of the time. When the load gets very heavy the man spends most of his time filling the queue from the database and you see the no work sign. It doesn't matter how much work is available to fill his queue.

____________

HAL
Send message
Joined: 28 Mar 03
Posts: 704
Credit: 870,617
RAC: 0
United States
Message 913106 - Posted: 2 Jul 2009, 0:06:26 UTC

thanks for the anology - since the guy is too busy I guess I just shut down!
____________

Classic WU= 7,237 Classic Hours= 42,079

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 913111 - Posted: 2 Jul 2009, 0:16:37 UTC - in response to Message 913106.

thanks for the anology - since the guy is too busy I guess I just shut down!

If you shut down, it'll help those who don't, and it'll help the servers a little.

If you don't shut down, the guy does turn around and face customers and pass out work -- and you've got as good a chance as anybody to get that work.
____________

JohnDKProject donor
Volunteer tester
Avatar
Send message
Joined: 28 May 00
Posts: 862
Credit: 46,465,391
RAC: 71,962
Denmark
Message 913130 - Posted: 2 Jul 2009, 1:03:05 UTC - in response to Message 913091.

And out of all that work available I get a message that the project has no jobs available.
Curiouser and Cusriouser! :)

Nothing funny at all.

There is this job to hand out work and it has a queue. Here is how it works.
A guy behind a window (the job) gets a request for a work unit and he grabs one off the queue and hands it to you. He keep doing this until the queue is empty. When it is empty he hangs up a sign No Jobs Available and swivels his chair around to the database and grabs 100 work units to fill his queue up. When he is done and his queue full again he turns back around and takes the sign down.

When the load is light, the man is facing you most of the time and you get work most of the time. When the load gets very heavy the man spends most of his time filling the queue from the database and you see the no work sign. It doesn't matter how much work is available to fill his queue.

Have checked the status page the last hours, shouldn't the "Results out in the field" get bigger? MBs are again under 4mill, don't think APs have risen.

Last WU I got was about 4 hours ago.

Profile Geek@PlayProject donor
Volunteer tester
Avatar
Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,143,646
RAC: 1,271
United States
Message 913138 - Posted: 2 Jul 2009, 1:30:46 UTC - in response to Message 913106.

thanks for the anology - since the guy is too busy I guess I just shut down!


That's not the answer. Another request just 30 milliseconds later could be granted some work. You just need to keep asking and hit it right.

____________
Boinc....Boinc....Boinc....Boinc....

Profile Shannock9
Avatar
Send message
Joined: 4 Jul 99
Posts: 1305
Credit: 529,327
RAC: 403
United Kingdom
Message 913146 - Posted: 2 Jul 2009, 2:00:08 UTC

Thank you for the videos. They're entertaining but also frustrating, because there's discussion of the slides but in general the slides are not shown. Are the ppts posted anywhere please?
____________

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12702
Credit: 7,190,935
RAC: 15,523
United States
Message 913163 - Posted: 2 Jul 2009, 3:21:25 UTC - in response to Message 913138.

thanks for the anology - since the guy is too busy I guess I just shut down!


That's not the answer. Another request just 30 milliseconds later could be granted some work. You just need to keep asking and hit it right.

BINGO!
That is why there is an automatic random back off built into BOINC.

____________

Andreas
Send message
Joined: 21 Jan 02
Posts: 16
Credit: 9,911,789
RAC: 0
Germany
Message 913241 - Posted: 2 Jul 2009, 11:09:24 UTC - in response to Message 912982.

The Astropulse numbers up until this morning reflected version 5 workunits/results, now they reflect version 5.05.

- Matt


Does this mean AP workunits are now astropulse_v5 version 505 or astropulse_v505 version 505?
Just to get my app_info.xml right :-)

Andreas
____________

Profile [seti.international] Dirk SadowskiProject donor
Volunteer tester
Avatar
Send message
Joined: 6 Apr 07
Posts: 7101
Credit: 60,862,640
RAC: 17,263
Germany
Message 913244 - Posted: 2 Jul 2009, 11:13:30 UTC


AFAIK, AP V5.05 is a new application.
It can't be calculated with the old opt. V5.03 app.

You need to wait that the opt. crew publish a new opt. app.
Or you calculate with stock AP V5.05 .

____________
BR

SETI@home Needs your Help ... $10 & U get a Star!

Team seti.international

Das Deutsche Cafe. The German Cafe.

Profile [seti.international] Dirk SadowskiProject donor
Volunteer tester
Avatar
Send message
Joined: 6 Apr 07
Posts: 7101
Credit: 60,862,640
RAC: 17,263
Germany
Message 913246 - Posted: 2 Jul 2009, 11:16:09 UTC


BTW.

http://setiathome.berkeley.edu/apps.php

____________
BR

SETI@home Needs your Help ... $10 & U get a Star!

Team seti.international

Das Deutsche Cafe. The German Cafe.

Andreas
Send message
Joined: 21 Jan 02
Posts: 16
Credit: 9,911,789
RAC: 0
Germany
Message 913249 - Posted: 2 Jul 2009, 11:20:40 UTC - in response to Message 913244.

Thanks Sutaru, the link definitely answers my initial question.

I understand this http://lunatics.kwsn.net/index.php?module=Downloads;sa=dlview;id=198 as if there already is an opt app for astropulse 505 :-)

Andreas
____________

1 · 2 · 3 · Next

Message boards : Technical News : Increasing Numbers (Jul 01 2009)

Copyright © 2014 University of California