Increasing Numbers (Jul 01 2009)

Message boards : Technical News : Increasing Numbers (Jul 01 2009)
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 912982 - Posted: 1 Jul 2009, 19:38:40 UTC

Sorry about the forums (and other web site features) being shut off for over a day. These Tuesday outages are really taking forever. I guess we've been really busy, which means our tables get ridiculously fragmented throughout the week. Plus I noticed our database is easily 50% larger than it has been about 2 months ago. And the replica lost a couple of its CPUs recently (it's a used/donated system and the CPUs were known to be flakey from the start). Anyway, since the normal recovery procedure was so painful last week I opted to keep all web page database lookups offline until the replica was caught up. Once again, I'm sorry for the inconvenience.

To make up for that, how about some videos from the SETI@home 10 year anniversary? I'll link these to the home page soon enough. Consider this a sneak preview for those who read these threads. Let me know if there are problems downloading/viewing these mpegs.

Data recorder-wise... After all the effort to work with what we got, we're finally throwing in the towel on the current set of data drive enclosures. We have a plan B and plan C already in place - just a matter of deciding which one to enact. Meanwhile, I'm pulling old data off the archives at a pretty good clip - hopefully fast enough to keep up with demand.

Otherwise, I'm still working on NTPCkr and radar stuff. And I adjusted the stats scripts that generate the numbers on the server status page. The Astropulse numbers up until this morning reflected version 5 workunits/results, now they reflect version 5.05.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 912982 · Report as offensive
B-Man
Volunteer tester

Send message
Joined: 11 Feb 01
Posts: 253
Credit: 147,366
RAC: 0
United States
Message 912985 - Posted: 1 Jul 2009, 19:50:02 UTC

Nit-picky comment. It is not June 1 but July 1. I know I want the last month as a do over to.

ID: 912985 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 912997 - Posted: 1 Jul 2009, 20:16:51 UTC - in response to Message 912982.  

The Astropulse numbers up until this morning reflected version 5 workunits/results, now they reflect version 5.05.


Will a new option be added to the preferences page for v5.05 for opt-in and opt-out?
ID: 912997 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30608
Credit: 53,134,872
RAC: 32
United States
Message 913023 - Posted: 1 Jul 2009, 21:09:45 UTC

Thanks for the update and the videos!

Q: Can we expect more day long data driven web page outages?

ID: 913023 · Report as offensive
Invisible Man

Send message
Joined: 24 Jun 01
Posts: 22
Credit: 1,129,336
RAC: 0
United Kingdom
Message 913053 - Posted: 1 Jul 2009, 22:16:55 UTC

Thanks for update Matt.

Quote "01/07/2009 23:00:37 SETI@home Message from server: (Project has no jobs available).

Server status - "Results ready to send 1,568,646".

Any problems here?
ID: 913053 · Report as offensive
JohnDK Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 1222
Credit: 451,243,443
RAC: 1,127
Denmark
Message 913056 - Posted: 1 Jul 2009, 22:21:22 UTC - in response to Message 913053.  
Last modified: 1 Jul 2009, 22:21:43 UTC

Quote "01/07/2009 23:00:37 SETI@home Message from server: (Project has no jobs available).

Server status - "Results ready to send 1,568,646".

Any problems here?

Something about these numbers, they're supposed to be a lot higher if all was working optimal:
Current result creation rate 0.3913/sec 0.7114/sec 5m
ID: 913056 · Report as offensive
HAL

Send message
Joined: 28 Mar 03
Posts: 704
Credit: 870,617
RAC: 0
United States
Message 913060 - Posted: 1 Jul 2009, 22:29:02 UTC - in response to Message 913056.  

And out of all that work available I get a message that the project has no jobs available.
Curiouser and Cusriouser! :)

Classic WU= 7,237 Classic Hours= 42,079
ID: 913060 · Report as offensive
Profile Mike O
Avatar

Send message
Joined: 1 Sep 07
Posts: 428
Credit: 6,670,998
RAC: 0
United States
Message 913085 - Posted: 1 Jul 2009, 23:17:16 UTC

Matt...
You are Da Man!
I always read your thread each day.. I dont always post as..my mouth's not required.
I have talked about this before.. I really think you need to cut your losses and move some of the data from MySQL over to archives.
Your poor servers are trying so hard to keep up but unless you replace them with something that can handle it, its only going to get worse as the tables grow to 100's of gigs.
Your a genious.. Write some code and move the table entries based on date(or something) over to a big ol' hard drive and put it away somewhere safe. Myabe even replicate it just incase.
There is so much data just from the old forums here, it would probably choke my server's 480 gig just trying to deal with that let'alone the results in/out and storage. Good Grief! I can already smell the cpus smoldering.
I understand the need to keep the Results Data.. I have read your posts about this before and it's all clear to me. Breaking that up would make it a MF to put back together so .. best to leave that one alone.

I never did ask.. what do you play? Guitar? Drums?.. Tuba? <-- kidding ;)
I have been in many bands and miss those days at times. (I don't miss the work of hauling the gear to a gig tho!)
Maybe.. one of these days, i'll upload some pics from my 80's haircut bands :)

Anyway... When are you guys going to 'bite the bullet' and get a the power for the server you need? I posted a link a while abck to a killer 8x dual core xeon (16 cores) IBM server with heaps of memory and 3 new raid drives that was listed on Ebay.
This thing was at $999.00 and no one bid on it. You could a had it for 999.10 USD. Sad thing.. it weighed almost 200 pounds! Shipping would cost a chunk!
I wish I could help some way.. It must be a rush when things are going wrong and even more of a rush when things are going 'just right'
Good luck and...
If you took a poll, I think you would find that most here would rather the servers be off for 5 days then take a chance on losing any of the results data.
10 years is an long a## time of collection. I dont think anyone can even touch it. I know a lot are looking forward with huge.. 'on the edge of the chair' eggerness to see this data crunched down. Besides ET.. who knows what this will produce? There may be things in there that no one expected!
crap.. gotta go
--Mike
Not Ready Reading BRAIN. Abort/Retry/Fail?
ID: 913085 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30608
Credit: 53,134,872
RAC: 32
United States
Message 913091 - Posted: 1 Jul 2009, 23:44:40 UTC - in response to Message 913060.  

And out of all that work available I get a message that the project has no jobs available.
Curiouser and Cusriouser! :)

Nothing funny at all.

There is this job to hand out work and it has a queue. Here is how it works.
A guy behind a window (the job) gets a request for a work unit and he grabs one off the queue and hands it to you. He keep doing this until the queue is empty. When it is empty he hangs up a sign No Jobs Available and swivels his chair around to the database and grabs 100 work units to fill his queue up. When he is done and his queue full again he turns back around and takes the sign down.

When the load is light, the man is facing you most of the time and you get work most of the time. When the load gets very heavy the man spends most of his time filling the queue from the database and you see the no work sign. It doesn't matter how much work is available to fill his queue.

ID: 913091 · Report as offensive
HAL

Send message
Joined: 28 Mar 03
Posts: 704
Credit: 870,617
RAC: 0
United States
Message 913106 - Posted: 2 Jul 2009, 0:06:26 UTC

thanks for the anology - since the guy is too busy I guess I just shut down!

Classic WU= 7,237 Classic Hours= 42,079
ID: 913106 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 913111 - Posted: 2 Jul 2009, 0:16:37 UTC - in response to Message 913106.  

thanks for the anology - since the guy is too busy I guess I just shut down!

If you shut down, it'll help those who don't, and it'll help the servers a little.

If you don't shut down, the guy does turn around and face customers and pass out work -- and you've got as good a chance as anybody to get that work.
ID: 913111 · Report as offensive
JohnDK Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 1222
Credit: 451,243,443
RAC: 1,127
Denmark
Message 913130 - Posted: 2 Jul 2009, 1:03:05 UTC - in response to Message 913091.  

And out of all that work available I get a message that the project has no jobs available.
Curiouser and Cusriouser! :)

Nothing funny at all.

There is this job to hand out work and it has a queue. Here is how it works.
A guy behind a window (the job) gets a request for a work unit and he grabs one off the queue and hands it to you. He keep doing this until the queue is empty. When it is empty he hangs up a sign No Jobs Available and swivels his chair around to the database and grabs 100 work units to fill his queue up. When he is done and his queue full again he turns back around and takes the sign down.

When the load is light, the man is facing you most of the time and you get work most of the time. When the load gets very heavy the man spends most of his time filling the queue from the database and you see the no work sign. It doesn't matter how much work is available to fill his queue.

Have checked the status page the last hours, shouldn't the "Results out in the field" get bigger? MBs are again under 4mill, don't think APs have risen.

Last WU I got was about 4 hours ago.
ID: 913130 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 913138 - Posted: 2 Jul 2009, 1:30:46 UTC - in response to Message 913106.  

thanks for the anology - since the guy is too busy I guess I just shut down!


That's not the answer. Another request just 30 milliseconds later could be granted some work. You just need to keep asking and hit it right.

Boinc....Boinc....Boinc....Boinc....
ID: 913138 · Report as offensive
Profile Shannock9
Avatar

Send message
Joined: 4 Jul 99
Posts: 1396
Credit: 634,964
RAC: 0
United Kingdom
Message 913146 - Posted: 2 Jul 2009, 2:00:08 UTC

Thank you for the videos. They're entertaining but also frustrating, because there's discussion of the slides but in general the slides are not shown. Are the ppts posted anywhere please?

ID: 913146 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30608
Credit: 53,134,872
RAC: 32
United States
Message 913163 - Posted: 2 Jul 2009, 3:21:25 UTC - in response to Message 913138.  

thanks for the anology - since the guy is too busy I guess I just shut down!


That's not the answer. Another request just 30 milliseconds later could be granted some work. You just need to keep asking and hit it right.

BINGO!
That is why there is an automatic random back off built into BOINC.

ID: 913163 · Report as offensive
Andreas

Send message
Joined: 21 Jan 02
Posts: 16
Credit: 9,911,789
RAC: 0
Germany
Message 913241 - Posted: 2 Jul 2009, 11:09:24 UTC - in response to Message 912982.  

The Astropulse numbers up until this morning reflected version 5 workunits/results, now they reflect version 5.05.

- Matt


Does this mean AP workunits are now astropulse_v5 version 505 or astropulse_v505 version 505?
Just to get my app_info.xml right :-)

Andreas
ID: 913241 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 913244 - Posted: 2 Jul 2009, 11:13:30 UTC


AFAIK, AP V5.05 is a new application.
It can't be calculated with the old opt. V5.03 app.

You need to wait that the opt. crew publish a new opt. app.
Or you calculate with stock AP V5.05 .

ID: 913244 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 913246 - Posted: 2 Jul 2009, 11:16:09 UTC

ID: 913246 · Report as offensive
Andreas

Send message
Joined: 21 Jan 02
Posts: 16
Credit: 9,911,789
RAC: 0
Germany
Message 913249 - Posted: 2 Jul 2009, 11:20:40 UTC - in response to Message 913244.  

Thanks Sutaru, the link definitely answers my initial question.

I understand this http://lunatics.kwsn.net/index.php?module=Downloads;sa=dlview;id=198 as if there already is an opt app for astropulse 505 :-)

Andreas
ID: 913249 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 913252 - Posted: 2 Jul 2009, 11:26:50 UTC
Last modified: 2 Jul 2009, 11:27:30 UTC


Ahh.. O.K., the opt. crew is very quick! ;-)

I don't crunch AP.. because current I have only a GPU cruncher for SETI@home Enhanced (MB). :-)

ID: 913252 · Report as offensive
1 · 2 · 3 · Next

Message boards : Technical News : Increasing Numbers (Jul 01 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.