Serpentine (Nov 10 2008)

Message boards : Technical News : Serpentine (Nov 10 2008)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 829100 - Posted: 10 Nov 2008, 23:24:13 UTC

Reminder: Tuesday (tomorrow) is a national holiday. We'll be having our weekly outage on Wednesday.

It's been a bit of a rocky weekend. A rocky week, actually - since the last regular Tuesday outage the CPU load on anakin (the scheduling server) has been kinda high. Like around 100. The obvious answer - turning on client logging for debugging on Tuesday - wasn't so obvious at first as we thought we vindicated that and moved on to finding other possible problems which were all red herrings. Eventually we were brought back to client logging and Dave made an optimization fix which we tested in beta this morning, and Jeff applied to the public project around noon. This brought the load back down to 1. I guess the fix worked.

Our raw data pipeline management still needs work. Lots of bottlenecks make it impossible to keep a steady, automatic flow of work. In a perfect world it would be simple and serial - data drives sent up here, files brought online and acted upon by both multibeam and astropulse splitters, copied down to archival off-site storage, and then deleted. However each step takes a rather long time (hours per file if not days), and storage is limited, so we have to parallelize as much as possible. One possible effect of this, and one we're seeing now, is that we currently don't have raw data available for astropulse to split. We're loathe to copy data back up from the archives unless we really have to. We still might do so, but we are expecting a new shipment of drives directly from Arecibo today or Wednesday so astropulse at least be will be fed then.

The bright side is this is now very clear on the server status page now that I made some updates to finally split out database counts/rates and splitter activity per application. There's still more updates to be done, but now it's much easier to tell what's going on between the two.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 829100 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 829107 - Posted: 10 Nov 2008, 23:50:14 UTC

It sounds like Anakin has gone over to the dark side...


BOINC WIKI
ID: 829107 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 829109 - Posted: 11 Nov 2008, 0:05:09 UTC - in response to Message 829107.  

It sounds like Anakin has gone over to the dark side...


WWEELLLL, Anakin Skywalker is Darth Vader before the dark side... ;)
.

Hello, from Albany, CA!...
ID: 829109 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 829207 - Posted: 11 Nov 2008, 6:42:46 UTC

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

Looking forward to more AP's to crunch. I like the RAC boost they give when they validate. :D
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 829207 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 829299 - Posted: 11 Nov 2008, 17:19:49 UTC - in response to Message 829207.  

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

Looking forward to more AP's to crunch. I like the RAC boost they give when they validate. :D

The AP splitters were producing new work until some time on Sunday, now the only results being created are reissues. The change before that was the -allapps Feeder argument, which was turned on Oct. 30. That configuration has a "weight" for the WUs for each app, and the Feeder tries to tell the Scheduler about work in the proprtion implied by those weights. The weight for AP work is apparently higher than the relative production rate of the ap_splitter set vs. the mb_splitter set, so any extra in the "Ready to send" queue was soon exhausted. AP work ended up being sent just as it was created. That is, there was probably one new AP WU and the two associated tasks about every 7 seconds. If your host happened to ask at the right time it would get one AP task, the odds being approximately one in however many (hundreds?) of work requests the Scheduler receives each 7 seconds.

Before that Oct. 30 change, the AP work was largely being delivered in bursts such that a host with a large enough work appetite could get several for one request.
                                                            Joe
ID: 829299 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 829304 - Posted: 11 Nov 2008, 17:33:18 UTC - in response to Message 829299.  

That's right, I forgot about -allapps already. I noticed last week after it was turned on for a few days that the cricket graph looked pretty stable instead of having 95mbit spikes (presumably the AP WUs being DLed) for half an hour at a time.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 829304 · Report as offensive
Bob Merrill

Send message
Joined: 7 Jun 99
Posts: 120
Credit: 8,531,677
RAC: 19
United States
Message 829381 - Posted: 11 Nov 2008, 22:33:24 UTC - in response to Message 829207.  

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?
ID: 829381 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30608
Credit: 53,134,872
RAC: 32
United States
Message 829402 - Posted: 11 Nov 2008, 23:36:28 UTC - in response to Message 829381.  

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?

http://setiweb.ssl.berkeley.edu/beta/forum_thread.php?id=942



ID: 829402 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 829454 - Posted: 12 Nov 2008, 2:06:51 UTC - in response to Message 829381.  

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?


Better would be this one, optimized for the Mac and made by Crunch3r.

http://www.arkayn.us/mf/ap_4.35-windows_x86_64_sse3.zip


ID: 829454 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30608
Credit: 53,134,872
RAC: 32
United States
Message 829545 - Posted: 12 Nov 2008, 5:15:01 UTC - in response to Message 829454.  

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?


Better would be this one, optimized for the Mac and made by Crunch3r.

http://www.arkayn.us/mf/ap_4.35-windows_x86_64_sse3.zip

I don't believe that is a Mac application with windows_x86 in it's name.
Less so since Crunch3r's Seti App's page doesn't say he develops for Mac.
Even less since the Mac O/S doesn't see it as an executable file.

Could be wrong.


ID: 829545 · Report as offensive
6dj72cn8

Send message
Joined: 3 Sep 99
Posts: 24
Credit: 163,811
RAC: 0
Australia
Message 829558 - Posted: 12 Nov 2008, 6:22:56 UTC

I think this is the link arkayn intended to provide:
http://calbe.dw70.de/astrop/astropulse-4.28.i686-apple-darwin_sse3.zip
ID: 829558 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 829597 - Posted: 12 Nov 2008, 12:43:02 UTC

I was half asleep last night when I posted that, I forgot that I was hosting the windows version.

ID: 829597 · Report as offensive

Message boards : Technical News : Serpentine (Nov 10 2008)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.