Serpentine (Nov 10 2008)


log in

Advanced search

Message boards : Technical News : Serpentine (Nov 10 2008)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1391
Credit: 74,079
RAC: 10
United States
Message 829100 - Posted: 10 Nov 2008, 23:24:13 UTC

Reminder: Tuesday (tomorrow) is a national holiday. We'll be having our weekly outage on Wednesday.

It's been a bit of a rocky weekend. A rocky week, actually - since the last regular Tuesday outage the CPU load on anakin (the scheduling server) has been kinda high. Like around 100. The obvious answer - turning on client logging for debugging on Tuesday - wasn't so obvious at first as we thought we vindicated that and moved on to finding other possible problems which were all red herrings. Eventually we were brought back to client logging and Dave made an optimization fix which we tested in beta this morning, and Jeff applied to the public project around noon. This brought the load back down to 1. I guess the fix worked.

Our raw data pipeline management still needs work. Lots of bottlenecks make it impossible to keep a steady, automatic flow of work. In a perfect world it would be simple and serial - data drives sent up here, files brought online and acted upon by both multibeam and astropulse splitters, copied down to archival off-site storage, and then deleted. However each step takes a rather long time (hours per file if not days), and storage is limited, so we have to parallelize as much as possible. One possible effect of this, and one we're seeing now, is that we currently don't have raw data available for astropulse to split. We're loathe to copy data back up from the archives unless we really have to. We still might do so, but we are expecting a new shipment of drives directly from Arecibo today or Wednesday so astropulse at least be will be fed then.

The bright side is this is now very clear on the server status page now that I made some updates to finally split out database counts/rates and splitter activity per application. There's still more updates to be done, but now it's much easier to tell what's going on between the two.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 530,903
RAC: 336
United States
Message 829107 - Posted: 10 Nov 2008, 23:50:14 UTC

It sounds like Anakin has gone over to the dark side...
____________


BOINC WIKI

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 2007
Credit: 11,248,622
RAC: 15,243
United States
Message 829109 - Posted: 11 Nov 2008, 0:05:09 UTC - in response to Message 829107.

It sounds like Anakin has gone over to the dark side...


WWEELLLL, Anakin Skywalker is Darth Vader before the dark side... ;)
____________
.

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2357
Credit: 8,953,535
RAC: 3,946
United States
Message 829207 - Posted: 11 Nov 2008, 6:42:46 UTC

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

Looking forward to more AP's to crunch. I like the RAC boost they give when they validate. :D
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

Josef W. SegurProject donor
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4348
Credit: 1,128,170
RAC: 930
United States
Message 829299 - Posted: 11 Nov 2008, 17:19:49 UTC - in response to Message 829207.

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

Looking forward to more AP's to crunch. I like the RAC boost they give when they validate. :D

The AP splitters were producing new work until some time on Sunday, now the only results being created are reissues. The change before that was the -allapps Feeder argument, which was turned on Oct. 30. That configuration has a "weight" for the WUs for each app, and the Feeder tries to tell the Scheduler about work in the proprtion implied by those weights. The weight for AP work is apparently higher than the relative production rate of the ap_splitter set vs. the mb_splitter set, so any extra in the "Ready to send" queue was soon exhausted. AP work ended up being sent just as it was created. That is, there was probably one new AP WU and the two associated tasks about every 7 seconds. If your host happened to ask at the right time it would get one AP task, the odds being approximately one in however many (hundreds?) of work requests the Scheduler receives each 7 seconds.

Before that Oct. 30 change, the AP work was largely being delivered in bursts such that a host with a large enough work appetite could get several for one request.
Joe

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2357
Credit: 8,953,535
RAC: 3,946
United States
Message 829304 - Posted: 11 Nov 2008, 17:33:18 UTC - in response to Message 829299.

That's right, I forgot about -allapps already. I noticed last week after it was turned on for a few days that the cricket graph looked pretty stable instead of having 95mbit spikes (presumably the AP WUs being DLed) for half an hour at a time.
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

Bob Merrill
Send message
Joined: 7 Jun 99
Posts: 109
Credit: 3,357,942
RAC: 759
United States
Message 829381 - Posted: 11 Nov 2008, 22:33:24 UTC - in response to Message 829207.

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?
____________

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 13197
Credit: 7,966,366
RAC: 15,835
United States
Message 829402 - Posted: 11 Nov 2008, 23:36:28 UTC - in response to Message 829381.

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?

http://setiweb.ssl.berkeley.edu/beta/forum_thread.php?id=942



____________

Profile arkaynProject donor
Volunteer tester
Avatar
Send message
Joined: 14 May 99
Posts: 3747
Credit: 48,777,915
RAC: 1,076
United States
Message 829454 - Posted: 12 Nov 2008, 2:06:51 UTC - in response to Message 829381.

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?


Better would be this one, optimized for the Mac and made by Crunch3r.

http://www.arkayn.us/mf/ap_4.35-windows_x86_64_sse3.zip

____________

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 13197
Credit: 7,966,366
RAC: 15,835
United States
Message 829545 - Posted: 12 Nov 2008, 5:15:01 UTC - in response to Message 829454.

So that must be why I've been seeing Astropulse less frequently in my queues in the past week-ish. My main cruncher usually has 10-20 of them, and I've got only four right now with a ton of short multibeams in between.

I have never seen an Astropulse on my Mac. When are we going to get a client for the Mac that can crunch AP units?


Better would be this one, optimized for the Mac and made by Crunch3r.

http://www.arkayn.us/mf/ap_4.35-windows_x86_64_sse3.zip

I don't believe that is a Mac application with windows_x86 in it's name.
Less so since Crunch3r's Seti App's page doesn't say he develops for Mac.
Even less since the Mac O/S doesn't see it as an executable file.

Could be wrong.


____________

6dj72cn8
Send message
Joined: 3 Sep 99
Posts: 24
Credit: 163,811
RAC: 0
Australia
Message 829558 - Posted: 12 Nov 2008, 6:22:56 UTC

I think this is the link arkayn intended to provide:
http://calbe.dw70.de/astrop/astropulse-4.28.i686-apple-darwin_sse3.zip
____________

Profile arkaynProject donor
Volunteer tester
Avatar
Send message
Joined: 14 May 99
Posts: 3747
Credit: 48,777,915
RAC: 1,076
United States
Message 829597 - Posted: 12 Nov 2008, 12:43:02 UTC

I was half asleep last night when I posted that, I forgot that I was hosting the windows version.
____________

Message boards : Technical News : Serpentine (Nov 10 2008)

Copyright © 2014 University of California