Curious about AP

Message boards : Number crunching : Curious about AP
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 5 · Next

AuthorMessage
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 964217 - Posted: 17 Jan 2010, 22:06:49 UTC

Almost 120,000 AP's are still awaiting validation, and the AP validators are *NOT* off (unlike the AP assimilators). As there are only 250 AP results yet to be returned, the math seems waaaaay off ;-)

As we're past all reasonable 30 day time outs, might these be AP Wingmen who bailed and for some reason these AP's weren't re-issued? Or might the number represent that in the AP DB woes, about 120K in-process records got trashed? Other thoughts?
ID: 964217 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 964233 - Posted: 17 Jan 2010, 23:52:38 UTC

Waiting for validation are WUs that were processed and returned, had two wingmen do the same thing, and were waiting for the validator to grant credit and move on to assimilation. Since the AP database is not available, the validator can't access the database to grant the credit (I think..) and move the WU on to the assimilator, which is disabled.

Once the database gets fixed and becomes available again, credit will be granted, WUs will be validated, and then assimilated, then purged.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 964233 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 964244 - Posted: 18 Jan 2010, 0:39:24 UTC - in response to Message 964233.  
Last modified: 18 Jan 2010, 0:40:47 UTC

the validator can't access the database to grant the credit

I'm not so sure about that. Credit is tracked elsewhere (User ID), thus those AP's should get validated fine by the running AP validator processes, credited, and then moved off into the assimilator queue (which, yes, perhaps those 3 steps are atomically applied, thus blocking --- In which case, why leave the AP validators on when the rest of AP is shut down (presumably not to avoid time-outs as the DB's & services exhibit plenty of downtime as it is without such timeouts occuring, thus that potential is already architected in)?).
ID: 964244 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 964247 - Posted: 18 Jan 2010, 0:51:37 UTC
Last modified: 18 Jan 2010, 0:53:49 UTC

I had some AP work units that were uploaded in early Dec. just go through validation. wuid 534401745 was finished on Dec 1st & just validated a day or two ago. I've got a total of 3 that validated the past few days over the 3 machines I have enabled for AP.

I figure it's part of their ongoing process of trying to fix the few corrupt rows in the AP database.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 964247 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 965951 - Posted: 26 Jan 2010, 2:11:37 UTC

Just in case nobody noticed.. AP has been turned back on with the exception of the splitters. There are results and WUs that are in the purge queue now, and the "awaiting validation" count is shrinking.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 965951 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 966221 - Posted: 27 Jan 2010, 22:36:53 UTC

Astropulse Splitters are now online. :)

Claggy
ID: 966221 · Report as offensive
Profile skildude
Avatar

Send message
Joined: 4 Oct 00
Posts: 9541
Credit: 50,759,529
RAC: 60
Yemen
Message 966223 - Posted: 27 Jan 2010, 22:41:11 UTC - in response to Message 966221.  

I'll be looking for work when I get home


In a rich man's house there is no place to spit but his face.
Diogenes Of Sinope
ID: 966223 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 966226 - Posted: 27 Jan 2010, 23:16:00 UTC - in response to Message 966223.  

I'll be looking for work when I get home

Form an orderly queue - the Cricket Graph has jumped from below 50 Mbit/sec (daily average) to over 85 Mbit/sec already.
ID: 966226 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 966230 - Posted: 27 Jan 2010, 23:59:51 UTC - in response to Message 966226.  

... and they're splitting AP's from *OLD* tapes !! That potentially opens up a floodgate of AP packets all the way back to 2005 (unless there is something special about March 2007 and forward).
ID: 966230 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 966239 - Posted: 28 Jan 2010, 1:07:58 UTC

I was more interested in the "results in the field".. this morning before the splitters were turned on it was 65. Just 65. Now look at it. each one of those is 8MB pushed across that link.
Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 966239 · Report as offensive
Profile rebest Project Donor
Volunteer tester
Avatar

Send message
Joined: 16 Apr 00
Posts: 1296
Credit: 45,357,093
RAC: 0
United States
Message 966253 - Posted: 28 Jan 2010, 2:10:04 UTC - in response to Message 966239.  

I was more interested in the "results in the field".. this morning before the splitters were turned on it was 65. Just 65. Now look at it. each one of those is 8MB pushed across that link.


Uh oh! Over 40 new AP results returned in the last hour. That probably means errors.


Join the PACK!
ID: 966253 · Report as offensive
Cosmic_Ocean
Avatar

Send message
Joined: 23 Dec 00
Posts: 3027
Credit: 13,516,867
RAC: 13
United States
Message 966310 - Posted: 28 Jan 2010, 4:25:34 UTC - in response to Message 966253.  

I was more interested in the "results in the field".. this morning before the splitters were turned on it was 65. Just 65. Now look at it. each one of those is 8MB pushed across that link.


Uh oh! Over 40 new AP results returned in the last hour. That probably means errors.

I'm thinking so..

Result turnaround time (last hour average) 1.70 hours

Linux laptop:
record uptime: 1511d 20h 19m (ended due to the power brick giving-up)
ID: 966310 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 966313 - Posted: 28 Jan 2010, 4:31:49 UTC - in response to Message 966310.  

I was more interested in the "results in the field".. this morning before the splitters were turned on it was 65. Just 65. Now look at it. each one of those is 8MB pushed across that link.


Uh oh! Over 40 new AP results returned in the last hour. That probably means errors.

I'm thinking so..

Result turnaround time (last hour average) 1.70 hours

Maybe they are just AP shorties? Anyone seen a result form the new ones yet? Only for 1 on all of my machines. I might force it to run tomorrow when I get into work to see if it fails quickly.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 966313 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 966370 - Posted: 28 Jan 2010, 11:07:34 UTC - in response to Message 966253.  

Uh oh! Over 40 new AP results returned in the last hour. That probably means errors.

Agreed, and since then the return % rate has tripled with very little passage of time. Something seems amiss with the AP's being split from the scrubbed '07 data.

A few threads back, SkilDude said something about these older tapes (and they might have actually still been tapes back then) being too noisy for AP. So perhaps that's still the case despite radar blanking, if these actually are erroring out.

Ok, a round of kudos for anyone who can provide a link to one of these AP result sets ;-)
ID: 966370 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 966415 - Posted: 28 Jan 2010, 15:57:14 UTC

Something I never understood about AP....

When AP work is being split and sent, the Cricket graphs show the increased server bandwidth. Given the increased size of the AP WUs, this is not too surprising, at first.

But wouldn't the increased crunch time of AP work eventually offset the increased bandwidth to send them out and settle down a bit once the pipeline is filled? Or does the percentage of increased bandwidth to send them exceed the increase in the time to crunch them?

This of course, would be if they were crunching normally and not showing signs of possible error problems as is currently being discussed.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 966415 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1643
Credit: 12,921,799
RAC: 89
New Zealand
Message 966472 - Posted: 28 Jan 2010, 20:25:39 UTC - in response to Message 966415.  



When AP work is being split and sent, the Cricket graphs show the increased server bandwidth. Given the increased size of the AP WUs, this is not too surprising, at first.

But wouldn't the increased crunch time of AP work eventually offset the increased bandwidth to send them out and settle down a bit once the pipeline is filled? Or does the percentage of increased bandwidth to send them exceed the increase in the time to crunch them?

That's the way I understand it.

ID: 966472 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 966475 - Posted: 28 Jan 2010, 20:34:40 UTC - in response to Message 966472.  



When AP work is being split and sent, the Cricket graphs show the increased server bandwidth. Given the increased size of the AP WUs, this is not too surprising, at first.

But wouldn't the increased crunch time of AP work eventually offset the increased bandwidth to send them out and settle down a bit once the pipeline is filled? Or does the percentage of increased bandwidth to send them exceed the increase in the time to crunch them?

That's the way I understand it.

8mb data vs 300-400kb data. So AP tasks are about 20 times more data to transfer, but only require 10 times the amount of time to process.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 966475 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 966538 - Posted: 29 Jan 2010, 2:23:04 UTC - in response to Message 966475.  



When AP work is being split and sent, the Cricket graphs show the increased server bandwidth. Given the increased size of the AP WUs, this is not too surprising, at first.

But wouldn't the increased crunch time of AP work eventually offset the increased bandwidth to send them out and settle down a bit once the pipeline is filled? Or does the percentage of increased bandwidth to send them exceed the increase in the time to crunch them?

That's the way I understand it.

8mb data vs 300-400kb data. So AP tasks are about 20 times more data to transfer, but only require 10 times the amount of time to process.

Ahh....
And here all along I was under the impression that AP work was bandwidth sparing...
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 966538 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 966557 - Posted: 29 Jan 2010, 4:05:54 UTC - in response to Message 966538.  



When AP work is being split and sent, the Cricket graphs show the increased server bandwidth. Given the increased size of the AP WUs, this is not too surprising, at first.

But wouldn't the increased crunch time of AP work eventually offset the increased bandwidth to send them out and settle down a bit once the pipeline is filled? Or does the percentage of increased bandwidth to send them exceed the increase in the time to crunch them?

That's the way I understand it.

8mb data vs 300-400kb data. So AP tasks are about 20 times more data to transfer, but only require 10 times the amount of time to process.

Ahh....
And here all along I was under the impression that AP work was bandwidth sparing...

cursed math getting in the way!
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 966557 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 966560 - Posted: 29 Jan 2010, 4:10:29 UTC - in response to Message 966557.  



When AP work is being split and sent, the Cricket graphs show the increased server bandwidth. Given the increased size of the AP WUs, this is not too surprising, at first.

But wouldn't the increased crunch time of AP work eventually offset the increased bandwidth to send them out and settle down a bit once the pipeline is filled? Or does the percentage of increased bandwidth to send them exceed the increase in the time to crunch them?

That's the way I understand it.

8mb data vs 300-400kb data. So AP tasks are about 20 times more data to transfer, but only require 10 times the amount of time to process.

Ahh....
And here all along I was under the impression that AP work was bandwidth sparing...

cursed math getting in the way!

Indeed......
This was a major misunderstanding on my part from the launch of AP.
Don't know quite where I got it from, but I always thought that the massive crunch times of AP compared to MB were gonna save some work on the server's part...
But now I have been enlightened.

No matter, AP is enabled on all of my rigs, and the kitties shall be happy to process any work the servers send our way.

Meow meow.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 966560 · Report as offensive
1 · 2 · 3 · 4 . . . 5 · Next

Message boards : Number crunching : Curious about AP


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.