Forcing Download causes quick completion error?

Author	Message
Guy Send message Joined: 17 May 99 Posts: 41 Credit: 1,796,972 RAC: 13,151	Message 1329863 - Posted: 21 Jan 2013, 17:59:57 UTC
Hello, In the BOINC clients 'Transfers' tab there is a [Retry Now] button. As the download speeds for SETI are slow and indeed stop altogether, I've been pressing this to complete the download.. Now I'm running LHC as well. WUs from this project are taken only by the dual core CPU (no OpenCL for LHC) whereas I've made SETI take only GPU WUs. So the set-up looks like this in the 'Tasks' tab: SETI@home PROGRESS Running (0.771 CPUs + 1 ATI GPU (device 1)) ... LHC@home PROGRESS Running ... LHC@home PROGRESS Running ... (Despite having two physical GPU cards, only one is scheduled if both CPU cores are in use.) OK. When I accelerate the download like this, on download completion the task is whipped up by the spare GPU and 'finishes' in < 5 seconds - 21/01/2013 16:27:27 \| SETI@home \| Finished download of ap_30no12ac_B3_P1_00353_20130121_14377.wu 21/01/2013 16:27:27 \| SETI@home \| Starting task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 using astropulse_v6 version 604 (opencl_ati_100) in slot 1 21/01/2013 16:27:32 \| SETI@home \| Computation for task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 finished 21/01/2013 16:27:32 \| LHC@home 1.0 \| Restarting task W0w7cbb_w7cbb__44__s__64.31_59.32__13_13.5__6__13.5_1_sixvf_boinc105390_0 using sixtrack version 44401 (pni) in slot 0 21/01/2013 16:27:35 \| SETI@home \| Started upload of ap_30no12ac_B3_P1_00353_20130121_14377.wu_1_0 21/01/2013 16:27:42 \| SETI@home \| Finished upload of ap_30no12ac_B3_P1_00353_20130121_14377.wu_1_0 Is this a computational error? Thank you. ____________
ID: 1329863 ·

Claggy Volunteer tester Send message Joined: 5 Jul 99 Posts: 3500 Credit: 26,343,301 RAC: 16,543	Message 1329904 - Posted: 21 Jan 2013, 18:58:30 UTC - in response to Message 1329863. Last modified: 21 Jan 2013, 19:00:46 UTC
When I accelerate the download like this, on download completion the task is whipped up by the spare GPU and 'finishes' in < 5 seconds - 21/01/2013 16:27:27 \| SETI@home \| Starting task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 using astropulse_v6 version 604 (opencl_ati_100) in slot 1 21/01/2013 16:27:32 \| SETI@home \| Computation for task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 finished Is this a computational error? Thank you. No, because of what band it is, ie B3_P1, it is probably a 100% Blanked task, so it is not worth running, you're not reported it yet so we can't be 100% sure of that yet, a lot of tasks (but not all) with eithier B3_P1 or B6_P0 end like this. Claggy
ID: 1329904 ·

Guy Send message Joined: 17 May 99 Posts: 41 Credit: 1,796,972 RAC: 13,151	Message 1329913 - Posted: 21 Jan 2013, 19:18:41 UTC - in response to Message 1329904.
Thanks, Claggy. That's a load off my mind. But - when you say not worth running Why? -[/quote] ____________
ID: 1329913 ·

Claggy Volunteer tester Send message Joined: 5 Jul 99 Posts: 3500 Credit: 26,343,301 RAC: 16,543	Message 1329928 - Posted: 21 Jan 2013, 19:44:38 UTC - in response to Message 1329913.
Thanks, Claggy. That's a load off my mind. But - when you say not worth running Why? - Because of RADAR interference they Blank some of the Data on the Wu with white noise, it gets to a point where there's not much point looking for signals, as 100% of the Wu is now just white noise, Claggy
ID: 1329928 ·

Guy Send message Joined: 17 May 99 Posts: 41 Credit: 1,796,972 RAC: 13,151	Message 1329935 - Posted: 21 Jan 2013, 20:43:42 UTC - in response to Message 1329928.
I see. Thanks. And I can confirm all is well with BOTH GPUs on SETI and the old dual-core simulating 4 cores (cc_config.xml) with LHC - although when both GPUs are running, one of the simulated cores is given over to serving them, whereas with only one GPU loaded, the set-up perseveres in a 4xCPU+1xGPU configuration. For the moment anyhow. I'm uncertain as to whether the simulation of 4 CPUs produces a benefit yet... We'll see. You know, it occurs to me that these radar affected WUs might be issued in a size reduced form. It sure would make DLs faster! Minor Digression - If anyone could possibly shed some light on this... Why can't the WUs subjected to 'blanking' be smaller? I mean why not? What is the issue with DownLoading from SETI@home? Is it just a simple lack of bandwidth? -That last one might cause a few groans from our regular readers, but I've never seen a simple statement explaining it. Thanks - ____________
ID: 1329935 ·

Lionel Send message Joined: 25 Mar 00 Posts: 464 Credit: 152,489,359 RAC: 121,905	Message 1329970 - Posted: 21 Jan 2013, 23:05:59 UTC - in response to Message 1329935.
Guy Don't worry about the circa 5 second WUs. You will get them every now and then. I had a bunch of them last week. If anything, they are slightly annoying given the current situation re download speeds and times. cheers ____________
ID: 1329970 ·

Guy Send message Joined: 17 May 99 Posts: 41 Credit: 1,796,972 RAC: 13,151	Message 1330049 - Posted: 22 Jan 2013, 3:40:28 UTC - in response to Message 1329970.
Thank you for your calming words. I guess it was coincidence. It did - you may tell - worry me! I've watched it and 'seen' the effect and it makes sence now. Thanks. BOINC Rocks! ____________
ID: 1330049 ·

Ianab Volunteer tester Send message Joined: 11 Jun 08 Posts: 568 Credit: 9,358,280 RAC: 11,726	Message 1330082 - Posted: 22 Jan 2013, 6:43:28 UTC - in response to Message 1329935.
Why can't the WUs subjected to 'blanking' be smaller? I mean why not? It takes processing power to work out if the WU is only noise. It takes your machine 5 secs to decide, I'm guessing it would take a server at the lab a similar amount of time. As 10-15 WUs are produced per second, they don't have the spare CPU power to "pre-process" everything. End result, they get sent out, and the machines in the field sorts them out. Not ideal, but there isn't a simple way around this. Ian
ID: 1330082 ·

Lionel Send message Joined: 25 Mar 00 Posts: 464 Credit: 152,489,359 RAC: 121,905	Message 1330094 - Posted: 22 Jan 2013, 8:44:20 UTC - in response to Message 1330082.
Ianab It may only take 5 seconds to sort out at our end but in the current climate it can can up to 10 - 20 minutes or more to get the WU and if you get more than 1 at once as I was last week it becomes very annoying. To the Head Shed Can we lift the limits from 100 to 150 to push out some of the demand from lower order machines?? cheers ____________
ID: 1330094 ·

Guy Send message Joined: 17 May 99 Posts: 41 Credit: 1,796,972 RAC: 13,151	Message 1330148 - Posted: 22 Jan 2013, 13:44:14 UTC
I'm missing something - precisely where in the pipe from Arecibo to my front room, this 'noise' is applied to the signals we process. And is that assumption correct? The signal data is modified like someone going through a sensitive document with a black marker pen. Raise your flags! Just kidding. *** Tangentially- In Message 1329935 I ranted about some configuration nonsense - I have stopped emulation of 4 cores on my Opteron 185. I'm just letting them run native for a better, more adaptable system. Well, for my set-up it is. Tally Ho! ____________
ID: 1330148 ·

Josef W. Segur Volunteer developer Volunteer tester Send message Joined: 30 Oct 99 Posts: 3968 Credit: 937,375 RAC: 148	Message 1330278 - Posted: 23 Jan 2013, 1:47:07 UTC - in response to Message 1330148.
I'm missing something - precisely where in the pipe from Arecibo to my front room, this 'noise' is applied to the signals we process. And is that assumption correct? The signal data is modified like someone going through a sensitive document with a black marker pen. ... Two places (for Astropulse). The first method of replacing data affected by RFI is done at Berkeley. When they get a 2 TB disk full of data from Arecibo it is broken up into ~50.2 GB files. Before those files are made available for splitting there's another process which reads the data and checks whether multiple channels have patterns matching those produced by the RADARs in Puerto Rico. A 15th channel gets the signal indicating good or bad data. Then when either an mb_splitter or ap_splitter is reading in data to split, the bad sections are replaced with pseudo-random data which has been shaped to match the frequency distribution of the good data. The second method is within the Astropulse applications. The RFI detection method is different, based on the level in the DC bin of an FFT (probably detecting partial overload of the receiver IMO). The replacement data is like that used server-side. For each detection of RFI, it makes sense to replace some data both before and after the detection point. The server-side replacement range is less than that in the Astropulse application. Another obvious difference is that MB processing is narrow band, so the AP detection method wouldn't make sense. There are obvious possibilities to do things more efficiently, and at least one is being worked on. Code is being developed to not split and distribute data if it's too bad. If more than 0.28% of volunteers would donate cash to the project, it might be possible to get more staff so development of such improvements would go faster. Joe
ID: 1330278 ·

Guy Send message Joined: 17 May 99 Posts: 41 Credit: 1,796,972 RAC: 13,151	Message 1330461 - Posted: 23 Jan 2013, 19:37:43 UTC - in response to Message 1330278.
Thank you. ____________
ID: 1330461 ·

Author

Message

Send message
Joined: 17 May 99
Posts: 41
Credit: 1,796,972
RAC: 13,151

Hello,
In the BOINC clients 'Transfers' tab there is a [Retry Now] button. As the download speeds for SETI are slow and indeed stop altogether, I've been pressing this to complete the download.. Now I'm running LHC as well. WUs from this project are taken only by the dual core CPU (no OpenCL for LHC) whereas I've made SETI take only GPU WUs. So the set-up looks like this in the 'Tasks' tab:

SETI@home PROGRESS Running (0.771 CPUs + 1 ATI GPU (device 1)) ...
LHC@home PROGRESS Running ...
LHC@home PROGRESS Running ...

(Despite having two physical GPU cards, only one is scheduled if both CPU cores are in use.)
OK. When I accelerate the download like this, on download completion the task is whipped up by the spare GPU and 'finishes' in < 5 seconds -

21/01/2013 16:27:27 | SETI@home | Finished download of ap_30no12ac_B3_P1_00353_20130121_14377.wu
21/01/2013 16:27:27 | SETI@home | Starting task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 using astropulse_v6 version 604 (opencl_ati_100) in slot 1
21/01/2013 16:27:32 | SETI@home | Computation for task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 finished
21/01/2013 16:27:32 | LHC@home 1.0 | Restarting task W0w7cbb_w7cbb__44__s__64.31_59.32__13_13.5__6__13.5_1_sixvf_boinc105390_0 using sixtrack version 44401 (pni) in slot 0
21/01/2013 16:27:35 | SETI@home | Started upload of ap_30no12ac_B3_P1_00353_20130121_14377.wu_1_0
21/01/2013 16:27:42 | SETI@home | Finished upload of ap_30no12ac_B3_P1_00353_20130121_14377.wu_1_0

Is this a computational error?
Thank you.
____________

ID: 1329863 ·

Claggy

Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 3500
Credit: 26,343,301
RAC: 16,543

When I accelerate the download like this, on download completion the task is whipped up by the spare GPU and 'finishes' in < 5 seconds -

21/01/2013 16:27:27 | SETI@home | Starting task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 using astropulse_v6 version 604 (opencl_ati_100) in slot 1
21/01/2013 16:27:32 | SETI@home | Computation for task ap_30no12ac_B3_P1_00353_20130121_14377.wu_1 finished

Is this a computational error?
Thank you.

No, because of what band it is, ie B3_P1, it is probably a 100% Blanked task, so it is not worth running, you're not reported it yet so we can't be 100% sure of that yet,
a lot of tasks (but not all) with eithier B3_P1 or B6_P0 end like this.

Claggy

ID: 1329904 ·

Guy

Send message
Joined: 17 May 99
Posts: 41
Credit: 1,796,972
RAC: 13,151

Thanks, Claggy. That's a load off my mind. But - when you say

not worth running

Why? -[/quote]
____________

ID: 1329913 ·

Claggy

Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 3500
Credit: 26,343,301
RAC: 16,543

Thanks, Claggy. That's a load off my mind. But - when you say

not worth running

Why? -

Because of RADAR interference they Blank some of the Data on the Wu with white noise, it gets to a point where there's not much point looking for signals, as 100% of the Wu is now just white noise,

Claggy

ID: 1329928 ·

Guy

Send message
Joined: 17 May 99
Posts: 41
Credit: 1,796,972
RAC: 13,151

I see. Thanks. And I can confirm all is well with BOTH GPUs on SETI and the old dual-core simulating 4 cores (cc_config.xml) with LHC - although when both GPUs are running, one of the simulated cores is given over to serving them, whereas with only one GPU loaded, the set-up perseveres in a 4xCPU+1xGPU configuration. For the moment anyhow. I'm uncertain as to whether the simulation of 4 CPUs produces a benefit yet... We'll see.

You know, it occurs to me that these radar affected WUs might be issued in a size reduced form. It sure would make DLs faster!

Minor Digression - If anyone could possibly shed some light on this...

why not

-That last one might cause a few groans from our regular readers, but I've never seen a simple statement explaining it.

Thanks -

____________

ID: 1329935 ·

Lionel
Send message
Joined: 25 Mar 00
Posts: 464
Credit: 152,489,359
RAC: 121,905

Guy

Don't worry about the circa 5 second WUs. You will get them every now and then. I had a bunch of them last week. If anything, they are slightly annoying given the current situation re download speeds and times.

cheers
____________

ID: 1329970 ·

Guy

Send message
Joined: 17 May 99
Posts: 41
Credit: 1,796,972
RAC: 13,151

Thank you for your calming words. I guess it was coincidence. It did - you may tell - worry me! I've watched it and 'seen' the effect and it makes sence now.
Thanks.
BOINC Rocks!
____________

ID: 1330049 ·

Ianab
Volunteer tester
Send message
Joined: 11 Jun 08
Posts: 568
Credit: 9,358,280
RAC: 11,726

Why can't the WUs subjected to 'blanking' be smaller? I mean why not?

It takes processing power to work out if the WU is only noise. It takes your machine 5 secs to decide, I'm guessing it would take a server at the lab a similar amount of time. As 10-15 WUs are produced per second, they don't have the spare CPU power to "pre-process" everything.

End result, they get sent out, and the machines in the field sorts them out. Not ideal, but there isn't a simple way around this.

Ian

ID: 1330082 ·

Lionel
Send message
Joined: 25 Mar 00
Posts: 464
Credit: 152,489,359
RAC: 121,905

Ianab
It may only take 5 seconds to sort out at our end but in the current climate it can can up to 10 - 20 minutes or more to get the WU and if you get more than 1 at once as I was last week it becomes very annoying.

To the Head Shed
Can we lift the limits from 100 to 150 to push out some of the demand from lower order machines??

cheers
____________

ID: 1330094 ·

Guy

Send message
Joined: 17 May 99
Posts: 41
Credit: 1,796,972
RAC: 13,151

I'm missing something - precisely where in the pipe from Arecibo to my front room, this 'noise' is applied to the signals we process. And is that assumption correct? The signal data is modified like someone going through a sensitive document with a black marker pen.

Raise your flags! Just kidding.

***

Tangentially-
In Message 1329935 I ranted about some configuration nonsense -
I have stopped emulation of 4 cores on my Opteron 185. I'm just letting them run native for a better, more adaptable system. Well, for my set-up it is.
Tally Ho!
____________

ID: 1330148 ·

Josef W. Segur

Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 3968
Credit: 937,375
RAC: 148

I'm missing something - precisely where in the pipe from Arecibo to my front room, this 'noise' is applied to the signals we process. And is that assumption correct? The signal data is modified like someone going through a sensitive document with a black marker pen.
...

Two places (for Astropulse).

The first method of replacing data affected by RFI is done at Berkeley. When they get a 2 TB disk full of data from Arecibo it is broken up into ~50.2 GB files. Before those files are made available for splitting there's another process which reads the data and checks whether multiple channels have patterns matching those produced by the RADARs in Puerto Rico. A 15th channel gets the signal indicating good or bad data. Then when either an mb_splitter or ap_splitter is reading in data to split, the bad sections are replaced with pseudo-random data which has been shaped to match the frequency distribution of the good data.

The second method is within the Astropulse applications. The RFI detection method is different, based on the level in the DC bin of an FFT (probably detecting partial overload of the receiver IMO). The replacement data is like that used server-side.

For each detection of RFI, it makes sense to replace some data both before and after the detection point. The server-side replacement range is less than that in the Astropulse application. Another obvious difference is that MB processing is narrow band, so the AP detection method wouldn't make sense.

There are obvious possibilities to do things more efficiently, and at least one is being worked on. Code is being developed to not split and distribute data if it's too bad. If more than 0.28% of volunteers would donate cash to the project, it might be possible to get more staff so development of such improvements would go faster.

Joe

ID: 1330278 ·

Guy

Send message
Joined: 17 May 99
Posts: 41
Credit: 1,796,972
RAC: 13,151

Thank you.
____________

ID: 1330461 ·