Slow WU download speed


log in

Advanced search

Message boards : Number crunching : Slow WU download speed

1 · 2 · Next
Author Message
Profile Haakon
Volunteer tester
Send message
Joined: 27 Jul 01
Posts: 4
Credit: 219,757
RAC: 0
Norway
Message 1325235 - Posted: 6 Jan 2013, 12:44:28 UTC

First off sorry if this is posted in the wrong section.


It seems the last two days I've gotten very slow download speeds from the server, to the point where I'm actually crunching faster than it takes to download.

These are the 8MB WUs from AstroPulse (if that makes a difference, other than size). Log says:

06/01/2013 13:36:12 | SETI@home | Temporarily failed download of ap_20oc12ac_B3_P0_00013_20130104_14697.wu: transient HTTP error
06/01/2013 13:36:12 | SETI@home | Backing off 5 hr 15 min 41 sec on download of ap_20oc12ac_B3_P0_00013_20130104_14697.wu


06/01/2013 13:40:54 | SETI@home | Started download of ap_20oc12ac_B4_P0_00072_20130104_03791.wu
06/01/2013 13:40:56 | | Project communication failed: attempting access to reference site
06/01/2013 13:40:58 | | Internet access OK - project servers may be temporarily down.



No proxy, and internet connection otherwise is okay.

Anyone else experiencing this also?
____________

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4067
Credit: 32,893,270
RAC: 7,328
United Kingdom
Message 1325241 - Posted: 6 Jan 2013, 13:01:14 UTC - in response to Message 1325235.
Last modified: 6 Jan 2013, 13:02:27 UTC

Just highlight the download, and press 'Retry Now' or wait for the timer to count down, that's quite normal around here when the Hurricane link is maxed out:

Graphs for gigabitethernet2_3

Claggy

Profile Haakon
Volunteer tester
Send message
Joined: 27 Jul 01
Posts: 4
Credit: 219,757
RAC: 0
Norway
Message 1325251 - Posted: 6 Jan 2013, 13:14:50 UTC

Thanks. It retires, while initial speed is 12KB/sec it just drops to a nothing after 10-15 seconds. I think average run time for WU dl is about 60 minutes.

Took a screen of it here https://dl.dropbox.com/u/11600062/boinc_transfer.jpg
____________

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4067
Credit: 32,893,270
RAC: 7,328
United Kingdom
Message 1325268 - Posted: 6 Jan 2013, 13:45:04 UTC - in response to Message 1325251.
Last modified: 6 Jan 2013, 13:47:57 UTC

Thanks. It retires, while initial speed is 12KB/sec it just drops to a nothing after 10-15 seconds. I think average run time for WU dl is about 60 minutes.

Took a screen of it here https://dl.dropbox.com/u/11600062/boinc_transfer.jpg

Setiathome has two download servers, DNS switches between them every five minutes, generally one server has a higher load (and slower downloads) on it than the other,

If you go to the Activity Menu, and Suspend Network Activity for a moment, then resume it,

1) Boinc will resume the (mostly complete) downloads at the top of the que, 2) If DNS has switched your downloads will start on the other server, hopefully the faster one,

Claggy

Profile ivan
Volunteer tester
Avatar
Send message
Joined: 5 Mar 01
Posts: 602
Credit: 135,457,199
RAC: 130,897
United Kingdom
Message 1325300 - Posted: 6 Jan 2013, 14:42:07 UTC - in response to Message 1325268.

Took a screen of it here https://dl.dropbox.com/u/11600062/boinc_transfer.jpg

If you go to the Activity Menu, and Suspend Network Activity for a moment, then resume it,

1) Boinc will resume the (mostly complete) downloads at the top of the que, 2) If DNS has switched your downloads will start on the other server, hopefully the faster one,

You should also check out <max_file_xfers_per_project> in Client Configuration. You're still using the default 2.
____________

Profile Haakon
Volunteer tester
Send message
Joined: 27 Jul 01
Posts: 4
Credit: 219,757
RAC: 0
Norway
Message 1325319 - Posted: 6 Jan 2013, 15:27:03 UTC - in response to Message 1325300.


You should also check out <max_file_xfers_per_project> in Client Configuration. You're still using the default 2.


Thanks! Made myself a config file now.. didnt know that was possible.. hopefully this will help.. I guess I should do a readup of this config file.. seems alot of usefull settings.

Just returned to boinc and seti (still want the old client back tho).. if any other tips that would be appreciated :-)
____________

ClaggyProject donor
Volunteer tester
Send message
Joined: 5 Jul 99
Posts: 4067
Credit: 32,893,270
RAC: 7,328
United Kingdom
Message 1325325 - Posted: 6 Jan 2013, 16:04:21 UTC - in response to Message 1325319.
Last modified: 6 Jan 2013, 16:05:27 UTC


You should also check out <max_file_xfers_per_project> in Client Configuration. You're still using the default 2.


Thanks! Made myself a config file now.. didnt know that was possible.. hopefully this will help.. I guess I should do a readup of this config file.. seems alot of usefull settings.

Just returned to boinc and seti (still want the old client back tho).. if any other tips that would be appreciated :-)

Another trick is to run SIV, it has a BOINC Status page where you can configure it to retry downloads (But you're got to keep that page open, so i'm running two instances)

Claggy

Profile Haakon
Volunteer tester
Send message
Joined: 27 Jul 01
Posts: 4
Credit: 219,757
RAC: 0
Norway
Message 1325333 - Posted: 6 Jan 2013, 16:44:30 UTC - in response to Message 1325325.


Another trick is to run SIV, it has a BOINC Status page where you can configure it to retry downloads (But you're got to keep that page open, so i'm running two instances)

Claggy


Thanks! I'll try that one out along with allowing more than one transfer.. hopefully I'll be able to get down WUs fast enough without any downtime.. Thanks again! :-)

____________

TBar
Volunteer tester
Send message
Joined: 22 May 99
Posts: 1218
Credit: 45,460,271
RAC: 117,538
United States
Message 1325353 - Posted: 6 Jan 2013, 18:44:39 UTC - in response to Message 1325325.


You should also check out <max_file_xfers_per_project> in Client Configuration. You're still using the default 2.


Thanks! Made myself a config file now.. didnt know that was possible.. hopefully this will help.. I guess I should do a readup of this config file.. seems alot of usefull settings.

Just returned to boinc and seti (still want the old client back tho).. if any other tips that would be appreciated :-)

Another trick is to run SIV, it has a BOINC Status page where you can configure it to retry downloads (But you're got to keep that page open, so i'm running two instances)

Claggy

...you can configure it to retry downloads...
My hero....

I currently have 10 APs on the transfer page. The Retry button is moaning.

Profile ivan
Volunteer tester
Avatar
Send message
Joined: 5 Mar 01
Posts: 602
Credit: 135,457,199
RAC: 130,897
United Kingdom
Message 1325355 - Posted: 6 Jan 2013, 18:49:09 UTC - in response to Message 1325353.
Last modified: 6 Jan 2013, 18:51:53 UTC

...you can configure it to retry downloads...
My hero....

I currently have 10 APs on the transfer page. The Retry button is moaning.

I use cron entries with bash scripts calling awk scripts in Cygwin [or Windows Task Scheduler if I don't have cygwin installed] or Linux to accomplish the same thing (and tickle reporting if uploads are ready). I've posted them before but I can do it again if anyone wants.
____________

Cosmic_Ocean
Avatar
Send message
Joined: 23 Dec 00
Posts: 2245
Credit: 8,595,344
RAC: 4,356
United States
Message 1325461 - Posted: 7 Jan 2013, 7:10:15 UTC

Yeah, I've never had these issues with downloading before. I've been reading about them for years now, but I have never personally experienced it. It really is painful. I waited about 90 minutes (elapsed download time was 54 minutes) for this one AP and discovered that after two downloads timed out 2-3 times, I was in project backoff of like 4 hours. Oh, and then it was a 100% blanked B3_P1, so less than a second of CPU time and it was done.

I've been nudging my downloads along for an hour or so now. Making progress. Slowly but surely. Tried switching servers, too, and it doesn't seem to make any difference. They both time-out and run slowly about the same.
____________

Linux laptop uptime: 1484d 22h 42m
Ended due to UPS failure, found 14 hours after the fact

Profile BilBg
Volunteer tester
Avatar
Send message
Joined: 27 May 07
Posts: 2642
Credit: 6,000,259
RAC: 4,280
Bulgaria
Message 1325468 - Posted: 7 Jan 2013, 8:08:15 UTC - in response to Message 1325333.


Another trick is to run SIV, it has a BOINC Status page where you can configure it to retry downloads (But you're got to keep that page open, so i'm running two instances)

Claggy


Thanks! I'll try that one out along with allowing more than one transfer.. hopefully I'll be able to get down WUs fast enough without any downtime.. Thanks again! :-)

Another program which can retry downloads/uploads is BoincTasks:
http://www.efmer.eu/boinc/boinc_tasks/

Put config.xml file in the install directory (C:\Program Files\eFMer\BoincTasks\)
(you can copy it from 'examples' directory)

I have the following lines in it:
<config> <refresh> <downloads> 611 </downloads> <uploads> 613 </uploads> <refresh> 36777 </refresh> </refresh> </config>


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Morten Ross
Volunteer tester
Avatar
Send message
Joined: 30 Apr 01
Posts: 183
Credit: 378,289,433
RAC: 4
Norway
Message 1325532 - Posted: 7 Jan 2013, 15:53:07 UTC - in response to Message 1325468.

I'm wondering if this is a Scandinavian problem, as I have the same issue. After a disable/enable network, the download seems speedy for a few seconds, but then quickly grinds to a halt.

Looking at the top 10 hosts, they are all getting what they need according to the increased RAC, except my hosts which are steadily crunching 0-10% of capacity due to failing downloads.

Uploads are superspeedy.

I don't have any proxy to test, so don't know where the bottleneck could be....
____________
Morten Ross

Rolf
Send message
Joined: 16 Jun 09
Posts: 114
Credit: 7,816,984
RAC: 151
Switzerland
Message 1325575 - Posted: 7 Jan 2013, 19:45:05 UTC - in response to Message 1325532.

I'm wondering if this is a Scandinavian problem, as I have the same issue. After a disable/enable network, the download seems speedy for a few seconds, but then quickly grinds to a halt.

No, it's not a Scandinavian problem! It appears also in Switzerland, so you could call the problem a European problem!

Profile Bill GProject donor
Avatar
Send message
Joined: 1 Jun 01
Posts: 347
Credit: 39,763,506
RAC: 68,375
United States
Message 1325594 - Posted: 7 Jan 2013, 20:57:14 UTC - in response to Message 1325575.

I'm wondering if this is a Scandinavian problem, as I have the same issue. After a disable/enable network, the download seems speedy for a few seconds, but then quickly grinds to a halt.

No, it's not a Scandinavian problem! It appears also in Switzerland, so you could call the problem a European problem!


It just took 54 minutes to download an AP....I do not think it is just Europe.
____________

rob smithProject donor
Volunteer tester
Send message
Joined: 7 Mar 03
Posts: 8302
Credit: 55,213,551
RAC: 75,314
United Kingdom
Message 1325622 - Posted: 7 Jan 2013, 22:21:40 UTC
Last modified: 7 Jan 2013, 22:21:52 UTC

ONLY 54 minute - I haven't had an AP go through that fast for a few days, they are typically taking 90-150 minutes (excluding the time wasting, band width saturating, server clogging so called "retries" and "back-offs" - all these accomplish is to waste resources and frustrate users - and more than double the REAL download times, to more like 180-300 minutes).
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Chris
Send message
Joined: 11 Apr 12
Posts: 9
Credit: 354,608
RAC: 0
United States
Message 1325931 - Posted: 9 Jan 2013, 3:06:35 UTC

That wasn't much fun. Boinc thought I had enough work prior to the Tues blackout, except most the AP were hanging and the pile of 6.10s took 5 minutes each. What looked like a > 24 hour cache burned up really quick!

Still downloading slow, but it works out better now that the project is up for retrying.

Aero
Send message
Joined: 4 Feb 04
Posts: 29
Credit: 1,457,623
RAC: 360
Message 1326130 - Posted: 9 Jan 2013, 18:29:57 UTC
Last modified: 9 Jan 2013, 18:46:28 UTC

I've noticed the prolonged downloads too, but noticed today that it's improved significantly. It's been a few months since I payed much attention to what the clients are doing, so not sure how unusual the slower DL's on my end has been.
____________

Profile Fred J. Verster
Volunteer tester
Send message
Joined: 21 Apr 04
Posts: 3238
Credit: 31,690,770
RAC: 6,190
Netherlands
Message 1326218 - Posted: 9 Jan 2013, 22:14:29 UTC - in response to Message 1326130.
Last modified: 9 Jan 2013, 22:16:03 UTC

I've noticed the prolonged downloads too, but noticed today that it's improved significantly. It's been a few months since I payed much attention to what the clients are doing, so not sure how unusual the slower DL's on my end has been.


Unfortunatly true, crunching is done faster then down- and uploading them .
____________

Profile NottiusMaximus
Send message
Joined: 31 Oct 99
Posts: 1
Credit: 398,931
RAC: 442
Canada
Message 1334734 - Posted: 4 Feb 2013, 22:41:42 UTC - in response to Message 1325235.

I am also getting VERY VERY slow download speed for wu's and anything else SETI sends. I am beginning to think SETI uses a 56K dialup for sending data since I have never received a wu faster than 5kps. Well back to watching a 5M wu come down. It's at 75% and it's only been 30 mins. :(
____________

1 · 2 · Next

Message boards : Number crunching : Slow WU download speed

Copyright © 2014 University of California