Error while downloading - ideas??

Message boards : Number crunching : Error while downloading - ideas??
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1797424 - Posted: 19 Jun 2016, 20:56:31 UTC

Anybody else getting errors while downloading work? I have had a couple on each machine in the last two weeks or so. Never had them before. The error tasks are happening while the machines are doing normal crunching and it is not happening during startup or shutdown.

I was running some of the beta client versions for a while during this period and thought they might be the cause of the errors. But, I have returned to the stock 7.6.22 client and am still getting these errors.

I have checked disk'ed and sfc /scannow'd both machines with no errors and I don't see any other irregularities in either machine.

Anyone else seeing these errors? Any thought on why they are occurring?

Task 4982633795
Task 4986282509
Task 4989238656
Task 4992616352
Task 4994556308
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1797424 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1797432 - Posted: 19 Jun 2016, 21:51:15 UTC - in response to Message 1797424.  
Last modified: 19 Jun 2016, 21:51:49 UTC

Have you tried re-booting your modem?
Do you have a second modem about you could try for a while?
Grant
Darwin NT
ID: 1797432 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1797440 - Posted: 19 Jun 2016, 22:29:33 UTC - in response to Message 1797432.  

Think that might be grasping ... but it couldn't hurt likely. Looked at the gateway and see that it was last rebooted 13 days ago. So that would be before the period the errored tasks started arriving. Don't see any comms errors in its logs though. No uncorrected errors at all over several hundreds of gigabytes. Don't have another gateway to try though. Will give it a shot. Thanks for the idea, Grant.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1797440 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1797451 - Posted: 19 Jun 2016, 23:06:53 UTC - in response to Message 1797440.  
Last modified: 19 Jun 2016, 23:10:29 UTC

I have had a couple on each machine in the last two weeks or so.

The fact that it's been on each machine, not just one, indicates it's something common to all of them.
When it come to uploads & downloads i'd expect BOINC just makes use of the Windows services; no need to reinvent the wheel, so the manager itself is very unlikely to be causing the download errors (some truly bizarre weirdness might result in one of the managers causing issues, but multiple machines? And if it were an issue within the manager i'd expect there to be lots of people reporting problems).
So the most likely candidate is the modem.
Grant
Darwin NT
ID: 1797451 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1797483 - Posted: 20 Jun 2016, 2:59:33 UTC - in response to Message 1797451.  

Well I rebooted the gateway and router, so time will tell. Actually I wonder if it might have something to do with recent Windows updates. There is some suggestion in another thread related to Windows updates causing weird error codes. That is the only thing that has changed recently on both machines within the two week window of the download errors appearing. Nothing else really has changed in the computing environments of both machines for quite a while.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1797483 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799043 - Posted: 27 Jun 2016, 20:34:13 UTC

My rebooting of both the router and gateway did not change the download errors in both of my crunchers. So I am out of ideas of what could be causing these random downloads to fail. Anyone have any further ideas of what is going on and any remedies I have not tried yet?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799043 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11360
Credit: 29,581,041
RAC: 66
United States
Message 1799049 - Posted: 27 Jun 2016, 20:56:39 UTC

Two thoughts.
Do you get these errors on Einstein?
Are you using image file verification on your Boinc preferences in network usage?
ID: 1799049 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1799056 - Posted: 27 Jun 2016, 21:11:57 UTC - in response to Message 1799049.  

Are you using image file verification on your Boinc preferences in network usage?

Down the bottom of the Computing preferences page, under Network.
Skip data verification for image files shouldn't be selected, but apparently some ISPs do odd things & it needs to be selected to sort out the issues resulting from that.

If that doesn't sort it, then you could run SFC (System File Checker) on both systems to check for corrupted OS files, however my suspicions are the modem itself is the most likely culprit.
Grant
Darwin NT
ID: 1799056 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1799070 - Posted: 27 Jun 2016, 21:44:42 UTC - in response to Message 1799043.  

My rebooting of both the router and gateway did not change the download errors in both of my crunchers. So I am out of ideas of what could be causing these random downloads to fail. Anyone have any further ideas of what is going on and any remedies I have not tried yet?

I looked back over the various download errors and their associated error codes that I've gotten over the past 3+ years and found only one group that ended up with the "<error_code>-200 (wrong size)</error_code>" that you appear to be getting. That happened a couple years ago when an AT&T technician was checking my DSL line and managed to disconnect it right smack in the middle of a download sequence. I ended up with 11 tasks with that "-200" error. There could have been only 2 of them actually downloading at the time of the interruption, but apparently all the files waiting to be downloaded got whacked too.

So, I would say that a temporary interruption in your Internet connection could be at the root of the problem. Whether that might be your modem, router, or something further up the line, is anybody's guess. Does your Event Log show anything for those files that might supplement what's shown in the task Stderr?
ID: 1799070 · Report as offensive
Profile petri33
Volunteer tester

Send message
Joined: 6 Jun 02
Posts: 1668
Credit: 623,086,772
RAC: 156
Finland
Message 1799080 - Posted: 27 Jun 2016, 22:14:01 UTC - in response to Message 1799043.  

... and

there is always the old virus scanner effect. A scanner could think your wu's are somehow dangerous. (they are'nt)
To overcome Heisenbergs:
"You can't always get what you want / but if you try sometimes you just might find / you get what you need." -- Rolling Stones
ID: 1799080 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799096 - Posted: 27 Jun 2016, 23:04:28 UTC - in response to Message 1799080.  

Except that I have excluded the BOINC directory and the /Program Data directory and sub-directories from any scanning or monitoring by my AV app. Also have put into Exclusive Application status all BOINC apps. I only get download errors from Seti@Home, NOT ever from Einstein or MilkyWay. My AV program shows no action against SETI in its logs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799096 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799097 - Posted: 27 Jun 2016, 23:08:52 UTC - in response to Message 1799049.  

Two thoughts.
Do you get these errors on Einstein?
Are you using image file verification on your Boinc preferences in network usage?

No. Never any download errors on either MilkyWay or Einstein which run on the crunchers along with SETI.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799097 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799098 - Posted: 27 Jun 2016, 23:15:07 UTC - in response to Message 1799056.  

Are you using image file verification on your Boinc preferences in network usage?

Down the bottom of the Computing preferences page, under Network.
Skip data verification for image files shouldn't be selected, but apparently some ISPs do odd things & it needs to be selected to sort out the issues resulting from that.

If that doesn't sort it, then you could run SFC (System File Checker) on both systems to check for corrupted OS files, however my suspicions are the modem itself is the most likely culprit.

No, I do not have [Skip Data verification] checked. Never had it checked since I started running BOINC. Have already check-disked both crunchers and have also run SFC /scannow against both computers with no errors found. If the gateway is the common component with both computers, why no download errors with either Einstein or Milky Way?
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799098 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13722
Credit: 208,696,464
RAC: 304
Australia
Message 1799101 - Posted: 27 Jun 2016, 23:28:53 UTC - in response to Message 1799098.  
Last modified: 27 Jun 2016, 23:30:09 UTC

Skip data verification for image files shouldn't be selected, but apparently some ISPs do odd things & it needs to be selected to sort out the issues resulting from that.

No, I do not have [Skip Data verification] checked. Never had it checked since I started running BOINC.

Might be worth giving it a go.
If the issues still persist, then deselect it again.

If the gateway is the common component with both computers, why no download errors with either Einstein or Milky Way?

No idea.
Does Seti have smaller files, and shorter crunching times? If there is an intermittent issue then the project that has the most frequent downloads is the most likely to be affected.
Worth checking your modem's logs as Jeff Buck suggested to see if there are occasional modem dropouts, or lots of CRC errors or retries.

In the past I had a modem that required re-booting every few weeks or so; only because there were a couple of web sites that would take ages to load. Absolute ages; one was in the US & one in Europe. They may or may not have been on the same server (I never checked with traceroute), but without fail when they started behaving badly (and no other sites) re-booting would resolve it.
Replacing the modem sorted it completely.
Grant
Darwin NT
ID: 1799101 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799102 - Posted: 27 Jun 2016, 23:30:28 UTC - in response to Message 1799070.  

My rebooting of both the router and gateway did not change the download errors in both of my crunchers. So I am out of ideas of what could be causing these random downloads to fail. Anyone have any further ideas of what is going on and any remedies I have not tried yet?

I looked back over the various download errors and their associated error codes that I've gotten over the past 3+ years and found only one group that ended up with the "<error_code>-200 (wrong size)</error_code>" that you appear to be getting. That happened a couple years ago when an AT&T technician was checking my DSL line and managed to disconnect it right smack in the middle of a download sequence. I ended up with 11 tasks with that "-200" error. There could have been only 2 of them actually downloading at the time of the interruption, but apparently all the files waiting to be downloaded got whacked too.

So, I would say that a temporary interruption in your Internet connection could be at the root of the problem. Whether that might be your modem, router, or something further up the line, is anybody's guess. Does your Event Log show anything for those files that might supplement what's shown in the task Stderr?
Nothing happening on the DSL technician front as far as I know. No trucks on the street when these errors happen. I have never found anything in the gateway logs coinciding with the times of the errors either. Also, since I run MilkyWay at the same time as Seti, the download activity is darn near constant since Milkyway downloads every minute or so and often very near the times that Seti tried downloading a work unit and failed. Again, no download errors for either Einstein nor MilkyWay. I did see the last download error report in the logfile as it happened for the single CPU task BOINC asked for. Slightly different entries than what got reported in stderr.txt for that task.

27-Jun-2016 13:28:45 [SETI@home] Requesting new tasks for CPU and NVIDIA GPU
27-Jun-2016 13:28:49 [SETI@home] Scheduler request completed: got 1 new tasks
27-Jun-2016 13:28:51 [SETI@home] Started download of 30no10ab.31740.5903.12.39.204
27-Jun-2016 13:28:53 [SETI@home] Incomplete read of 43.000000 < 5KB for 30no10ab.31740.5903.12.39.204 - truncating
27-Jun-2016 13:28:53 [SETI@home] Finished download of 30no10ab.31740.5903.12.39.204
27-Jun-2016 13:28:53 [SETI@home] File 30no10ab.31740.5903.12.39.204 has wrong size: expected 365927, got 0
27-Jun-2016 13:28:53 [SETI@home] Checksum or signature error for 30no10ab.31740.5903.12.39.204

Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799102 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799104 - Posted: 27 Jun 2016, 23:40:50 UTC - in response to Message 1799101.  
Last modified: 27 Jun 2016, 23:54:56 UTC

Skip data verification for image files shouldn't be selected, but apparently some ISPs do odd things & it needs to be selected to sort out the issues resulting from that.

No, I do not have [Skip Data verification] checked. Never had it checked since I started running BOINC.

Might be worth giving it a go.
If the issues still persist, then deselect it again.

If the gateway is the common component with both computers, why no download errors with either Einstein or Milky Way?

No idea.
Does Seti have smaller files, and shorter crunching times? If there is an intermittent issue then the project that has the most frequent downloads is the most likely to be affected.
Worth checking your modem's logs as Jeff Buck suggested to see if there are occasional modem dropouts, or lots of CRC errors or retries.

In the past I had a modem that required re-booting every few weeks or so; only because there were a couple of web sites that would take ages to load. Absolute ages; one was in the US & one in Europe. They may or may not have been on the same server (I never checked with traceroute), but without fail when they started behaving badly (and no other sites) re-booting would resolve it.
Replacing the modem sorted it completely.

I'll have a go at the [Skip data verification] toggle. I can't think of anything else to try other than ask for another gateway from my provider. Milkyway has the smallest work units and the fastest completion times. MW work only takes 1 minute to complete and report. Seti ~ 10 - 30 minutes. Einstein ~ 1-3 hours.
Gateway CRC errors by Keith Myers, on Flickr
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799104 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1799118 - Posted: 28 Jun 2016, 0:28:56 UTC - in response to Message 1799102.  

My rebooting of both the router and gateway did not change the download errors in both of my crunchers. So I am out of ideas of what could be causing these random downloads to fail. Anyone have any further ideas of what is going on and any remedies I have not tried yet?

I looked back over the various download errors and their associated error codes that I've gotten over the past 3+ years and found only one group that ended up with the "<error_code>-200 (wrong size)</error_code>" that you appear to be getting. That happened a couple years ago when an AT&T technician was checking my DSL line and managed to disconnect it right smack in the middle of a download sequence. I ended up with 11 tasks with that "-200" error. There could have been only 2 of them actually downloading at the time of the interruption, but apparently all the files waiting to be downloaded got whacked too.

So, I would say that a temporary interruption in your Internet connection could be at the root of the problem. Whether that might be your modem, router, or something further up the line, is anybody's guess. Does your Event Log show anything for those files that might supplement what's shown in the task Stderr?
Nothing happening on the DSL technician front as far as I know. No trucks on the street when these errors happen. I have never found anything in the gateway logs coinciding with the times of the errors either. Also, since I run MilkyWay at the same time as Seti, the download activity is darn near constant since Milkyway downloads every minute or so and often very near the times that Seti tried downloading a work unit and failed. Again, no download errors for either Einstein nor MilkyWay. I did see the last download error report in the logfile as it happened for the single CPU task BOINC asked for. Slightly different entries than what got reported in stderr.txt for that task.

27-Jun-2016 13:28:45 [SETI@home] Requesting new tasks for CPU and NVIDIA GPU
27-Jun-2016 13:28:49 [SETI@home] Scheduler request completed: got 1 new tasks
27-Jun-2016 13:28:51 [SETI@home] Started download of 30no10ab.31740.5903.12.39.204
27-Jun-2016 13:28:53 [SETI@home] Incomplete read of 43.000000 < 5KB for 30no10ab.31740.5903.12.39.204 - truncating
27-Jun-2016 13:28:53 [SETI@home] Finished download of 30no10ab.31740.5903.12.39.204
27-Jun-2016 13:28:53 [SETI@home] File 30no10ab.31740.5903.12.39.204 has wrong size: expected 365927, got 0
27-Jun-2016 13:28:53 [SETI@home] Checksum or signature error for 30no10ab.31740.5903.12.39.204

Can you try going back into your stdoutdae.txt and even the stdoutdae.old files and see if that "has wrong size: expected" response has ever shown up on any files where the download was immediately started again and then succeeded without getting the error? I was just looking at a couple of old forum threads where that situation also occurred. For instance, see "[error] File ... has wrong size: expected ... got ...". Again, the suggestion there was that "It is displayed when the connection is interrupted during download." If it is a temporary interruption, though, I have no idea why only S@H would be affected and not MilkyWay or Einstein, unless they're more fault tolerant somehow and always initiate a retry of any interrupted downloads.
ID: 1799118 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799154 - Posted: 28 Jun 2016, 4:46:29 UTC - in response to Message 1799118.  
Last modified: 28 Jun 2016, 4:50:31 UTC


Can you try going back into your stdoutdae.txt and even the stdoutdae.old files and see if that "has wrong size: expected" response has ever shown up on any files where the download was immediately started again and then succeeded without getting the error? I was just looking at a couple of old forum threads where that situation also occurred. For instance, see "[error] File ... has wrong size: expected ... got ...". Again, the suggestion there was that "It is displayed when the connection is interrupted during download." If it is a temporary interruption, though, I have no idea why only S@H would be affected and not MilkyWay or Einstein, unless they're more fault tolerant somehow and always initiate a retry of any interrupted downloads.

It looks like it never asks for that same errorred task again. The very next request for work after 305 seconds is for just 1 CPU task again ... and it succeeded normally.

24-Jun-2016 13:53:21 [SETI@home] Started download of 16se10ae.20221.19699.8.35.105
24-Jun-2016 13:53:21 [SETI@home] Started download of 18se10ab.23675.18881.6.33.201
24-Jun-2016 13:53:23 [SETI@home] Incomplete read of 43.000000 < 5KB for 18se10ab.23675.18881.6.33.201 - truncating
24-Jun-2016 13:53:23 [SETI@home] Finished download of 16se10ae.20221.19699.8.35.105
24-Jun-2016 13:53:23 [SETI@home] Finished download of 18se10ab.23675.18881.6.33.201
24-Jun-2016 13:53:23 [SETI@home] File 18se10ab.23675.18881.6.33.201 has wrong size: expected 365930, got 0
24-Jun-2016 13:53:23 [SETI@home] Checksum or signature error for 18se10ab.23675.18881.6.33.201
24-Jun-2016 13:56:48 [SETI@home] Message from task: 0


This is the entry on the other computer:

24-Jun-2016 00:16:35 [SETI@home] Scheduler request completed: got 3 new tasks
24-Jun-2016 00:16:37 [SETI@home] Started download of 13jn10aa.3165.11519.15.42.162
24-Jun-2016 00:16:37 [SETI@home] Started download of 13jn10aa.3204.18877.16.43.227
24-Jun-2016 00:16:37 [SETI@home] Started download of 16oc10aa.32000.12750.7.34.166
24-Jun-2016 00:16:38 [SETI@home] Incomplete read of 43.000000 < 5KB for 16oc10aa.32000.12750.7.34.166 - truncating
24-Jun-2016 00:16:38 [SETI@home] Finished download of 16oc10aa.32000.12750.7.34.166
24-Jun-2016 00:16:38 [SETI@home] File 16oc10aa.32000.12750.7.34.166 has wrong size: expected 365926, got 0
24-Jun-2016 00:16:38 [SETI@home] Checksum or signature error for 16oc10aa.32000.12750.7.34.166
24-Jun-2016 00:16:39 [SETI@home] Finished download of 13jn10aa.3165.11519.15.42.162
24-Jun-2016 00:16:39 [SETI@home] Finished download of 13jn10aa.3204.18877.16.43.227


It looks like once it fails to download the task, it reports it immediately and never asks for it to be downloaded again. So it doesn't match the scenario in the other thread. I've been getting these errors every 3 or 4 days apart and only one event on one work unit. I could be asking for a dozen or so per request but it only fails on one unit of the request and not the first or last request. Seems random which work unit of the request fails.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799154 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1799156 - Posted: 28 Jun 2016, 5:05:00 UTC - in response to Message 1799154.  

I notice that on your second example, BOINC is starting to download 3 tasks simultaneously. I believe the default for a single project is just 2 tasks (or files). Do you have a different value set in cc_config.xml for <max_file_xfers_per_project>N</max_file_xfers_per_project>? If so, perhaps your pipeline is getting clogged and packets are getting dropped. Could you try going back to the default and see if that makes a difference?
ID: 1799156 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 13161
Credit: 1,160,866,277
RAC: 1,873
United States
Message 1799171 - Posted: 28 Jun 2016, 6:17:05 UTC - in response to Message 1799156.  

I notice that on your second example, BOINC is starting to download 3 tasks simultaneously. I believe the default for a single project is just 2 tasks (or files). Do you have a different value set in cc_config.xml for <max_file_xfers_per_project>N</max_file_xfers_per_project>? If so, perhaps your pipeline is getting clogged and packets are getting dropped. Could you try going back to the default and see if that makes a difference?

You are correct. I had it set to:
<max_file_xfers>8</max_file_xfers>
<max_file_xfers_per_project>4</max_file_xfers_per_project>
I'll set it to the default of 2.

Of course I had to jinx things. As soon as I stated I had no download errors on my other projects .... I just got a download error on both computers for Einstein. Still no errors on MilkyWay though.
Seti@Home classic workunits:20,676 CPU time:74,226 hours

A proud member of the OFA (Old Farts Association)
ID: 1799171 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Error while downloading - ideas??


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.