Questions and Answers :
Web site :
SETI server backing off downloads problem
Message board moderation
Author | Message |
---|---|
Andrzej Mackiewicz Send message Joined: 25 Apr 12 Posts: 29 Credit: 7,218,146 RAC: 0 |
I'm sure everyone has probably had the same problem at one point or another.. Boinc isn't doing any SETI work due to downloads stalling and the retry time being rather long. As a step to reduce the load on the servers and hopefully avoid a lot of transfers stalling and causing the rest of the downloads to pause, why don't we limit all Boinc clients to one download attempt at a time? I have limited my download speed to like 25kb in an attempt to maintain a slow but stable connection, but notice that when Boinc begins a second simultaneous download, one of the downloads can stall as a result. I think by immediately halving the number of connections the server is receiving and serving we can improve the stability of the server significantly. <3 |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
It depends on when you are trying to download. If it is soon after the end of the weekly outage, most downloads fail. If it is later in the week, most downloads succeed. BOINC WIKI |
Andrzej Mackiewicz Send message Joined: 25 Apr 12 Posts: 29 Credit: 7,218,146 RAC: 0 |
So that's a vote of no confidence in the idea of halving concurrent TCP attempts on the server then? XD |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
So what if you want to fill your cache for a 4 or 8 core CPU? What if you want to fill it extra for the 4 GPUs in your system? What if you do not have high-speed internet? What if you are still on pay-per-minute dial-up, just because you do not have any other choice in your part of the world? In all these cases you will want more than one download going. Plus it's only this project at this time that has the problems, e.g. my downloads for Einstein come flying in at 835,450 bytes/sec, so why do those need to be hampered with a one download at a time non-option? If you want to restrict your downloads, you can do so with the client configuration file (cc_config.xml): <cc_config> <options> <max_file_xfers>2</max_file_xfers> </options> </cc_config> The default of the above is 8, you can put it back to 2 or 1 if you want to. Me? I'll let BOINC figure it out. If it can't get work here, it'll retry and retry until it can. In the mean time, the downloads from Einstein will still come in. :) |
Andrzej Mackiewicz Send message Joined: 25 Apr 12 Posts: 29 Credit: 7,218,146 RAC: 0 |
Those situations are irrelevant though mate, as if the server is overloaded you can have 20 connections for download and won't necessarily get anything to fill your cache. One steady and relatively fast download at a time is better than none at all until the retry timer expires. I don't think my client initiates quite enough requests that changing my own max file xfers will help the server load problem, but if everyone did it... I believe it would. Thanks for mentioning the config option- at least it's ultra easy to implement a global change if it's deemed to be beneficial to everyone that uses SETI. |
Andrzej Mackiewicz Send message Joined: 25 Apr 12 Posts: 29 Credit: 7,218,146 RAC: 0 |
ah actually SETI-specific limitation is needed obviously not BOINC global >.< |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
Those situations are irrelevant though mate, as if the server is overloaded you can have 20 connections for download and won't necessarily get anything to fill your cache. Yes. If every computer is asking for a connection at the same time, the server has to spend all its time telling almost everyone to go away and try again later. Better is to not have them all hit the server at the same time in the first place. BOINC WIKI |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
ah actually SETI-specific limitation is needed obviously not BOINC global >.< There is a server side setting for minimum backoff. This is set per project. If the project does not respond the client makes a decision on when to try next. If the connection goes through correctly, the client sets a very short backoff (one minute if I recall correctly). If things are going badly the backoffs approximately double until some cap is reached. It used to be 2 weeks, and has crept down slowly over the years. BOINC WIKI |
Andrzej Mackiewicz Send message Joined: 25 Apr 12 Posts: 29 Credit: 7,218,146 RAC: 0 |
I can appreciate that, but my point is that we can possibly avoid being told to back off by only serving one download per user at any given time and preferably having all clients only ask for one download at a time, to further reduce incoming waste. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
I can appreciate that, but my point is that we can possibly avoid being told to back off by only serving one download per user at any given time and preferably having all clients only ask for one download at a time, to further reduce incoming waste. In bad times, that is still over a hundred thousand computers all hitting the servers at once. BOINC WIKI |
Andrzej Mackiewicz Send message Joined: 25 Apr 12 Posts: 29 Credit: 7,218,146 RAC: 0 |
Again, if all those computers are asking for 2+ connections each simultaneously, that is double, triple or god knows how many times more connections than actually need to be served for SETI users to get their WU's and results in less of those 100,000+ users getting a usable connection.. |
Bernie Vine Send message Joined: 26 May 99 Posts: 9954 Credit: 103,452,613 RAC: 328 |
Perhaps Andrzej simply means why does Boinc (or SETI@Home) try and download 8 WU's at at time and therefore have 8 waiting and backing off. Why not just one WU, wait till that has downloaded then move on, if say 10,000 PC's are asking for work then 80,000 WU's are trying to upload. I am sure there is a reason for this it just seems odd. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
Perhaps Andrzej simply means why does Boinc (or SETI@Home) try and download 8 WU's at at time and therefore have 8 waiting and backing off. One at a time helps when the server is overloaded. A few at a time optimizes band width when it isn't. BOINC WIKI |
Bernie Vine Send message Joined: 26 May 99 Posts: 9954 Credit: 103,452,613 RAC: 328 |
One at a time helps when the server is overloaded. A few at a time optimizes band width when it isn't. Totally agree, however isn't the server overloaded most of the time and isn't that the point? After being on holiday I have just restarted 2 of my crunchers, one is currently trying to download 22 Wu's all at different % of progress. Why doesn't ONE complete BEFORE moving on to the next 21. There are another 11 that have not started, why, what is the magic formula that says how many can be downloaded at once. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
One at a time helps when the server is overloaded. A few at a time optimizes band width when it isn't. BOINC != SETI@Home. Most of the projects do not have this problem. BOINC WIKI |
Bernie Vine Send message Joined: 26 May 99 Posts: 9954 Credit: 103,452,613 RAC: 328 |
One at a time helps when the server is overloaded. A few at a time optimizes band width when it isn't. So are you saying that it is Boinc that works that way and ignores the project that made Boinc possible? Is there no way to make Boinc download one task at a time, then move onto the next? Not sure what the B in Boinc stands for then Seems shortsighted. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
One at a time helps when the server is overloaded. A few at a time optimizes band width when it isn't. David Anderson has indicated an unwillingness to put project specific code into the client. If there is a general fix it might make the cut. BOINC WIKI |
Andrzej Mackiewicz Send message Joined: 25 Apr 12 Posts: 29 Credit: 7,218,146 RAC: 0 |
Thanks for the insight! I can understand the point of view that BOINC does nothing project-specific, so asking for it to limit SETI downloads to one at a time is unreasonable. It would be a major change to BOINC, made especially for SETI (or any other projects with the same backing off problems though I'm not involved with / haven't seen any).. hmmm.. Maybe we could convince him to make the max_file_xfers config setting apply per-server and not globally? or is it already...? |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
Thanks for the insight! I can understand the point of view that BOINC does nothing project-specific, so asking for it to limit SETI downloads to one at a time is unreasonable. It would be a major change to BOINC, made especially for SETI (or any other projects with the same backing off problems though I'm not involved with / haven't seen any).. hmmm.. The setting is global. BOINC WIKI |
BilBg Send message Joined: 27 May 07 Posts: 3720 Credit: 9,385,827 RAC: 0 |
Maybe we could convince him to make the max_file_xfers config setting apply per-server and not globally? or is it already...? Mine is set to: <max_file_xfers>20</max_file_xfers> <max_file_xfers_per_project>10</max_file_xfers_per_project> ... so not everybody wants "one at a time", most people will be angry if some new BOINC version decides to limit them to "one at a time" Â - ALF - "Find out what you don't do well ..... then don't do it!" :) Â |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.