SETI server backing off downloads problem

As I explained years ago to a user that wanted 30 WU's downloading at a time, YOu are taking the space of 15 standard users. If everyone did what you do the servers would be overwhelmed.

Since most of us aren't running out of work as fast as we get it, it doesn't make much sense to overwhelm to transfer more than 2 at a time which is the standard rate. Getting all your transfers done at the same time doesn't help finish work faster. all it does is help clog an already overwhelmed pipeline.

For those that insist on flooding the servers, so be it. Nobody says you can't. However, it's pretty inconsiderate to all the others that want more work as well.
____________
Proud member of TSWB.

End terrorism by building a school

Yes. What I'm hoping for is to allow SETI to set that max dl's setting in everyone's config by default, should that setting become per-project. That would at least temporarily relieve the strain on SETI servers until people change it for no good reason.

As you point out, if we could do this, nobody need change their setting above 1 as they can't finish calculating one work unit faster than they can download one work unit. Admittedly dialup users may not be able to download as fast as they can compute but then more connections certainly wouldn't even help those users..

Petition to change max downloads in BOINC to be per-project? ;)

lol :)

Guess not enough people are interested to put it forward to the BOINC coders

No one else wants to be the messenger, no. So why don't you sign up to the BOINC development email list and ask for this yourself? Do write the whole request in the email, there's nothing more annoying to the developers than doing a request and pointing to a (big) thread on some forums somewhere.

The BOINC Dev email list requires registration in order to be able to post to it.
____________
Jord

- BOINC FAQ Service
- BOINC User Wiki

Real is just a matter of perception.

No, I was more thinking of a large list of users backing the idea.. But I appreciate your telling me to do it myself ^_^

@BilBg

I just re-read this thread so I can draft an email to try and get this issue addressed and noticed two things:

1. You are asking SETI for up to 10 simultaneous downloads? Nice contributing to the problem with 5 times the potential simultaneous requests being sent to SETI servers for no benefit whatsoever mate.
2. The <max_file_xfers_per_project>1</max_file_xfers_per_project> you kindly pointed out (and I cleverly overlooked) is exactly what is needed- you will not have any reason to complain if you have it set to 1 download at a time..

Your CPU/GPU is very frigging unlikely to be able to crunch data faster than you can download the next work unit. Even if it has to wait for 1 minute while the next work unit is downloading, it still beats being told to back off for 1 hour+!

So yeah, my idea is already in practice on BOINC's end; we just need to tell/force people's max_file_xfers_per_project to 1. No need to email Dave ^_^

I think what they are inquiring about, at least I am, is that why is it taking many minutes, in some cases hours, to download a few Kbytes, or Mbytes of information? It took me about 40min to download a 51KB size file before BOINC started downloading other files for SETI@HOME. Hours to download what looks like the CUDA executable files at roughly 10KB total?

I understand this funded by limited grants and/or donations as it does cost money to host/maintain/upgrade the bandwidth, software and hardware; but 10KB/s? That is roughly 128Kbps or 2, maybe 3 times as fast as dial-up. These files could be downloaded in mere seconds and the idle processing power of my computer could already be crunching the numbers. =)

I would have posted a screenshot but it does not appear we can upload a picture to the message boards but to only link the image from another website, which I do not have one to host the image. =)

The line at SETI is 100 Mbps and is saturated almost all the time by ~ 200 000 computers asking for work:

http://fragment1.berkeley.edu/newcricket/grapher.cgi?target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;view=Octets


____________



- ALF - "Find out what you don't do well ..... then don't do it!" :)

Ahhhh. Yeah, That would do it. Wish I had the $ to donate a GB link or two. Of course that would fill up soon enough, I'm sure. =)

I remember the days when our ISP upgraded to its 4th T1 and we thought we would never have saturated the 3rd one. Then we got our first DS3... and then our 3rd... and so on. You never think you would exceed X bandwidth so why get Y. I was sad to see it sold off and parted but hey, such the life of a business. Someone waves enough $$ and the owner will sell!

We had our graph moved to internal network access after the first DS3 because you had those, like me =), that would email or call and complain that your 60% saturated! When are you going to increase your bandwidth? You are now 60.5% saturated... 61% saturated...!

Thanks for the link to the graph at SETI. It's always interesting to know how much bandwidth is being used for a project.

Thanks for those numbers BilBg. Certainly made me think..

Do you know if the 200,000 is SETI's active member count or an approximated count of simultaneous users based on observed statistics?

If it's the approximate number of active members, this single download request per project idea could still help; if it's an approximation of how many users are simultaneously requesting work given that we have more than that many users, I think my idea can't help sadly, as that calculates to half a byte per second per user even if we could maintain connections to them all simultaneously...

CRAAAAPPP XD

Active users (individual user IDs that got credit): http://boincstats.com/en/stats/0/project/detail/user, around 142,000
Active hosts (computers that got granted credit): http://boincstats.com/en/stats/0/project/detail/host, around 212,000
____________
Jord

- BOINC FAQ Service
- BOINC User Wiki

Real is just a matter of perception.

Thanks for that mate.

212000 active hosts / 24 hours = approx. 8834 computers making requests per hour

100 Megabit SETI connection / 8834 average required connections = 1.5 KB/s download speed theoretically available per user

Everyone asking for 2-4+ downloads at a time from SETI is only hurting their chances of finishing a transfer. 4 simultaneous downloads utilising 1.5KB/s potentially available bandwidth in total will take forever and can result in your being told to Back Off due to there being bugger-all bandwidth available for you.

If everyone only used one download connection to SETI at any given time, the server should be able to cope with giving them their tiny share of bandwidth and will finish sending the work unit in 6 mins or so even at 1.5KB/s.

Those downloading 2-4+ WU's at a time from SETI are essentially cutting their own throats as 1.5KB/s is not hardly enough to support 4 simultaneous downloads.

Sound?

I understand and agree; the total bandwidth available doesn't change. By the same token, if I am downloading 4 files with a potential available bandwidth of 1.5KB/s, it takes me WAY longer to get a usable work unit. That being the problem; we don't always have work ready to do.

There is no sense in anybody trying to download more than one file at a time when bandwidth is so scarce. Just like you say, the bandwidth available to you is the same for 1 or 4 transfers- though you want a complete work unit ASAP, not 4 partials.

Also it takes more CPU and a little more bandwidth to service 4 times the number of connections; it's not exactly the same load on the server. I submit that any positive change is better than no change at all, and everyone limiting themselves to 1 download at a time will do this.

The server can always manage its bandwidth to ensure people aren't stalling however if it reaches its maximum simultaneous connections it is totally unable to service any more requests until someone is finished, which, when you consider the number of users and the number of potential simultaneous requests results in a DDOS-like effect where the server is essentially rendered non-communicable at times by the sheer number of connections, not the bandwidth being consumed strictly.

You will always get a usable work unit faster if you're using all available bandwidth for the one transfer; how is that going to "actually make things worse" exactly?

You will always get a usable work unit faster if you're using all available bandwidth for the one transfer; how is that going to "actually make things worse" exactly?