A Modest Proposal for Easy FREE Bandwidth Relief

I think that the main reason SETI uses so much bandwidth is the number of shorties that we are inundated with lately. Without that, we would be using much less BW.

Example: my 2 crunchers. Each has 2 GTX 460s running two threads each, so I am crunching 8 WUs at once. On my machines, a shortie takes roughly 5 minutes, so I do about 100/hour. So I cause d/l of about 36MB/hour when doing shorties - 10KB/sec just for me. So I am using roughly 1/100 of the available BW all by myself. No wonder things get clogged up! When I do regular WUs (15-20 minutes each) I use 1/3 to 1/4 that much BW.

So here's a proposal for a BW workaround: send shorties only to CPUs, just like VLARs! Then they will take much longer to do, so the BW becomes less choked. It can't be much of a programming change, since the servers are aware that shorties are shorties, by estimated GFLOPs.

Alternatively, or in addition, they could be stored for sending when things are less hectic - a 2TB drive could hold > 5 million of them in reserve.

The amount of science being done (measured by GFLOPs) wouldn't change, just the use of BW.

What do you think?
____________

In that case, if all MB WUs being produced were shorties, at least they wouldn't be fighting with other MB WUs for use of the BW. Surely, the servers could detect this situation ("we have no more stuff for GPUs") and allow some shorties to flow to graphics cards.
____________

Thank you JRavin for an eminently sensible but temp idea, until we can get co-located or procure more bandwidth. As far as running out of GPU tasks
We do that all the time with our current problems.

Bill

IIRC, more bandwidth is more a political problem at Berkeley than a technical one, so it may not happen for a LONG time. That's why I thought this up.

And: you're welcome!
____________

Example: my 2 crunchers. Each has 2 GTX 460s running two threads each, so I am crunching 8 WUs at once. On my machines, a shortie takes roughly 5 minutes, so I do about 100/hour. So I cause d/l of about 36MB/hour when doing shorties - 10KB/sec just for me. So I am using roughly 1/100 of the available BW all by myself. No wonder things get clogged up! When I do regular WUs (15-20 minutes each) I use 1/3 to 1/4 that much BW.

Example: my 2 crunchers. Each has 2 GTX 460s running two threads each, so I am crunching 8 WUs at once. On my machines, a shortie takes roughly 5 minutes, so I do about 100/hour. So I cause d/l of about 36MB/hour when doing shorties - 10KB/sec just for me. So I am using roughly 1/100 of the available BW all by myself. No wonder things get clogged up! When I do regular WUs (15-20 minutes each) I use 1/3 to 1/4 that much BW.

This ratio is about the same for CPUs, "normal" AR WU takes about 3-4 times longer than VHAR. If we consider the total available computing power constant, sending shorties to CPUs won't help with anything, you will download less data for your GPUs and more for the CPUs.

(snippage)

This might be useful if we hit another issue like Nvidia cards choking on VLAR work. Then that selection could be removed/disabled or something along those lines.

Perhaps they could tag tasks as VHAR like they do with VLAR & then setup sub projects for each type. I'm thinking something like how Primegrid works.
Example for those that don't run that project. http://www.hal6000.com/seti/images/pg_subs.png
Then users could disable/enable their hardware for each type.
Seti@Home Enhanced VHAR
Seti@Home Enhanced
Seti@Home Enhanced VLAR
Seti@Home v7 VHAR
Seti@Home v7
Seti@Home v7 VLAR
SETI@home v7 large workunits
Astropulse v6

This might be useful if we hit another issue like Nvidia cards choking on VLAR work. Then that selection could be removed/disabled or something along those lines.

Example: On my machines, shorty is 5 minutes on GPU, 50 minutes on CPU, normal WU is roughly 2 hours on CPU but 12 minutes on GPU. Say 4 cores of CPU and 4 threads of GPU. Then, in an hour (now) 100 shorties on GPU, 2 normals on CPU. If we reverse that, we have about 5 shorties on CPU and 20 normals on GPU, so we need to send only about 1/4 the total amount of data.

Example: my 2 crunchers. Each has 2 GTX 460s running two threads each, so I am crunching 8 WUs at once. On my machines, a shortie takes roughly 5 minutes, so I do about 100/hour. So I cause d/l of about 36MB/hour when doing shorties - 10KB/sec just for me. So I am using roughly 1/100 of the available BW all by myself. No wonder things get clogged up! When I do regular WUs (15-20 minutes each) I use 1/3 to 1/4 that much BW.

This ratio is about the same for CPUs, "normal" AR WU takes about 3-4 times longer than VHAR. If we consider the total available computing power constant, sending shorties to CPUs won't help with anything, you will download less data for your GPUs and more for the CPUs.

No - because work on CPUs is much slower than GPUs (by a factor of roughly 10:1 in my case). So fewer shorties will be needed to be sent to CPUs per unit time, and more normal WUs will go to GPUs, but the ratios should be more favorable.

Example: On my machines, shorty is 5 minutes on GPU, 50 minutes on CPU, normal WU is roughly 2 hours on CPU but 12 minutes on GPU. Say 4 cores of CPU and 4 threads of GPU. Then, in an hour (now) 100 shorties on GPU, 2 normals on CPU. If we reverse that, we have about 5 shorties on CPU and 20 normals on GPU, so we need to send only about 1/4 the total amount of data. Of course, YMMV.

The problem is that most shorties seem to be marked for GPU, not CPU, which is causing the BW problem.

there is also reschedule...
____________

there is also reschedule...

Has anyone considered the options of splitting and distributing the work units from a location other than berkeley? It will then redirect a good amount of 'outbound' data away from these overloaded berkeley bandwidth. The completed WUs will only be reported back to berkeley.

http://setiathome.berkeley.edu/forum_thread.php?id=70718

You might want to read the first post in that thread.

Thanks for that link bill. However, that means the servers are still located within the berkeley network, clogged by all the politics going on. I do remember reading someone offering to host some of the servers for free. Now, I am no expert in this area, just thinking out loud.

Apologies if the above point has already been discussed.

Has anyone considered the options of splitting and distributing the work units from a location other than berkeley? It will then redirect a good amount of 'outbound' data away from these overloaded berkeley bandwidth. The completed WUs will only be reported back to berkeley.

(snippage)

This might be useful if we hit another issue like Nvidia cards choking on VLAR work. Then that selection could be removed/disabled or something along those lines.

Not all Nvidia cards choke on VLARs. It would be nice people
that should know better would quit regurgitating this untruth.

I am aware it only is an issue for some devices. I was using that as an example because it was a significant issue to cause a change in the project.

I would not mind seeing a way to limit or stop or limit VLAR tasks to my 24 core box. As it tends to get rather unresponsive running 24 VLAR's at once. VHAR's it chews through like a mad man.