To S@H Crew -Please shut down S@H ( for week, or two)

Why?
In that case you will have time to fix all thing that bother you, examine servers, and do all thing you do every Tuesday.
After that I hope that most of your problems will be fixed, and all parts of system will work normal.
Thanks.

P.S Please do not throw flame at me, it is just my view of solving this.
____________
I am cruncher :)
I LOVE SETI BOINC :)

Why?
In that case you will have time to fix all thing that bother you, examine servers, and do all thing you do every Tuesday.
After that I hope that most of your problems will be fixed, and all parts of system will work normal.
Thanks.

P.S Please do not throw flame at me, it is just my view of solving this.

Why?
In that case you will have time to fix all thing that bother you, examine servers, and do all thing you do every Tuesday.
After that I hope that most of your problems will be fixed, and all parts of system will work normal.
Thanks.

P.S Please do not throw flame at me, it is just my view of solving this.

Their current issue is the 100mbit network connection to the rest of the world can't cope with the 100,000+ users all trying to upload and download. It's not a server issue. Shutting the project down for a week isn't going to solve this.

Why?
In that case you will have time to fix all thing that bother you, examine servers, and do all thing you do every Tuesday.
After that I hope that most of your problems will be fixed, and all parts of system will work normal.
Thanks.

P.S Please do not throw flame at me, it is just my view of solving this.

Their current issue is the 100mbit network connection to the rest of the world can't cope with the 100,000+ users all trying to upload and download. It's not a server issue. Shutting the project down for a week isn't going to solve this.

Please tell me number of aprox number of active hosts before one month?
Then 100mbit network connection wasn't problem?

Please tell me number of aprox number of active hosts before one month?
Then 100mbit network connection wasn't problem?

Thanks for explanation.
But in that case, what SETI crew can do?
Nothing.
Crunchers well be unpleased, system will be "minutes" away from total collapse,and whole project will be very, very slow. :(
I think it is not bright future of SETI @ home :(
____________
I am cruncher :)
I LOVE SETI BOINC :)

If network traffic is the bottleneck, and it would appear to be so, then there is a slightly different bold solution.

If my PCs are typical of the rest of the S@H community then a fair amount of network traffic is due to re-tries due to time-outs of one sort or another within "the BOINC environment". This is bad news as each time out requires several messages to be passed between the two communicating system = network traffic. Each time-out requires a proportion of the data to be retransmitted = network traffic (it appears that sometimes a WU download/upload restarts and other times it is a complete retransmission - retransmission is a lot of network traffic if the WU is an AP download that was 90% complete, whereas the completion of a results upload is a relatively small network traffic load).

So how about increasing the time allowed before a client, or server, end triggered re-try? I know this would impact on the size of buffer required, but it would reduce the amount of network traffic required to trigger and manage retries. I doubt that it would take a substantial change to have a significant impact, I'd guess increasing the time-out by 10% would reduce the number of retries by between 20 and 50% - now that's some saving!
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

In that case:many , especially fast crunchers can freely stop doing S@H.
He can have additional work for lets say 5 days, crunch that in 1.5 days, and in rest time waiting to get new one. And if you increase re-try time it is obvious that new work will come in very slow rate...
____________
I am cruncher :)
I LOVE SETI BOINC :)

If the astronomers are studying point objects intensively, they keep the telescope pointing at the same place in the sky for extended durations. Those recordings have small changes in 'Angle Range' in the resulting SETI tasks - the dreaded VLARs beloved of CUDA users. Other astonomers use the telescope for survey work, and sweep the sky rapidly looking for new objects. These recordings have large Angle Ranges (the telescope aiming point changes quickly), and the resulting SETI tasks are VHARs, or 'shorties'. Seti can't get as much useful information out of them, so doesn't spend as long looking at them.

I doubt very much you will ever see a mass exodus from Seti@Home from the longterm dedicated users. You have to admit the problems right now are tiny compared to last year, Before we had the 3 new servers.

When we got the new servers I went from a 3 day cache to the one day I have now. If I run out of work its no big deal,as I run back up projects.

Now Im not saying that it isnt frustrating to run out of work or up / down loads jam up because it is. Its frustrating in the fact that im still trying to find out what my max rac is for my i7:)

In any case no matter what happens, I will crunch till the guys in the lab turn off the lights and pull all the plugs.
____________

Anyone interested in doing a bit of amateur 'storm forecasting' can look at the Arecibo Observatory Telescope Schedule (http://www.naic.edu/vscience/schedule/scedfra2.htm). The projects to look for are those which mention 'ALFA' on the schedule pages (those are the ones which SETI@home can eavesdrop on).

It looks as it the early-April recordings which are causing most of the current problems originated with A2130 - The G-ALFA Continuum Transit Survey (details, PDF format). The good news? They took a break after 3 May. The bad news? They're due back on Monday 6 June.....

Apart from throwing hardware at the problem, there are some suggestions we could make to alleviate it in the short term (before Moore's Law catches up with us again).

1) Hire a real, top-notch, world-class specialist networking consultant to get the most out of the existing pipe. As Rob Smith says, all the back-off / retry traffic adds to the overall load. More sophisticated traffic-shaping and demand management might actually improve the effective, useful, throughput. Although our much-loved project staff have a wealth of practical experience with the SETI hardware, they are generalists, and I don't think it's unfair to suggest that a specialist could help out.

2) Look at the 'tapes' and determine the work mix, before loading them into the splitters. If tapes with a high proportion of VHARs could be rationed, and only split one or two at a time alongside tapes with other recording patterns, the average download volume could be smoothed out, rather than bunching into these 'storm peaks' (which we've seen before, and will no doubt see again).

1) Hire a real, top-notch, world-class specialist networking consultant to get the most out of the existing pipe. As Rob Smith says, all the back-off / retry traffic adds to the overall load. More sophisticated traffic-shaping and demand management might actually improve the effective, useful, throughput. Although our much-loved project staff have a wealth of practical experience with the SETI hardware, they are generalists, and I don't think it's unfair to suggest that a specialist could help out.

If I understand correctly, each work unit is processed by two different clients, say, me and my wingman. If I guess correctly, this means that the same work unit data is sent twice by the Seti servers, say once to me and once to my wingman. When I look at the work units details, this seems to happen most often at the same time (within the same second), or just within a few seconds interval.

So am I wrong in thinking that it might be possible to improve the situation in most cases by putting the data only ONCE on the pipe by the Seti servers, and having both clients listen to the same data? I let the details to a network specialist :-), but there would probably exist several possible solutions.

If my assumptions are correct and this could be possible for all the work units, it would then be possible to save up to 50% of the bandwith ? Or, in other words, to nearly double the download throughput ?

Apologies if this seems a bit naive; I would be glad to learn from my mistakes. Thanks and respect to all the people who spend their time helping naive users like me.

Hi Richard.

Hire a real, top-notch, world-class specialist networking consultant to get the most out of the existing pipe.

Look at the 'tapes' and determine the work mix, before loading them into the splitters

I did not find the post yet, but I thought I remembered Matt saying that getting a 1Gb link to Seti was indeed a possibility, depending on some investment in hardware, but mostly working through politics with Berkeley IT.

The possibility was also suggested at the time to keep the existing 1Mb link, and just add some bandwidth from the campus link.

Several folks had offered to donate for the hardware to make this happen.

Wonder...has any progress has been made on that front?
____________
******

"Ask not, what your kitty can do for you. Ask what you can do for your kitty."

As it is kitten, so shall it be done.

Very interesting thread. Thanks Richard for a very detailed explanation.

Possible (long term) solution?

Is it possible for the Project Admin in conjunction with all our dedicated crunchers, to allocate specifically, part of donated funds (if stated by donator of course) to build up a dedicated fibre optic fund?

This of course being ongoing until target is met. I should think that Berkeley Admin would not stand in the way as they would benefit from it as well....

just a thought........

I'm pretty sure that there are many out there who do not wish to see this project in trouble.
____________

I seem to remember seeing a cost of either $38,000 or $48,000 to run it for the lab. Either way that's a lot of fund raising and there is a lot of other stuff that needs replaced that could use that kind of money and is more crucial than the line.