Tenaya (Feb 24 2009)
Tenaya (Feb 24 2009) |
|
| log in |
Message boards : Technical News : Tenaya (Feb 24 2009)
Previous · 1 · 2 · 3 · 4
| Author | Message |
|---|---|
Simply locating the download server downhill with 1 GB access to the world is what would improve the situation. Each WU or application would only go from the SSL lab to that server once, multiple downloads by hosts would not be seen on the 100 MB link. The issue is maintenance and remote management, not any technical difficulty in having that server at a distance. So if the download server(s) were located at the 1GB connection point - 100 Mb/s going out to clients would only equate to ~45 Mb/s traffic from lab to download server over 100 Mb/s link (plus whatever ??? bw was needed for the download server to interact with the other servers up the hill).( This would seem to reduce the constriction imposed by the 100 Mb/s link by a a considerable amount = Good effect. But as you state would cause some servicing difficulties = Bad effect. And probably the extra latency between servers = Very bad effect?. | |
ID: 871297 ·  | |
You have yet to propose a solution. Let me rephrase...propose a viable solution. Moving the download server improves the current situation. Yes there are drawbacks, including increased system complexity and probably some extra cost too. Throttling the server in its existing location only guarantees more DDOS 'attacks' when the network clogs up. | |
| ID: 871303 · | |
|
As I/others have suggested before, a better solution would be to distribute the servers to other sites. Keep the main results database at Berkeley, but send the raw data to one or more sites willing to host splitting and downloading. | |
| ID: 871350 · | |
This is really important: A solution needs to solve the problem, not just move it around. Matt has repeatedly said the server cluster is pretty "atomic" -- that things are all sufficiently interrelated that splitting off one function or another would be a problem. The only exception I can see is downloading the science application. That is one sliver that could be moved, and since trying to do that with coral cache(?) caused problems last weekend, I'd think about the work involved in moving those files off-site. It's also a very simple function -- same files for everybody. I don't think two result files are the same, but I've never seen two different result files for the same WU -- and I doubt anyone else has. Sure, there could be some meta format that modified one "meta-result" into the two result files and that'd do for all but reissues. ... but without that, you don't get the 2:1 leverage that some posts on the thread suggest would be possible. I don't know enough about where the connection goes from gigabit to 100 megabits, so I don't know how much rack space would be available. I don't know the access issues, and the people who have to do the work keeping all of this running would be concerned about access. In my opinion (having run servers on-site and off-site) the personal cost of having the servers off-site is kind-of high. I lost a small server over the weekend, and it was a minor hassle because the servers are all right here. My suggestion is to make the BOINC client less aggressive, so we don't have these big loads right after an outage -- and to add a communications path from the project to BOINC so it can tell everyone to slow down. But I'm not going to make any demands that this be implemented. If I get time when some of my personal projects are done, I'm going to write the code. ____________ | |
| ID: 871372 · | |
Throttling the server in its existing location only guarantees more DDOS 'attacks' when the network clogs up. I used the word "throttling" in a very narrow way -- to point out that the splitter process, pushing data to an off-site download server must be controlled or that one process could easily dominate the 100 megabit connection. I pointed out that throttling that one process would be useful to maximize throughput between the splitter and a hypothetical off-site download server. If we're talking about the client-server connection, it is important to NOT THROTTLE them. If you slow up the bandwidth between "A" and "B" you tie up all the associated resources -- processes and connections last longer, so you likely have more of them running at once, and overhead increases. For the client/server communications, to maximize throughput, you want to figure out how much bandwidth you can afford for downloads (70 megabits??) and then control the clients so that you have that right around that many clients downloading at any one time. Let's say for simplicity that the average cruncher has a megabit (some faster, some slower). If you could keep about 70 downloads going at all times, things will go very fast. Doubling that probably doesn't hurt much. If you have 7,000 attempted downloads, overhead goes up, timeouts and retries go up (because of congestion) and throughput goes way down. The server might not be able to take 7,000 simultaneous connections, so the SYN packets come in and RSTs go back and they use bandwidth, and CPU time. ____________ | |
| ID: 871375 · | |
... "throttling" in a very narrow way ... The use of linux (*nix) "tc" to give all icmp/ack/syn packets priority and queue up all other packets with greater latency to 'slow' them down would go a good way to gracefully limiting the bandwidth at source, and all before packets get randomly dumped because the link is choked. Yet more clever "tc" rules can be used to positively favour completing existing connections to again help to clear the deluge with a minimum of lost connections and resend requests. Can the cisco head be configured so?... Another trick would be to do the bandwidth management in the boinc server in the first place... And as for getting 1 Gbit 'down the hill', I'd still expect a uWave link to cost an awful lot less than $80k including labour. There's no trenches to dig! Or are the end points actually more a case of "over" a hill rather than line-of-sight down? If fibre is to be used, does s@h get payback when capacity is used by others?... Regards, Martin ____________ Mandriva Linux A user friendly OS! See new freedom Mageia2 The Future is what We make IT (GPLv3) | |
| ID: 871390 · | |
|
One of the proposals, I've seen on the Boinc mailing list, was to include torrent capability in the Boinc manager. This would obviously need to be a user selectable option since most companies and some campuses would object to using this option on their computers. This idea does however have a great potential for reducing the bandwidth required to distribute new applications. | |
| ID: 871391 · | |
|
Just a newbie question ... please no flames. | |
| ID: 871395 · | |
|
what about physical transportanion of the data to a 'out of closet' server? | |
| ID: 871404 · | |
Just a newbie question ... please no flames. We were all been newbies at one time. does bandwidth include stuff going out and stuff going in? I believe that in theory you could have 100 MB in both directions. In reality once you get over 90 MB you can start to have trouble. The problem occurs if one direction gets saturated. Every packet of data requires acknowledgement of receipt in the opposite direction. If, as happened last week, the downloads are saturated then upload have difficulty because there is no download bandwidth left to confirm that the uploaded packet has been received. What if you had another site with a 100 whatever line. The Seti servers have too many interdependencies to split things up into separate locations and too few people to take care of problems when they do occur, which is all the time. I know this seems very basic to you folks but I'm trying to figgure this stuff out. We are all learning together. | |
| ID: 871411 · | |
|
Richard Haselgrove  Volunteer tester Send message Joined: 4 Jul 99 Posts: 7670 Credit: 40,284,218 RAC: 21,307  | |
|
Last time we had a conversation like this (probably after the last big outage, or the last big data-storm - it comes round in cycles like that), about the only thing that seemed to make sense technically was a complete, self-contained, black-box 'SETI on a stick' - or more likely, SETI on a server on a pallet. | |
| ID: 871416 · | |
|
Thanks for the info. | |
| ID: 871418 · | |
And as for getting 1 Gbit 'down the hill', I'd still expect a uWave link to cost an awful lot less than $80k including labour. There's no trenches to dig! Or are the end points actually more a case of "over" a hill rather than line-of-sight down? They aren't quite on top of the hill from what I can tell, so it's a question of finding something they can "see" from the hilltop that has bandwidth without some other expense. ____________ | |
| ID: 871450 · | |
|
uWave is a viable solution in a broad sense. A 1Gb link would be in the region of $55,000 and very quick to install. As always the devil is in the detail - and the "extras" that turn out to be practical essentials rolf:) - however in a generalised sense the cost of the main carrier infrastructure is well below the $80K pain point being bandied about for cable. Worth revisiting even if looked at before, as prices are coming down relentlessly, as usual when technology moves onwards and upwards. | |
| ID: 871496 · | |
uWave is a viable solution in a broad sense. A 1Gb link would be in the region of $55,000 and very quick to install. As always the devil is in the detail - and the "extras" that turn out to be practical essentials rolf:) - however in a generalised sense the cost of the main carrier infrastructure is well below the $80K pain point being bandied about for cable. Worth revisiting even if looked at before, as prices are coming down relentlessly, as usual when technology moves onwards and upwards. Remember that SETI@Home can't "bootleg" this in. The on-campus part has to be ordered, provisioned and supported by CNS. I think that has advantages and disadvantages, but on the whole it probably works out best. It just eliminates some of the competitive shopping that might be done. ____________ | |
| ID: 871529 · | |
|
John McLeod VII Volunteer developer Volunteer tester  Send message Joined: 15 Jul 99 Posts: 22343 Credit: 440,201 RAC: 100  | |
Im wondwering of it would be posible to use off the shelve WI-FI to get from the closet to the remote server. Do they make such a thing as a repeater like CB radio? Im not sure it would handle the data even if they could run 2 units together. My wireless maxes at 58MBS. As far as I know CB is not allowed to use repeaters, Amatuer (HAM) radio is. Off the shelf WI-FI is not suitable as the range is much too limited. The required range is about a mile. The range of standard WI-FI is less than half that in perfect conditions (and when are conditions EVER perfect). As far as I know, repeaters are not done with WI-FI, and each step would need electricity and an enclosure. WI-FI is also not a speed boost over 100Mb. ____________   BOINC WIKI | |
| ID: 871539 · | |
Message boards : Technical News : Tenaya (Feb 24 2009)
| Copyright © 2013 University of California |
