Message boards :
Technical News :
Tenaya (Feb 24 2009)
Message board moderation
Previous · 1 · 2 · 3 · 4
Author | Message |
---|---|
Virtual Boss* Send message Joined: 4 May 08 Posts: 417 Credit: 6,440,287 RAC: 0 |
Simply locating the download server downhill with 1 GB access to the world is what would improve the situation. Each WU or application would only go from the SSL lab to that server once, multiple downloads by hosts would not be seen on the 100 MB link. The issue is maintenance and remote management, not any technical difficulty in having that server at a distance. So if the download server(s) were located at the 1GB connection point - 100 Mb/s going out to clients would only equate to ~45 Mb/s traffic from lab to download server over 100 Mb/s link (plus whatever ??? bw was needed for the download server to interact with the other servers up the hill).( This would seem to reduce the constriction imposed by the 100 Mb/s link by a a considerable amount = Good effect. But as you state would cause some servicing difficulties = Bad effect. And probably the extra latency between servers = Very bad effect?. |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
You have yet to propose a solution. Let me rephrase...propose a viable solution. Moving the download server improves the current situation. Yes there are drawbacks, including increased system complexity and probably some extra cost too. Throttling the server in its existing location only guarantees more DDOS 'attacks' when the network clogs up. |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
As I/others have suggested before, a better solution would be to distribute the servers to other sites. Keep the main results database at Berkeley, but send the raw data to one or more sites willing to host splitting and downloading. Distribution is a solution to the scalability question. It would take coordination and planning, yes. But alternative suggestions, such as the ones discussed here, also take effort to plan and implement. Why not break the parochial model of everything has to occur at Berekeley? Of course, I don't know who may be stable enough out there to provide a second host for download services, but it is worth asking the question. I say, lets spend 'our' efforts to make plans and take steps to have 1-3M active hosts, 3-10x increase over today. |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
This is really important: A solution needs to solve the problem, not just move it around. Matt has repeatedly said the server cluster is pretty "atomic" -- that things are all sufficiently interrelated that splitting off one function or another would be a problem. The only exception I can see is downloading the science application. That is one sliver that could be moved, and since trying to do that with coral cache(?) caused problems last weekend, I'd think about the work involved in moving those files off-site. It's also a very simple function -- same files for everybody. I don't think two result files are the same, but I've never seen two different result files for the same WU -- and I doubt anyone else has. Sure, there could be some meta format that modified one "meta-result" into the two result files and that'd do for all but reissues. ... but without that, you don't get the 2:1 leverage that some posts on the thread suggest would be possible. I don't know enough about where the connection goes from gigabit to 100 megabits, so I don't know how much rack space would be available. I don't know the access issues, and the people who have to do the work keeping all of this running would be concerned about access. In my opinion (having run servers on-site and off-site) the personal cost of having the servers off-site is kind-of high. I lost a small server over the weekend, and it was a minor hassle because the servers are all right here. My suggestion is to make the BOINC client less aggressive, so we don't have these big loads right after an outage -- and to add a communications path from the project to BOINC so it can tell everyone to slow down. But I'm not going to make any demands that this be implemented. If I get time when some of my personal projects are done, I'm going to write the code. |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
Throttling the server in its existing location only guarantees more DDOS 'attacks' when the network clogs up. I used the word "throttling" in a very narrow way -- to point out that the splitter process, pushing data to an off-site download server must be controlled or that one process could easily dominate the 100 megabit connection. I pointed out that throttling that one process would be useful to maximize throughput between the splitter and a hypothetical off-site download server. If we're talking about the client-server connection, it is important to NOT THROTTLE them. If you slow up the bandwidth between "A" and "B" you tie up all the associated resources -- processes and connections last longer, so you likely have more of them running at once, and overhead increases. For the client/server communications, to maximize throughput, you want to figure out how much bandwidth you can afford for downloads (70 megabits??) and then control the clients so that you have that right around that many clients downloading at any one time. Let's say for simplicity that the average cruncher has a megabit (some faster, some slower). If you could keep about 70 downloads going at all times, things will go very fast. Doubling that probably doesn't hurt much. If you have 7,000 attempted downloads, overhead goes up, timeouts and retries go up (because of congestion) and throughput goes way down. The server might not be able to take 7,000 simultaneous connections, so the SYN packets come in and RSTs go back and they use bandwidth, and CPU time. |
ML1 Send message Joined: 25 Nov 01 Posts: 20147 Credit: 7,508,002 RAC: 20 |
... "throttling" in a very narrow way ... The use of linux (*nix) "tc" to give all icmp/ack/syn packets priority and queue up all other packets with greater latency to 'slow' them down would go a good way to gracefully limiting the bandwidth at source, and all before packets get randomly dumped because the link is choked. Yet more clever "tc" rules can be used to positively favour completing existing connections to again help to clear the deluge with a minimum of lost connections and resend requests. Can the cisco head be configured so?... Another trick would be to do the bandwidth management in the boinc server in the first place... And as for getting 1 Gbit 'down the hill', I'd still expect a uWave link to cost an awful lot less than $80k including labour. There's no trenches to dig! Or are the end points actually more a case of "over" a hill rather than line-of-sight down? If fibre is to be used, does s@h get payback when capacity is used by others?... Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Aurora Borealis Send message Joined: 14 Jan 01 Posts: 3075 Credit: 5,631,463 RAC: 0 |
One of the proposals, I've seen on the Boinc mailing list, was to include torrent capability in the Boinc manager. This would obviously need to be a user selectable option since most companies and some campuses would object to using this option on their computers. This idea does however have a great potential for reducing the bandwidth required to distribute new applications. I have no idea if any of the volunteer developers has taken up the challenge of pursuing this option or how quickly it could be implemented. Boinc V7 perhaps... Boinc V7.2.42 Win7 i5 3.33G 4GB, GTX470 |
benDan Send message Joined: 28 Nov 04 Posts: 18 Credit: 6,188,677 RAC: 0 |
Just a newbie question ... please no flames. does bandwidth include stuff going out and stuff going in? If both, do you get more if stuff only goes 1 way? What if you had another site with a 100 whatever line. if one site sent stuff out and the other site received stuff would that speed things up? I know this seems very basic to you folks but I'm trying to figgure this stuff out. Thanks -- benDan |
Mike O Send message Joined: 1 Sep 07 Posts: 428 Credit: 6,670,998 RAC: 0 |
what about physical transportanion of the data to a 'out of closet' server? Swapable HDDs? Scratch that.. They would need. 1. a nother room to put the server in. 2. the hardware to pull this off. 3. someone to make sure the data is transfered. I dont think its worth the time, effort or expence. WUs would need to be created fast enough for a full 24 hours of downloads (72 for weekends). I dont think LANDO and BAMBI can keep up. Im wondwering of it would be posible to use off the shelve WI-FI to get from the closet to the remote server. Do they make such a thing as a repeater like CB radio? Im not sure it would handle the data even if they could run 2 units together. My wireless maxes at 58MBS. Good news.. I had NO problems with getting work over the weekend. Not Ready Reading BRAIN. Abort/Retry/Fail? |
Aurora Borealis Send message Joined: 14 Jan 01 Posts: 3075 Credit: 5,631,463 RAC: 0 |
Just a newbie question ... please no flames. We were all been newbies at one time. does bandwidth include stuff going out and stuff going in? I believe that in theory you could have 100 MB in both directions. In reality once you get over 90 MB you can start to have trouble. The problem occurs if one direction gets saturated. Every packet of data requires acknowledgement of receipt in the opposite direction. If, as happened last week, the downloads are saturated then upload have difficulty because there is no download bandwidth left to confirm that the uploaded packet has been received. What if you had another site with a 100 whatever line. The Seti servers have too many interdependencies to split things up into separate locations and too few people to take care of problems when they do occur, which is all the time. I know this seems very basic to you folks but I'm trying to figgure this stuff out. We are all learning together. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14649 Credit: 200,643,578 RAC: 874 |
Last time we had a conversation like this (probably after the last big outage, or the last big data-storm - it comes round in cycles like that), about the only thing that seemed to make sense technically was a complete, self-contained, black-box 'SETI on a stick' - or more likely, SETI on a server on a pallet. If (and it's a very big if) you could put all the elements of the SETI workstream onto one server - BOINC database, science database, splitters, scheduler, validators, assimilators etc. etc. - you could load it up with one or more raw Arecibo recordings and ship it off to a closed community somewhere, then retrieve the science database when all was done. If you could work out a suitable ratio of tapes per college campus per semester, you could even set up a little league - first server back gets a small (non-monetary) prize - I believe US students will happily compete for pennants? It wouldn't be SETI as we know it - no international message boards (logins would be retricted to campus), no external stats sites - anything which related to user ID, host ID, WU ID or task ID would have to be self-contained without interaction with the main international SETI site - but it seems to be about the only way of overcoming the atomicity problem. |
benDan Send message Joined: 28 Nov 04 Posts: 18 Credit: 6,188,677 RAC: 0 |
Thanks for the info. people were talking about mirror sites and I understood that to be a different machine not necessarily a distant machine. guido.man wrote: <Right now Seti@Home is coping quite well with a 100 Mb connection to the internet for the most part.> So I was thinking add one more line, one for up and one for down. Perhaps mirror is the wrong term to use. -- benDan |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
And as for getting 1 Gbit 'down the hill', I'd still expect a uWave link to cost an awful lot less than $80k including labour. There's no trenches to dig! Or are the end points actually more a case of "over" a hill rather than line-of-sight down? They aren't quite on top of the hill from what I can tell, so it's a question of finding something they can "see" from the hilltop that has bandwidth without some other expense. |
Zydor Send message Joined: 4 Oct 03 Posts: 172 Credit: 491,111 RAC: 0 |
uWave is a viable solution in a broad sense. A 1Gb link would be in the region of $55,000 and very quick to install. As always the devil is in the detail - and the "extras" that turn out to be practical essentials rolf:) - however in a generalised sense the cost of the main carrier infrastructure is well below the $80K pain point being bandied about for cable. Worth revisiting even if looked at before, as prices are coming down relentlessly, as usual when technology moves onwards and upwards. Quite a few US Companies do this, and the door could be open to a sponsorship (or part sponsorship) deal that would bring the cost down even more. The demand for short range uWave systems is high, and competition is fierce - a link like this one just may attract a Marketing Director's eye .... |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
uWave is a viable solution in a broad sense. A 1Gb link would be in the region of $55,000 and very quick to install. As always the devil is in the detail - and the "extras" that turn out to be practical essentials rolf:) - however in a generalised sense the cost of the main carrier infrastructure is well below the $80K pain point being bandied about for cable. Worth revisiting even if looked at before, as prices are coming down relentlessly, as usual when technology moves onwards and upwards. Remember that SETI@Home can't "bootleg" this in. The on-campus part has to be ordered, provisioned and supported by CNS. I think that has advantages and disadvantages, but on the whole it probably works out best. It just eliminates some of the competitive shopping that might be done. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
Im wondwering of it would be posible to use off the shelve WI-FI to get from the closet to the remote server. Do they make such a thing as a repeater like CB radio? Im not sure it would handle the data even if they could run 2 units together. My wireless maxes at 58MBS. As far as I know CB is not allowed to use repeaters, Amatuer (HAM) radio is. Off the shelf WI-FI is not suitable as the range is much too limited. The required range is about a mile. The range of standard WI-FI is less than half that in perfect conditions (and when are conditions EVER perfect). As far as I know, repeaters are not done with WI-FI, and each step would need electricity and an enclosure. WI-FI is also not a speed boost over 100Mb. BOINC WIKI |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.