Message boards :
Technical News :
On Bandwidth (Jul 08 2009)
Message board moderation
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
Once again it took the replica all night to recover. I started it up this morning, and it's catching up now. Well, almost. I'll turn the "show tasks/results" feature back on once it really starts catching up. There's been a lot of discussion lately about our bandwidth woes. I actually talked to Blurf this morning on the phone regarding the (rather generous) push to donate money/hardware towards solving this problem. Let me try to paint a big picture here. We pay for a gigabit of bandwidth from our private ISP (Hurricane Electric), but can only use 100 Mbits given current campus infrastructure. Most of campus is on gigabit already, but our lab is all the way up the hill - so it's much harder and more expensive to improve the old wiring/routing. The entire rest of the Space Lab uses about 10 Mbits/sec, so there is absolutely zero push by anybody else to spend money/effort on this project. Luckily, there was a spare 100 Mbit cable which is what we are using for the Hurricane Electric link. While we pay for our bits, they still have to route through campus in order to ultimately hook up with the right backbones. That means we have to adhere to campus's network specs, which in turn means we can only use very specific brands/models of hardware, and can only act once they've fully researched our needs. We opened up a ticket months ago asking to start this research. We got word a couple days ago this research has more or less finally begun. Not much progress, but still non-zero. This may seem impossibly slow, but campus really pretty much always has much bigger fish to fry. Plus our requests usually present them with something new they haven't dealt with before, and therefore they are far more careful. Ultimately, we should be presented with a couple options from campus which include exact pieces of hardware to be obtained. It's still not clear how much cable has to be upgraded and where, but we know we'll need two new routers, if not also other hardware. When campus gives us this final report, only then can we start figuring out how to obtain the necessary hardware. As for other options, like going wireless... There actually used to be a building down in the flats that got wireless bandwidth from us. The experience was that it was quite slow and prone to suffering during bad weather, etc. This was a while ago, but still there is enough concern about reliability that nobody seems to want to go down this path. Of course, another option is relocating our whole project down the hill (where gigabit links are readily available), or at least the server closet. Since the backend is quite complicated with many essential and nested dependencies it's all or nothing - we can't just move one server or functionality elsewhere - we'd have to move everything (this has been explained by me and others in countless other threads over the years). If we do end up moving (always a possibility) then all the above issues are moot. Another important thing to consider is that we can always reduce are bandwidth demands via other means, which I also explained in another recent threads. Things like removing redundancy (and putting a cap on workunit downloads per day per host), or adding scientific analysis. Or, to be a little extreme, calling SETI@home done, turning off the downloads for good, and moving on to the next thing (something I am actually in favor of doing sooner than later, but the others around here seem to disagree). I definitely appreciate past and current efforts to help us get beyond the current bandwidth crisis. However, as noted above, there are enough variables involved that I'd hate for you all to start collecting money directed towards a solution to a problem which might just go away. In the meantime, thanks as always for your patience (and crunching time when you actually do get workunits) - we'll keep working with what we got and see if we can't get beyond the storm sooner. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Nick Fox Send message Joined: 5 Jan 04 Posts: 46 Credit: 2,834,922 RAC: 0 |
Thanks for the clarification Matt... and thanks for all your efforts to keep things moving! |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
Or, to be a little extreme, calling SETI@home done, turning off the downloads for good, and moving on to the next thing (something I am actually in favor of doing sooner than later, but the others around here seem to disagree). I didn't understand this point; could someone elaborate? |
Blurf Send message Joined: 2 Sep 06 Posts: 8964 Credit: 12,678,685 RAC: 0 |
Thank you for the time for our discussion today, Matt. Kind of nice to put a voice to a set of posts. |
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
That was very informative and quite clear, Matt. Thanks for giving us the low-down on the progress of things for the bandwidth issue. |
ML1 Send message Joined: 25 Nov 01 Posts: 21235 Credit: 7,508,002 RAC: 20 |
I didn't understand this point; could someone elaborate? American rye humo(u)r... Or... Perhaps... There is a "better" way? My own view is that we have a wealth of data from Arecibo that is there just waiting to be analysed in all manner of various ways, for the hope of finding ET and along the way for finding many other things unexpected. A good question with all that is: How long and how far can we usefully go with this type of technology for attempting to find signs of ET technology? Note that the mere search in itself is good science in searching for how best to search! Keep searchin', Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
ML1 Send message Joined: 25 Nov 01 Posts: 21235 Credit: 7,508,002 RAC: 20 |
... CallingSETI@home done, sounds as if you're saying that in your opinion all the works people are doing now is useless, or at least that you don't need the results any longer, for some reasons. ... That's all a question of your own interpretation... SETI is done only when we actually find ET. We haven't found ET yet. We also have a very good experiment in how to push various hardware, compute, funding, and human limits in finding ET. Even Matt is part of the social aspects of the experiment :-) ! Keep searchin'! (Even if all you find is yourself...) Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
ML1 Send message Joined: 25 Nov 01 Posts: 21235 Credit: 7,508,002 RAC: 20 |
... And for those that can't think that one through... Who knows?! Keep searchin', Martin BTW: That's just one possible answer for both the science and for my postings. See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
ML1 Send message Joined: 25 Nov 01 Posts: 21235 Credit: 7,508,002 RAC: 20 |
... And... Have we broken the limits yet for a monolithic database hosted on a single server? Happy crunchin', Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
No no no I didn't mean that at all. I'm not saying you're wasting electricity. We still haven't fully covered the Arecibo sky yet with multibeam observations, for example. What I am saying is we don't need to adhere to a 24/7/365 uptime policy because of the fear that we'll lose our participants forever. I have enough professional experience to prove that if you want to win customers (or fans) back after a long hiatus, you simply ask them nicely, and they return. In fact I've proven this time and time again with the reminder e-mails. Others feel we need to keep the workflow happening at all costs or we're ruined. What do you think? Basically all I meant was that we can afford to take more long hiatuses to take care of long standing sub projects (like final data analysis?!), and perhaps come up with new/improved SETI@home-esque projects - all of which would be far easier to do if we were not as preoccupied with keeping data flow happening 100% of the time. - Matt Or, to be a little extreme, calling SETI@home done, turning off the downloads for good, and moving on to the next thing (something I am actually in favor of doing sooner than later, but the others around here seem to disagree). -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
ML1 Send message Joined: 25 Nov 01 Posts: 21235 Credit: 7,508,002 RAC: 20 |
No no no I didn't mean that at all... Darn... That makes for a very short thread! What I am saying is we don't need to adhere to a 24/7/365 uptime policy because of the fear that we'll lose our participants forever. I have enough professional experience to prove that if you want to win customers (or fans) back after a long hiatus, you simply ask them nicely, and they return. ... I agree. I'd be happy to see a hiatus so that science can be done or good fixes be put in place if needed rather than waste effort just for the sake of 'appearances'. I would hope that at least the people that care enough to check out these forums are also caring enough to appreciate a wider picture beyond just achieving whatever RAC... And thanks Matt as ever for the postings and the server efforts! Keep searchin', Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
JimHilty2 Send message Joined: 30 Apr 03 Posts: 75 Credit: 7,199,464 RAC: 0 |
Thank you Matt, pobably the simplest most satisfying post I have read on the boards in all the years. I totally agree. Start the analysis and if nothing found then we do it again but differently. thank you agaain No no no I didn't mean that at all. I'm not saying you're wasting electricity. We still haven't fully covered the Arecibo sky yet with multibeam observations, for example. |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
What I am saying is we don't need to adhere to a 24/7/365 uptime policy because of the fear that we'll lose our participants forever. (much deleted) Kind of like LHC, who made similar promises to SETI@Home -- sometimes there will be work, sometimes there won't. |
zpm Send message Joined: 25 Apr 08 Posts: 284 Credit: 1,659,024 RAC: 0 |
i agree with matt.... until fresh data(and un-noisy) is here... i say that the servers and system could use a break and it would give time for yall to get some back-end duties done instead of babysitting. I recommend Secunia PSI: http://secunia.com/vulnerability_scanning/personal/ Go Georgia Tech. |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
Matt, isn't it possible to use some old telephone lines, if there are any available from the building up the hill to down where the ISP's connection is coming in? VDSL might then be an option, if only temporarily and if it meets the distance. But isn't a bog standard telephone line capable of up to 10Gbits/sec? |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
i agree with matt.... until fresh data(and un-noisy) is here... i say that the servers and system could use a break and it would give time for yall to get some back-end duties done instead of babysitting. It seems to me that there is a bit of a catch-22 here. The way one detects "noisy" work is to run it through the system. If you get back a lot of -9's, it was noisy -- but there could be quiet work mixed in. The other problem (and someone will find the link and point to it, I'm sure) is that different angle ranges are just inherently fast or inherently slow, depending on how the telescope was moving. So, basically, ya splits yer tapes and ya takes yer chances. |
ML1 Send message Joined: 25 Nov 01 Posts: 21235 Credit: 7,508,002 RAC: 20 |
But isn't a bog standard telephone line capable of up to 10Gbits/sec Nice hopeful idea. However... The simple answer to that is: no. Using DSL, you're lucky to get 8Mbit/s over 1km of phone line cable. Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
But isn't a bog standard telephone line capable of up to 10Gbits/sec? No where near. A plain telco line is "Category 3" and is good up to about 16 MHz. 16 MHz is 16 megabaud, and that's 16 megabits if you use simple encoding. Cat6a is good to 500 MHz, and by encoding multiple bits per state that gets to 10 Gigabits. There are companies like Hatteras Networks who have demonstrated Gigabit speeds over some serious distances (more than a mile) by running slower speeds over many, many pairs. ... but if there aren't enough pairs in the building, that doesn't work. It is still up to the CNS department to pick the technology, and then give 'em a price. |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
Using DSL, you're lucky to get 8Mbit/s over 1km of phone line cable. I wouldn't be so sure; I'm more than 1 click away from the local hub and I have 20Mbit already 9the modem/router can go as high as 100Mbit for internet). Bog standard telephone lines. The fiber lies next to it, but it's not in use yet. But we aren't necessarily talking xDSL here, just data transfer. Just a (home) network and for that you can use old electricity cables, old telephone cables, old TV cables, whatever is around. See network Media in Home network for some examples and their maximum speeds. As for the telephone wires and their maximum speed, it may not be 10Gbit, but 1Gbit is a possibility. Depends on the state of the old line, though and the amount of hops it takes. But otherwise.... ;-) |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
Using DSL, you're lucky to get 8Mbit/s over 1km of phone line cable. Jord, What we're talking here isn't really signalling, it's radio that happens to run along wires. ... and as the frequency goes up, the wavelength goes down. Shorter wavelength means smaller antenna. At 10 MHz, a wavelength is right at 30 meters, and a wire that is an odd multiple of 7.5 meters long is going to radiate (and receive) signals. At 10 GHz, it's more like every 7.5 millimeters. As the wavelength gets shorter, the tolerances in a good transmission line get smaller. You can tolerate bridge taps (on phone lines) at 10 MHz, but at 10 GHz every little bump is a place where signal gets reflected back instead of moving toward the receiver. Lots of tricks to get around that, and occasionally, we get lucky and get a pristine wire pair and perfect conditions. ... but it's tough. -- Ned |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.