Message boards :
Technical News :
Bit Ceiling (Oct 08 2008)
Message board moderation
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
Some nagging network issues, mostly due to the known liabilities/usual suspects. Very often we are maxing out our 100Mbit private connection to the world, due to peak periods (catchup after an outage, a spate of "fast" workunits, new client downloads added to the usual set of workunit downloads) or sending our raw data to offsite archival storage. This is why download/upload rates are abysmally slow at times - if you can get through at all. One solution would be to increase our bandwidth - we do pay for a 1Gbit connection, but due to campus infrastructure can only use 100Mbit of that. Getting campus to improve this infrastructure is currently prohibitive due to cost (which includes new routers and dragging new cables up a mountainside to our lab), bureaucratic red tape, and the backlog of higher priority networking tasks campus wishes to tackle first. In other words as far as I can tell it ain't never gonna happen. Another solution would be to reduce our result redundancy, as already discussed in recent threads. We also had our science db/raw data storage server choke a bit today - perhaps because of the recent swarm of fast workunits and therefore increased demand on the splitters. We do try to randomize the data to prevent such swarms but you can't win all of the time. And our web log rotating script sometimes barfs for one reason or another and fails to restart one server or another. For a moment there both the scheduler and upload server were off - I caught it fairly quickly though and restarted them. To clarify, result uploads and workunit downloads go over the private SETI net, along with scheduler traffic. Web pages and other stuff goes over the campus net (it's not that much - only a couple Mbits/sec at peak times). The archival storage (where we copy all our raw data offsite) sometimes goes over the campus net, sometimes over the private SETI net, sometimes both if we need to empty the disks as fast as possible to return to Arecibo. Other than all that.. I fixed the fonts of the status pages, and Jeff elsewhere posted a quick note about NTPCker progress. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Blurf Send message Joined: 2 Sep 06 Posts: 8962 Credit: 12,678,685 RAC: 0 |
As always, thanks Matt. |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
. . .Thank You for the Updates Matt (and also to Jeff for the NTPCker progress) BOINC Wiki . . . Science Status Page . . . |
Stephen Send message Joined: 1 Sep 06 Posts: 103 Credit: 11,155,194 RAC: 0 |
Keep up the great work, I know you guys do the best with what you have. That in mind, what is the cost difference between the 1Gbit and the 100Mbit? If you're paying for 1Gbit and not getting it, maybe you could shift some resources around to help in other areas. |
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
Keep up the great work, I know you guys do the best with what you have. That in mind, what is the cost difference between the 1Gbit and the 100Mbit? If you're paying for 1Gbit and not getting it, maybe you could shift some resources around to help in other areas. As I remember, the Gigabit deal they signed up for is cheaper than the 100Mbit deal they had earlier. |
Stephen Send message Joined: 1 Sep 06 Posts: 103 Credit: 11,155,194 RAC: 0 |
As I remember, the Gigabit deal they signed up for is cheaper than the 100Mbit deal they had earlier. Thanks, I guess I missed out on that tidbit of info. |
W-K 666 Send message Joined: 18 May 99 Posts: 19013 Credit: 40,757,560 RAC: 67 |
Is there any spare cabling between the Seti download and upload servers and the hurricane interface? i.e. would this be feasable; download1_____100Mb____________ [ r ] [ o ] download2_____100Mb ___________ [ u ]______Hurricane_1Gb [ t ] Upload________100Mb ___________ [ e ] [ r ] |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14649 Credit: 200,643,578 RAC: 874 |
No, the Hurricane interface is at the bottom of the hill, and the servers are at the top of the hill. There is only one available cable (actually, fibre) between the top and bottom of the hill, and it's that fibre which is the problem. Sounds like a job for a SETI volunteer work-camp...... |
Claggy Send message Joined: 5 Jul 99 Posts: 4654 Credit: 47,537,079 RAC: 4 |
No, the Hurricane interface is at the bottom of the hill, and the servers are at the top of the hill. There is only one available cable (actually, fibre) between the top and bottom of the hill, and it's that fibre which is the problem. I was thinking about that yesterday, we just need to arrange enough local volunteers with enough spades!! (or a digger) Claggy. |
vandehoef Send message Joined: 27 Dec 00 Posts: 1 Credit: 6,850,702 RAC: 2 |
In the spirit of Grid Computing, perhaps the Grid is not doing enough in this SETI model. The more nodes that come on to SETI the bigger the burden on the central core ... network pipe not big enough, too many validations required, indices required to sort out huge databases of information. Perhaps you can distribute some work done centrally to the Grid .. beyond the calculations. Also, are you compressing data before it hits the network. Perhaps you can put a server down the mountain that sits on the 1Gig interface and filter what goes back over the 100M up the Mountain. (my apologies if I am covering old ground). Keep up the good work! |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14649 Credit: 200,643,578 RAC: 874 |
No, the Hurricane interface is at the bottom of the hill, and the servers are at the top of the hill. There is only one available cable (actually, fibre) between the top and bottom of the hill, and it's that fibre which is the problem. I don't see why it has to be restricted to local volunteers - I'm sure there are people from other parts of the States, even overseas, who'd be interested in a trip to visit the sharp end, and pay for their guided tour of the lab with a bit of labour. Excess baggage charges on the digger might be a bit of a problem, though. |
Jon Golding Send message Joined: 20 Apr 00 Posts: 105 Credit: 841,861 RAC: 0 |
I have no idea about the feasibility of this, but one way to avoid having to lay another cable between the lab and the Hurricane router might be to make the link wirelessly using WiFi or WiMAX. Some of this new technology claims to achieve speeds similar to fibre. The publicity they'd get, for being an integral part of the World's biggest computing project, might even persuade a supplier to set up and run a wireless link for "free". |
kittyman Send message Joined: 9 Jul 00 Posts: 51468 Credit: 1,018,363,574 RAC: 1,004 |
I have no idea about the feasibility of this, but one way to avoid having to lay another cable between the lab and the Hurricane router might be to make the link wirelessly using WiFi or WiMAX. Some of this new technology claims to achieve speeds similar to fibre. The publicity they'd get, for being an integral part of the World's biggest computing project, might even persuade a supplier to set up and run a wireless link for "free". Along the same lines........ I would think there would be wireless point-to-point links available that would cost far less than physically running new fibre up the mountain.... "Freedom is just Chaos, with better lighting." Alan Dean Foster |
speedimic Send message Joined: 28 Sep 02 Posts: 362 Credit: 16,590,653 RAC: 0 |
Sounds like a job for a SETI volunteer work-camp...... I'm in. mic. |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
. . . . somebody should create a Special Forum for a 'call-outs' and Volunteers Needed Thread etc ,on these Boards What say all - Eric K. (i know your busy ;) plenty of people want to Volunteer . . . so, . . . BOINC Wiki . . . Science Status Page . . . |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
I've advocated for a more grid/cloud-like environment before and am glad to see at least someone out there (vandehof) seems to have the same idea. Before getting the picks, shovels, bureaucrats, and lawyers out to rebuild Berkeley, why not think about re-structuring the project a 'tiny' bit to permit another institution, geographically removed, to provide some of the backbone server capacity. Since more than 50% of the project credits derive from North America, but Europe has a strong participation, I suggest some place on the US East or Texas, obviously where the bandwidth is strong. I'm not so simple as to say this would be 'easy', but sharing the science in a more advanced way has to be worth the effort. We could call it DT. (Distributed Thinking!) Speaking of DT's, its time for my first beer of the day. Cheers. |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
I've advocated for a more grid/cloud-like environment before and am glad to see at least someone out there (vandehof) seems to have the same idea. . . . somebody @ one of the North Carolina Universities (Raleigh etc) could implement a plan - strike a Proposal ? BOINC Wiki . . . Science Status Page . . . |
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
First off, there is already a "help wanted" section of the forum which we set up a while ago - the problem is coming up with (a) problems that volunteers could effectively help with, and (b) time to manage potential solutions. We've been remiss in adding topics to that forum, mostly due to (b). Secondly, we actually did have a microwave link way back in the day. Well, not us, but another project I worked for here at the lab which once had a NASA grant of sufficient worth to temporarily occupy a building off campus. So, strangely enough, we'd get bits up the hill to the space lab, and then transmit them through the air back down the hill to reach this other building. It was a 1Mbit link, roughly, if I remember correctly. This was circa 1995. That said, our current private SETI ISP - Hurricane Electric (a.k.a "HE") - is not directly connected to anything on campus. In fact, the whole thing is a marriage of many parties. We connect to HE down at the PAIX (major exchange in Palo Alto). But to do so we needed space/access at the PAIX. This is where the generous support and donated routers (and rack space) from Bill Woodcock (of Packet Clearing House fame) comes in. He also walked me and Jeff through using IOS to get all this to connect properly. Anyway, bits go through the PAIX to CENIC, which is where they are tunneled (thanks to much help/support from central campus) over an extra 100Mbit cable up to our SETI router in our closet. That's the "simple version" of what's going on. Many parties are involved, all with varied hectic schedules and different priorities. Getting a better link from here to central campus is but one part of the puzzle. (So yeah.. when people on campus running BOINC download a workunit from our servers, their requests go all the way down to Palo Alto first and then back to us.) To clarify another point - we are paying a tiny fraction for the current 1Gbit than we were paying for the previous 100Mbit connection. The only real benefit of the latter was that, if I remember correctly, they had a connection to CENIC, so the previously mentioned chaos wasn't necessary (though there was still some minor chaos which we coordinated with campus). And Finally! The problem with relocating parts of our infrastructure elsewhere is that it's all or nothing - every backend service is extremely tightly coupled with each other. The science database, the BOINC database, the upload/download servers and their respective storage, the splitter servers, the websites... etc. etc. etc. We're experiencing dropouts between servers that are in our lab and the servers in our closet across the hall (which talk over a single dedicated switch as opposed to through our lab infrastructure). - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Dr. C.E.T.I. Send message Joined: 29 Feb 00 Posts: 16019 Credit: 794,685 RAC: 0 |
. . . Understood Matt - Thanks for the Clarification re: And Finally! The problem with relocating parts of our infrastructure elsewhere is that it's all or nothing - every backend service is extremely tightly coupled with each other. The science database, the BOINC database, the upload/download servers and their respective storage, the splitter servers, the websites... etc. etc. etc. We're experiencing dropouts between servers that are in our lab and the servers in our closet across the hall (which talk over a single dedicated switch as opposed to through our lab infrastructure). BOINC Wiki . . . Science Status Page . . . |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14649 Credit: 200,643,578 RAC: 874 |
... It was a 1Mbit link, roughly, if I remember correctly. This was circa 1995 ... So Moore's Law should get us up to a gigabit by about 2010? That sounds worth exploring - though I still like the sound of a volunteers' workcamp. Thanks for the detailled explanation, Matt. I'm sure it will stimulate us all to get our thinking caps on and see if we can come up with a solution for you. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.