Busy Bytes (Jul 06 2009)


log in

Advanced search

Message boards : Technical News : Busy Bytes (Jul 06 2009)

Previous · 1 · 2 · 3 · 4 · 5 · Next
Author Message
PhonAcq
Send message
Joined: 14 Apr 01
Posts: 1622
Credit: 22,376,557
RAC: 6,121
United States
Message 915438 - Posted: 7 Jul 2009, 22:29:32 UTC - in response to Message 915345.

As a scientific project the scientists in charge must keep extremely tight control of the data in order to preserve it's validity. It's not likely that they would move data to servers outside of their direct control because of this.

Even the data we are given is checked by a wingman before it's loaded into the data base. Constant checks and controls have to be maintained to maintain the validity of the science.


Strange perspective. Providing a 'copy' is not the same as giving up the 'original'. In this project, any positive results will quickly be scrutinized for validity, so the risk of distributing the data is minimal.

The wingman comment is actually the same as the quota debate that has raged on and off over the years on these messageboards. The resolution of the debate depends entirely on your aptitude and comfort for taking risks. (I.e. no resolution)

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915508 - Posted: 7 Jul 2009, 23:53:47 UTC - in response to Message 915270.

It's not possible to make it wireless?

It is.


And.. why they don't do it? ;-)

I guess it would be a lot cheaper than with cable..

There are several here who know RF, know networking, hold various certifications and have decades of experience.

What we don't know is:

1) Is there a good clear line of sight? (some have said "trees" which may or may not be a "no")

2) What is the RF environment like (mostly noise floor -- what other users are in various bands).

For gigabit data rates, you can't really get by on lower frequencies. As you move up, radio starts behaving less like radio and more like light (bends less, more easily blocked or absorbed).

... and the equipment gets more expensive.

For something this important, I'd want licensed spectrum: unlicensed equipment has to accept interference from other users in the band (and sometimes, has to shut down if the band is shared with licensed users).

You also want enough "link budget" to deal with weather. We'd all be complaining (loudly) if the link was down when it rained (not that it rains that much).

So, it might be $40,000 instead of $80,000 for the fiber.

... and the fiber, once installed, will work perfectly no matter what the weather.

But none of this matters if the people who have to maintain it (the Campus Telecom/Networking group) says "no."
____________

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12745
Credit: 7,288,755
RAC: 18,015
United States
Message 915536 - Posted: 8 Jul 2009, 0:42:52 UTC - in response to Message 915508.

It's not possible to make it wireless?

It is.


And.. why they don't do it? ;-)

I guess it would be a lot cheaper than with cable..

There are several here who know RF, know networking, hold various certifications and have decades of experience.

What we don't know is:

1) Is there a good clear line of sight? (some have said "trees" which may or may not be a "no")

2) What is the RF environment like (mostly noise floor -- what other users are in various bands).

For gigabit data rates, you can't really get by on lower frequencies. As you move up, radio starts behaving less like radio and more like light (bends less, more easily blocked or absorbed).

... and the equipment gets more expensive.

For something this important, I'd want licensed spectrum: unlicensed equipment has to accept interference from other users in the band (and sometimes, has to shut down if the band is shared with licensed users).

You also want enough "link budget" to deal with weather. We'd all be complaining (loudly) if the link was down when it rained (not that it rains that much).

IIRC their ISP's POP is across the bay in San Jose. Not to long a shot, but through fog most of the year which is just as bad as rain in signal attenuation. The real question is about the height of the ISP's building if there are buildings in the line of sight between them and the bay. The photo below from the building patio pretty much clears the local path from the bay to SSL, at least towards San Francisco, San Jose is off the left of the frame.

So, it might be $40,000 instead of $80,000 for the fiber.

... and the fiber, once installed, will work perfectly no matter what the weather.

But none of this matters if the people who have to maintain it (the Campus Telecom/Networking group) says "no."

You are assuming the link would be maintained by campus. Unless there is politics requiring it to be with campus SETI@Home could maintain it on their end. No cross link to campus net at all. Then only permission to mount the antenna mast to the building and for the cable races in the building are needed. Perhaps easier or harder to get than permission to light up the fiber that already exists between the server closet and campus IT.

Cost of course much higher than using something that is already in place. Politics of sysadmins I believe was the stumbling block not the couple of grand cost for the interfaces and routers.
____________

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915587 - Posted: 8 Jul 2009, 3:01:31 UTC - in response to Message 915536.
Last modified: 8 Jul 2009, 3:04:22 UTC

You are assuming the link would be maintained by campus. Unless there is politics requiring it to be with campus SETI@Home could maintain it on their end. No cross link to campus net at all. Then only permission to mount the antenna mast to the building and for the cable races in the building are needed. Perhaps easier or harder to get than permission to light up the fiber that already exists between the server closet and campus IT.

I'm assuming that when Matt wrote about it before, and he said that the link must be maintained by CNS that there really wasn't another option.

... and you're assuming that, if it isn't too far away, they'd be able to get roof rights at PAIX.

I have friends who work in that area: tower rights, roof rights and "vertical real estate" and the rules and permissions can get baroque.

Edit: since it has been stated that this has to come through the Campus network services, and it's easier to maintain a shorter link than a really long link, that's why I'd try for both ends of the link on-campus.
____________

Profile Gary CharpentierProject donor
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 12745
Credit: 7,288,755
RAC: 18,015
United States
Message 915613 - Posted: 8 Jul 2009, 5:23:26 UTC - in response to Message 915587.

You are assuming the link would be maintained by campus. Unless there is politics requiring it to be with campus SETI@Home could maintain it on their end. No cross link to campus net at all. Then only permission to mount the antenna mast to the building and for the cable races in the building are needed. Perhaps easier or harder to get than permission to light up the fiber that already exists between the server closet and campus IT.

I'm assuming that when Matt wrote about it before, and he said that the link must be maintained by CNS that there really wasn't another option.

... and you're assuming that, if it isn't too far away, they'd be able to get roof rights at PAIX.

I have friends who work in that area: tower rights, roof rights and "vertical real estate" and the rules and permissions can get baroque.

Edit: since it has been stated that this has to come through the Campus network services, and it's easier to maintain a shorter link than a really long link, that's why I'd try for both ends of the link on-campus.

If it must be on campus, why are we talking about microwave at all? Matt has already stated a fiber has been pulled from IT up the hill into the server closet and the fiber is dark, just waiting for a use. If it can't be used then the same politics that prevents its use will also prevent any other on campus path from being used. The only other option is to bypass the politics -- bypass the campus IT.

As for PAIX: "Roof Rights: Roof rights available with satellite space" Yes it will cost $$$. Likely the project URL would also have to change.

Sometimes you have to think outside the box to get the box to cooperate.

____________

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915623 - Posted: 8 Jul 2009, 6:25:49 UTC - in response to Message 915613.

You are assuming the link would be maintained by campus. Unless there is politics requiring it to be with campus SETI@Home could maintain it on their end. No cross link to campus net at all. Then only permission to mount the antenna mast to the building and for the cable races in the building are needed. Perhaps easier or harder to get than permission to light up the fiber that already exists between the server closet and campus IT.

I'm assuming that when Matt wrote about it before, and he said that the link must be maintained by CNS that there really wasn't another option.

... and you're assuming that, if it isn't too far away, they'd be able to get roof rights at PAIX.

I have friends who work in that area: tower rights, roof rights and "vertical real estate" and the rules and permissions can get baroque.

Edit: since it has been stated that this has to come through the Campus network services, and it's easier to maintain a shorter link than a really long link, that's why I'd try for both ends of the link on-campus.

If it must be on campus, why are we talking about microwave at all? Matt has already stated a fiber has been pulled from IT up the hill into the server closet and the fiber is dark, just waiting for a use. If it can't be used then the same politics that prevents its use will also prevent any other on campus path from being used. The only other option is to bypass the politics -- bypass the campus IT.

As for PAIX: "Roof Rights: Roof rights available with satellite space" Yes it will cost $$$. Likely the project URL would also have to change.

Sometimes you have to think outside the box to get the box to cooperate.

I've not seen the statement that there is dark fiber to the building (or the closet). My recollection is that there is no fiber.

Not sure why URLs/Domain Names would have to change.

One of the issues SETI@Home has had is money. If they have to pay for roof-rights (monthly) then they either have to get Campus to pay for it, or find a stable source of funding.

My belief (based on a number of posts by SETI@Home staff) is that getting a gigabit to the lab would happen reasonably quickly if SETI@Home could come up with the bucks.
____________

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24694
Credit: 522,659
RAC: 19
United States
Message 915666 - Posted: 8 Jul 2009, 11:16:56 UTC - in response to Message 915623.

You are assuming the link would be maintained by campus. Unless there is politics requiring it to be with campus SETI@Home could maintain it on their end. No cross link to campus net at all. Then only permission to mount the antenna mast to the building and for the cable races in the building are needed. Perhaps easier or harder to get than permission to light up the fiber that already exists between the server closet and campus IT.

I'm assuming that when Matt wrote about it before, and he said that the link must be maintained by CNS that there really wasn't another option.

... and you're assuming that, if it isn't too far away, they'd be able to get roof rights at PAIX.

I have friends who work in that area: tower rights, roof rights and "vertical real estate" and the rules and permissions can get baroque.

Edit: since it has been stated that this has to come through the Campus network services, and it's easier to maintain a shorter link than a really long link, that's why I'd try for both ends of the link on-campus.

If it must be on campus, why are we talking about microwave at all? Matt has already stated a fiber has been pulled from IT up the hill into the server closet and the fiber is dark, just waiting for a use. If it can't be used then the same politics that prevents its use will also prevent any other on campus path from being used. The only other option is to bypass the politics -- bypass the campus IT.

As for PAIX: "Roof Rights: Roof rights available with satellite space" Yes it will cost $$$. Likely the project URL would also have to change.

Sometimes you have to think outside the box to get the box to cooperate.

I've not seen the statement that there is dark fiber to the building (or the closet). My recollection is that there is no fiber.

Not sure why URLs/Domain Names would have to change.

One of the issues SETI@Home has had is money. If they have to pay for roof-rights (monthly) then they either have to get Campus to pay for it, or find a stable source of funding.

My belief (based on a number of posts by SETI@Home staff) is that getting a gigabit to the lab would happen reasonably quickly if SETI@Home could come up with the bucks.

I believe I heard the same thing - $100,000.
____________


BOINC WIKI

kaydavis
Send message
Joined: 15 May 99
Posts: 10
Credit: 1,023,550
RAC: 0
Canada
Message 915717 - Posted: 8 Jul 2009, 15:35:17 UTC

Ummmm there is one possibility to add a 1gb link, that would not be as secure against damage as one run trough a underground conduit... and it's a miserable job to install, but one way to run cables using sealed cables is trough the sewer system. It is sometimes used in Cities where digging up the street is not an option..... the cost is slightly higher than standard outdoor cable and the run may be slightly longer and you have to have a plumber change the last clean out in the building to a special one with a sealed cable pass trough, but it's all standard technology. The only risk is if someone digs up the sewer line (which also exists with a conduit) or runs a cleaning cutter trough the serer line, which is usually avoided by tagging the cable where it enters the sewage system as well as notices installed right under the manhole covers so when one is opened it is immediately visible.

and no I'm not kidding
Kay

____________

Profile Robi
Send message
Joined: 24 Oct 00
Posts: 33
Credit: 288,394
RAC: 100
United States
Message 915723 - Posted: 8 Jul 2009, 15:51:47 UTC - in response to Message 915259.

It's not possible to make it wireless?

It is.


Ah, but
one (1) of those antennae costs around $40'000 and you need 2.

And there goes the $80'000 - $100'000 price tag again.

____________
Robi

kaydavis
Send message
Joined: 15 May 99
Posts: 10
Credit: 1,023,550
RAC: 0
Canada
Message 915737 - Posted: 8 Jul 2009, 16:30:53 UTC

Actually one other thing, maybe Matt or Ozzy should do a sticky thread called "The fiber optic thread" this way the topic won't be rehashed every 3 weeks in a new thread and people can read what has already be said before posting

Kay
____________

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8499
Credit: 4,194,256
RAC: 1,721
United Kingdom
Message 915744 - Posted: 8 Jul 2009, 16:54:54 UTC - in response to Message 915737.

Actually one other thing, maybe Matt or Ozzy should do a sticky thread called "The fiber optic thread" this way the topic won't be rehashed every 3 weeks in a new thread and people can read what has already be said before posting

OK..., so why can't two 1.5Gb bidirectional fibre modems be plugged into the ends of the existing fibre?

The existing site users get a nice x5 speedup over their present non-existent 100Mb max link that s@h has squashed them off, and s@h gets a cool unadulterated 1Gb...

Yes?

(I'm sure that more than a few of us would be happy to take a few days holiday to go and visit CNS if needed... ;-) )

Regards,
Martin

____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915755 - Posted: 8 Jul 2009, 17:23:09 UTC - in response to Message 915744.

(I'm sure that more than a few of us would be happy to take a few days holiday to go and visit CNS if needed... ;-)

Turn this around for just a second.

If one of us showed up, with all the tools and all the qualifications, and offered to do some job for CNS, for free, and then leave, wouldn't CNS need to have someone make sure we did the work to CNS specifications, so they could maintain it afterwards?

Would you let me come into your shop and "fix things" on a volunteer basis?

I'm not being critical, I'm just trying to be realistic. If I show up to help, and then leave, someone has to pick up the ball from there, and keep things working.

____________

Profile Robi
Send message
Joined: 24 Oct 00
Posts: 33
Credit: 288,394
RAC: 100
United States
Message 915770 - Posted: 8 Jul 2009, 17:47:51 UTC - in response to Message 915755.
Last modified: 8 Jul 2009, 17:48:38 UTC

(I'm sure that more than a few of us would be happy to take a few days holiday to go and visit CNS if needed... ;-)

Turn this around for just a second.

If one of us showed up, with all the tools and all the qualifications, and offered to do some job for CNS, for free, and then leave, wouldn't CNS need to have someone make sure we did the work to CNS specifications, so they could maintain it afterwards?

Would you let me come into your shop and "fix things" on a volunteer basis?

I'm not being critical, I'm just trying to be realistic. If I show up to help, and then leave, someone has to pick up the ball from there, and keep things working.


I'm pretty sure if a professional (or a few) would actually jump in and get in touch with "shop" (CNS) saying I've done this or run into this, let me help along with one of your staff, as a "shop owner" I would say Greatly appreciated, but I can't pay you. In the end, if the help works, the staff member would have learned something, otherwise, if it doesn't work, nothing was lost except the staff member's time and the pride of the professional ;)
____________
Robi

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5868
Credit: 60,610,567
RAC: 47,415
Australia
Message 915774 - Posted: 8 Jul 2009, 17:53:47 UTC - in response to Message 915723.
Last modified: 8 Jul 2009, 17:55:04 UTC

It's not possible to make it wireless?

It is.


Ah, but
one (1) of those antennae costs around $40'000 and you need 2.

And there goes the $80'000 - $100'000 price tag again.

That was the absolute top end model, good for 10km. They have several other models good for up to a couple of km which i suspect would be considerably cheaper.
____________
Grant
Darwin NT.

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 915793 - Posted: 8 Jul 2009, 18:53:35 UTC - in response to Message 915770.

(I'm sure that more than a few of us would be happy to take a few days holiday to go and visit CNS if needed... ;-)

Turn this around for just a second.

If one of us showed up, with all the tools and all the qualifications, and offered to do some job for CNS, for free, and then leave, wouldn't CNS need to have someone make sure we did the work to CNS specifications, so they could maintain it afterwards?

Would you let me come into your shop and "fix things" on a volunteer basis?

I'm not being critical, I'm just trying to be realistic. If I show up to help, and then leave, someone has to pick up the ball from there, and keep things working.


I'm pretty sure if a professional (or a few) would actually jump in and get in touch with "shop" (CNS) saying I've done this or run into this, let me help along with one of your staff, as a "shop owner" I would say Greatly appreciated, but I can't pay you. In the end, if the help works, the staff member would have learned something, otherwise, if it doesn't work, nothing was lost except the staff member's time and the pride of the professional ;)

That sounds great.

So, let's say I show up to put in the fiber. I've got all the qualifications, and they let me do the job.

It's in, they test, accept the fiber, and move SETI@Home to it.

Then, it breaks.

They start looking at why, and they're having trouble getting it fixed because no one watched to see what I was doing, or I forgot to tell them the specifications on the equipment I supplied, or I put an amplifier in the middle that they don't know about.

Now, you have a project that's down, angry crunchers, and an all-hands operation at CNS to figure out what the hell I did, and what to do to make it play.

The skill and knowledge to maintain the fiber is as important as the installation, and if they don't have someone to follow me around and make sure they know (someone who could probably do it just as fast) then there is some risk involved -- and it may not be worth it.

Especially with the way emotions tend to run hot around here.

Even "free" resources come with a cost, I'm afraid.
____________

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8499
Credit: 4,194,256
RAC: 1,721
United Kingdom
Message 915805 - Posted: 8 Jul 2009, 19:22:14 UTC - in response to Message 915793.
Last modified: 8 Jul 2009, 19:22:55 UTC

... CNS ... Even "free" resources come with a cost, I'm afraid.

First step is for the more knowledgeable folk to offer a parts list and costs so that CNS can cost up for what they can or want to do. At least that might spur some options.

My understanding from previous posts is that there is grey optical fibre already in place.

So... Assume the losses and dispersion of some grotty old length of 2km. What is possible?


Regards,
Martin
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Profile [seti.international] Dirk SadowskiProject donor
Volunteer tester
Avatar
Send message
Joined: 6 Apr 07
Posts: 7101
Credit: 60,938,040
RAC: 17,240
Germany
Message 915836 - Posted: 8 Jul 2009, 20:49:47 UTC

I got a PM from Enrico Boccuni, because he would like to post the following message, but he can't because of a too low RAC.


Hi to all, i've a little idea that can give back to you some Mbits.
Can the programmer of the project add a p2p bittorent similar function?
In this case if you create some rules about the redistribution of the wu based on a credit or the reputation system said by John McLeod VII, you can save really some Mbits and you can have a parallel network always up.
Thanks for your attention.
Bye



____________
BR

SETI@home Needs your Help ... $10 & U get a Star!

Team seti.international

Das Deutsche Cafe. The German Cafe.

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13625
Credit: 31,060,621
RAC: 20,628
United States
Message 915840 - Posted: 8 Jul 2009, 20:54:29 UTC - in response to Message 915836.

I got a PM from Enrico Boccuni, because he would like to post the following message, but he can't because of a too low RAC.


Hi to all, i've a little idea that can give back to you some Mbits.
Can the programmer of the project add a p2p bittorent similar function?
In this case if you create some rules about the redistribution of the wu based on a credit or the reputation system said by John McLeod VII, you can save really some Mbits and you can have a parallel network always up.
Thanks for your attention.
Bye


This idea has been suggested before, and the problem with the idea is that eventually all the bits must flow from and return to the Space Sciences Lab (SSL). Things like bittorrent only work when you only need to upload a single copy for all to share (like a document or a media file).
____________

Profile S@NL - Eesger - www.knoop.nl
Avatar
Send message
Joined: 7 Oct 01
Posts: 384
Credit: 37,204,728
RAC: 14,319
Netherlands
Message 916143 - Posted: 9 Jul 2009, 13:25:17 UTC
Last modified: 9 Jul 2009, 13:29:50 UTC

But this idea does have potential with new versions.. like now with the version AP 5.05 the distribution of the software (which is one and the same for the standard application..) might flatten some bandwidth peaks..

And the torrent system could be (ab)used for moving the wu from the first downloader to the wingman.. This could (almost) split the bandwidth usage in half! but I have no idea if the torrent system can handle stuff like this.. (and the security/legal issues getting involved in making BOINC have a torrent component)
____________
The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS

DrFoo
Send message
Joined: 17 Jul 99
Posts: 26
Credit: 25,496,489
RAC: 27,425
United States
Message 916148 - Posted: 9 Jul 2009, 13:57:51 UTC

While yes, BitTorrent really isn't an option here since it only works in a one to many topology, I agree the idea of sending the data once to get it delivered to all the wingmen does have merrit. No it doesn't really solve the problem the way a gbit line would, but it would certainly help, essentially cutting the outgoing bandwidth in half.

As I understand things, the problem is getting the data to the ISP rather than getting it from there out to the crunchers, right? So how hard would it be to get a SETI box colo'd at the ISP? I expect this could be done with a very simple box just running a slightly tweaked version of squid.

I'm not sure that gains anything though. From the graphs I've seen the problem is greater on the incoming side than the outgoing. But frankly that makes no sense to me. Aren't results considerably smaller than workunits? Could it be that all this excess incoming traffic is just requests that are going unanswered? If so, then this idea might actually be worth something.
____________

Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Technical News : Busy Bytes (Jul 06 2009)

Copyright © 2014 University of California