Hot Day (Oct 02 2012)


log in

Advanced search

Message boards : Technical News : Hot Day (Oct 02 2012)

Previous · 1 · 2 · 3 · 4 · 5 · Next
Author Message
tbret
Volunteer tester
Avatar
Send message
Joined: 28 May 99
Posts: 2389
Credit: 164,146,939
RAC: 68,273
United States
Message 1290993 - Posted: 4 Oct 2012, 0:19:01 UTC
Last modified: 4 Oct 2012, 0:33:27 UTC

Matt,

Whatever you have done, however temporary the situation may be, thank you.

Edit: I still thank you, but I spoke too soon.

WinterKnight
Volunteer tester
Send message
Joined: 18 May 99
Posts: 8219
Credit: 21,785,757
RAC: 12,721
United Kingdom
Message 1291012 - Posted: 4 Oct 2012, 1:09:16 UTC

Uploads now working fine, BUT, now cannot report or request new work.

One computer run out of work and now switched off.

Profile tpl
Avatar
Send message
Joined: 12 Nov 03
Posts: 364
Credit: 191,477,166
RAC: 17,366
Germany
Message 1291116 - Posted: 4 Oct 2012, 9:52:21 UTC

Scheduler timeout and no work available....
UL ok....DL sometimes slow....must restart
DL often to become work....
Thats from here..... :-((
____________

Chris Campbell
Send message
Joined: 14 May 99
Posts: 17
Credit: 2,159,392
RAC: 865
United States
Message 1291514 - Posted: 5 Oct 2012, 5:13:22 UTC

Hi guys -

I waited three months before posting this :)

The "Tech News" page at this location:

http://setiathome.berkeley.edu/tech_news.php

... is linked from the home page and provides a view of the occasional reports from Matt et al, the same posts that lead off these threads.

Since at least July, that page has been slightly corrupted. It's like it's missing a header or something, as it is showing some sort of raw database dump.

- Chris

____________
Running S@H since Day 1 -- May 14th, 1999!

tbret
Volunteer tester
Avatar
Send message
Joined: 28 May 99
Posts: 2389
Credit: 164,146,939
RAC: 68,273
United States
Message 1291681 - Posted: 5 Oct 2012, 17:45:18 UTC

Matt -

Things are better; much better. Thank you.

We still have a nagging customary problem and, of course, out here we can't really tell why.

You asked about "weirdness" and here is some (you have to read this whole paragraph): If we close and restart the project, *sometimes* we get faster downloads for a few seconds before they slow to a trickle again. When we get the faster server, okay, we get the faster server, BUT that's not what I'm reporting. I'm saying that if we get the slow server it seems to "start fast" and "trails-off." If you interrupt that and get the slow server again it will "re-start faster" then slow to a trickle again. I don't know what could account for that behavior unless that server is having trouble loading the files it serves and what we are experiencing "out here" is really an I/O constraint "in there."

I don't know what I'm talking about and I admit it, but "a bandwidth problem" between the server and us doesn't make any sense as a reason for that behavior. If it were the connection to us the download shouldn't start or restart faster, right?

Of course, I'm blind to what you may be doing to throttle one server's access to the outside world on purpose.

Thanks for what you are doing. I can only imagine that all of this is a constant thorn in your side. I hope you can get a new worry-free and bulletproof solution in place before you leave us for a month and can enjoy your time away.

Wales_Family
Send message
Joined: 5 Sep 11
Posts: 3
Credit: 355,455
RAC: 451
New Zealand
Message 1291854 - Posted: 6 Oct 2012, 1:25:21 UTC

Hi Matt,

Someone's probably already invented this "wheel", but I can't seem to find references to it. Also, my knowledge of the back-end algorithms is nil, so I'm making a few assumptions about the Download Servers. Those caveats aside, here goes:

Individual work units are sent out multiple times for processing, traversing the campus 100Mbit link each time. The outbound bandwidth utilization of the campus 100Mbit link could be lowered by telehousing the Download Servers in the rack across campus where the 1Gigabit link terminates (might be politically impossible on campus), or at the ISP (might be expensive).

If the Download Servers have high bandwidth requirements to/from the other Project servers (or need to be backed up), Cache Servers could be telehoused instead.These Cache Servers would be requested by their respective Download Servers to pass Work Units to requesters (with a confirmation back to the DS). If a WU was not in cache, the DS would pass it through to the cache server. There would be a slight additional delay for uncached units, but cached units would not be delayed by having to use the campus 100Mbit link for subsequent transmits to requesters.

There would also need to be some sort of FIFO scavenging algorithm on a CS to maintain cache storage at sensible levels. As the data is always on the download servers, CS machines need not be expensive, nor would they need to be backed up.

Cheers,
Stephen

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1830
Credit: 7,522,509
RAC: 22,950
United States
Message 1291932 - Posted: 6 Oct 2012, 6:51:53 UTC

This may have been asked before, but why was the Hurricane Electric 1Gbit/sec line terminated down on the campus, instead of in the SSL, in the first place?
____________
.

Josef W. Segur
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4134
Credit: 1,003,611
RAC: 234
United States
Message 1292073 - Posted: 6 Oct 2012, 15:56:16 UTC - in response to Message 1291932.

This may have been asked before, but why was the Hurricane Electric 1Gbit/sec line terminated down on the campus, instead of in the SSL, in the first place?

Hurricane Electric is the Internet Service Provider, not hardware. The plan allows up to 1GB data rates at an annual cost of $12000 IIRC. Getting the data to PAIX so it can connect to H.E. is a separate issue.
Joe

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1830
Credit: 7,522,509
RAC: 22,950
United States
Message 1292121 - Posted: 6 Oct 2012, 17:48:22 UTC - in response to Message 1292073.
Last modified: 6 Oct 2012, 17:49:29 UTC

This may have been asked before, but why was the Hurricane Electric 1Gbit/sec line terminated down on the campus, instead of in the SSL, in the first place?

Hurricane Electric is the Internet Service Provider, not hardware. The plan allows up to 1GB data rates at an annual cost of $12000 IIRC. Getting the data to PAIX so it can connect to H.E. is a separate issue.
Joe


My question was: why is it down on campus, and not at the SSL? I know it's an ISP... another way of saying this is: Why does SETI need to use the 100Mbs line down to the campus, when the ISP's termination should have been on the hill all along?
____________
.

rob smith
Volunteer moderator
Send message
Joined: 7 Mar 03
Posts: 7667
Credit: 44,721,560
RAC: 74,891
United Kingdom
Message 1292140 - Posted: 6 Oct 2012, 18:25:30 UTC

I think you'll find the answer to be "University politics". And there is nothing logical about them.
____________
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?

Profile Gary Charpentier
Volunteer tester
Avatar
Send message
Joined: 25 Dec 00
Posts: 11732
Credit: 5,969,877
RAC: 0
United States
Message 1292422 - Posted: 7 Oct 2012, 16:04:10 UTC - in response to Message 1292121.

My question was: why is it down on campus, and not at the SSL? I know it's an ISP... another way of saying this is: Why does SETI need to use the 100Mbs line down to the campus, when the ISP's termination should have been on the hill all along?

The end of HE connection is actually in a building in Palo Alto, across the San Francisco bay from the campus. Seti uses the Campus ISP to get the data from there into the campus IT building. If you check the router pages you will see there is more than one link between the campus and Palo Alto.

As to the ISP link being at the SSL, well, Campus is the Landlord and you would need the permission of the Landlord to string any cable across his property. Also the ISP might want to be paid to string that cable. Giga connectivity isn't going to run on any existing cable. So we could be talking about stringing a undersea cable from Palo Alto to Berkeley. I suspect that might run into some dollars. Or perhaps they might be able to get permission to string a cable on the Bay Bridge from CalTrans. In any case this isn't something that can be a jury rig. So this is out.

Campus may be being charged by their ISP based on the total data on their link. Allowing SETI to go full bandwidth would change that charge. Obviously in this era of Government funding cutbacks, that would have to be run by the Board of Regents. I doubt they would agree to it unless they were paid back by SETI. As you know SETI is rather short of funds right now.

Frankly I'd much rather see work on Ntpckr than bigger bandwidth with the limited resources available.

____________

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 23702
Credit: 493,239
RAC: 133
United States
Message 1292658 - Posted: 8 Oct 2012, 2:28:31 UTC - in response to Message 1290694.

Now the UL are the problem, they take a long time to start then actualy goes fast, after that stuck at 100%, after some long time return with and error.

Try this in your cc_config.xml:

<cc_config>
<options>
<http_transfer_timeout>3000</http_transfer_timeout>
</options>
</cc_config>

____________


BOINC WIKI

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5561
Credit: 51,297,855
RAC: 39,553
Australia
Message 1292728 - Posted: 8 Oct 2012, 5:56:33 UTC - in response to Message 1292422.

Frankly I'd much rather see work on Ntpckr than bigger bandwidth with the limited resources available.

And i'd rather they sort out the bandwidth issue.
Here we are, most caches should be pretty much full, sod all shorites in the system, no AP going out, and still the network traffic is maxed out.

____________
Grant
Darwin NT.

Profile Slavac
Volunteer tester
Avatar
Send message
Joined: 27 Apr 11
Posts: 1932
Credit: 17,952,639
RAC: 0
United States
Message 1292882 - Posted: 8 Oct 2012, 19:00:24 UTC - in response to Message 1292728.

Frankly I'd much rather see work on Ntpckr than bigger bandwidth with the limited resources available.

And i'd rather they sort out the bandwidth issue.
Here we are, most caches should be pretty much full, sod all shorites in the system, no AP going out, and still the network traffic is maxed out.


Last time I talked with Jeff he mentioned that NTPCKR and RFI are almost always running however that doesn't typically show up on the server status page.
____________


Executive Director GPU Users Group Inc. -
brad@gpuug.org

Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1292933 - Posted: 8 Oct 2012, 21:00:42 UTC

Is hurricane electronic the only provider option ?

How much of that $12k does SAH pay for their 10% allotment?

How about a significant wireless link across the bay ...?




juan BFB
Volunteer tester
Avatar
Send message
Joined: 16 Mar 07
Posts: 4610
Credit: 232,718,976
RAC: 333,533
Brazil
Message 1292938 - Posted: 8 Oct 2012, 21:07:25 UTC - in response to Message 1292933.
Last modified: 8 Oct 2012, 21:08:02 UTC

Is hurricane electronic the only provider option ?

How much of that $12k does SAH pay for their 10% allotment?

How about a significant wireless link across the bay ...?


Probabily the wireless link will be to slow for the high demand of SETI but a couple of fast ADSL links will do the same work and sure will cost a fraction of that and requires no aditional wiring.
____________

Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1292951 - Posted: 8 Oct 2012, 21:22:52 UTC

Juan wrote:
Probabily the wireless link will be to slow for the high demand of SETI


To be more specific, a dedicated wireless link(or multiple links) , as in not on an independent provider.

example
HE.net termination to wireless link to wireless receiver at SSL building ... possibly one or two repeaters in-line.

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13307
Credit: 27,841,842
RAC: 15,861
United States
Message 1292952 - Posted: 8 Oct 2012, 21:25:18 UTC - in response to Message 1292951.

You would still need permission from the Regent of Berkeley to put up the repeaters, and likely the permission to use part of the wireless spectrum as such distances.

Then there's the problem of wireless latency and interference. Not sure that's a good idea.

Profile Tron
Send message
Joined: 16 Aug 09
Posts: 180
Credit: 2,236,055
RAC: 0
United States
Message 1292966 - Posted: 8 Oct 2012, 21:51:51 UTC - in response to Message 1292952.
Last modified: 8 Oct 2012, 22:00:46 UTC

You would still need permission from the Regent of Berkeley to put up the repeaters, and likely the permission to use part of the wireless spectrum as such distances.

Then there's the problem of wireless latency and interference. Not sure that's a good idea.


As a licensed HAM radio operator, I transmit minor amounts of data quite often with out any issue at all, over extraordinary distances, with the help of other HAM ops who act as repeaters.

The use of the wireless spectrum would be dictated by the FCC for mid-range data transmission.

latency is a hardware/load problem more common to multi user systems , consider that this system would be dedicated to SAH. And would logically also have a hard wired parity check for the transmitted/recieved data.


As for permission to put such a system in place , I can understand the red tape must be overwhelming. but SAH's usage of the campus network would be reduced to a tiny fraction ,as the only need for it would be the parity check I mentioned above.
Wouldn't that be enough justification alone ?

The antennas would probably be objectionable for cosmetic reasons unless there were already towers in place with space for rent.

The hardware requirements might be cost prohibitive as well. Though we've recently proven, with the right people in charge of donations and motivating said donations. This project can really get stuff done. ( /shameless plug for GPU users group < you folks are awesome! )

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13307
Credit: 27,841,842
RAC: 15,861
United States
Message 1292978 - Posted: 8 Oct 2012, 22:43:27 UTC - in response to Message 1292966.

I do believe the airspace is owned by Berkeley, and such radio transmissions may disrupt other buildings. Therefore I do believe this would still be an issue that needs the permission of the Regents of Berkeley so that it can be cleared whether those concerns are real.

Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : Technical News : Hot Day (Oct 02 2012)

Copyright © 2014 University of California