Gasping for Air (May 14 2007)

Message boards : Technical News : Gasping for Air (May 14 2007)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 3 · 4 · 5 · 6

AuthorMessage
Mithotar
Avatar

Send message
Joined: 11 Apr 01
Posts: 88
Credit: 66,037,385
RAC: 50
United States
Message 568309 - Posted: 16 May 2007, 4:19:42 UTC - in response to Message 568294.  

for the last 39 hours i have been getting the same message: 5/15/2007 10:27:23 PM||Project communication failed: attempting access to reference site
5/15/2007 10:27:24 PM||Access to reference site succeeded - project servers may be temporarily down.
5/15/2007 10:27:27 PM|SETI@home|Scheduler request failed: couldn't connect to server
5/15/2007 10:27:27 PM|SETI@home|Deferring scheduler requests for 1 minutes and 0 seconds
PLUS i have been getting this HTTP ERROR what gives my stats are droping not happy



WAH.......if you need stats to stay with Seti & BOINC maybe you should devote
your efforts elsewhere.......

Every one else's stats are in the dumper too.........

BUT if you want to help out a monumental research effort as part of a global team to benefit mankind then read learn and try to be more patient.......


ID: 568309 · Report as offensive
Profile Kenneth S Fees
Volunteer tester

Send message
Joined: 24 Jul 00
Posts: 3
Credit: 75,284,716
RAC: 57
United States
Message 568379 - Posted: 16 May 2007, 10:02:44 UTC

Thanks for all the hard work you have put in. I hope for you and your team that in the end this will leave you with a more stable system. Working with servers and networks on my submarine, I know how frustrating rebuilding and integrating modules can be.
ID: 568379 · Report as offensive
Profile Demiurg
Volunteer tester
Avatar

Send message
Joined: 2 Jul 02
Posts: 883
Credit: 28,286
RAC: 0
Sweden
Message 568387 - Posted: 16 May 2007, 10:26:58 UTC - in response to Message 568159.  


That said; after explaining my plight and what I've been seeing and experiencing via Time Warner/Road Runner and the vast changes at Berkeley switching from Cogent to Hurricane, my friend helped me to make some changes to the "hosts" file on Excalibur, (my Primary Cruncher), as well as changes to my Router, (changes that correlate to changes suggested by many of you here over the past few months that I couldn't make because they were "locked out" to me by the Router)

You may need to be careful about your HOSTS file.

What this does is completely replace DNS for names that are in the hosts file, so if Berkeley moves a machine to a new IP, your system(s) will not change.

I'd take out the entries in a few days, once things are stable and you can see what changes do.

Also, note that 4.2.2.1 and 4.2.2.2 belong to Verizon, if enough non-Verizon customers use those they'll probably configure them so they don't resolve outside their networks.

You could run your own name server....


I can just see a lot of fellow swedes and other euros doing the "4.2.2.1 and 4.2.2.2" switch and go poot! I would like to say here that this fix is a special for TL and should be avoided by everyone not using the same router and DNS carrier as him...
Carl
It is SEXY to DONATE!
Skype = demiurg2
ID: 568387 · Report as offensive
Profile TimeLord04
Volunteer tester
Avatar

Send message
Joined: 9 Mar 06
Posts: 21140
Credit: 33,933,039
RAC: 23
United States
Message 568421 - Posted: 16 May 2007, 12:00:09 UTC - in response to Message 568387.  
Last modified: 16 May 2007, 12:01:09 UTC


That said; after explaining my plight and what I've been seeing and experiencing via Time Warner/Road Runner and the vast changes at Berkeley switching from Cogent to Hurricane, my friend helped me to make some changes to the "hosts" file on Excalibur, (my Primary Cruncher), as well as changes to my Router, (changes that correlate to changes suggested by many of you here over the past few months that I couldn't make because they were "locked out" to me by the Router)

You may need to be careful about your HOSTS file.

What this does is completely replace DNS for names that are in the hosts file, so if Berkeley moves a machine to a new IP, your system(s) will not change.

I'd take out the entries in a few days, once things are stable and you can see what changes do.

Also, note that 4.2.2.1 and 4.2.2.2 belong to Verizon, if enough non-Verizon customers use those they'll probably configure them so they don't resolve outside their networks.

You could run your own name server....


I can just see a lot of fellow swedes and other euros doing the "4.2.2.1 and 4.2.2.2" switch and go poot! I would like to say here that this fix is a special for TL and should be avoided by everyone not using the same router and DNS carrier as him...
Carl



Ooops; sorry, Demiurg. I never even thought that anyone else would think to try the "4." addresses for their own DNS replacements/augments... I sincerely was responding to Ned and to give Matt some news about my ongoing issues with Time Warner and now the Berkeley DNS issues since switching from Cogent to Hurricane...

You are absolutely correct. This fix was presented to me by a friend who works for a DNS Software Company called Men and Mice. My friend has been in the DNS area professionally since 1999; and, actually has been dabbling with it and other areas of PCs and MACs for far longer. He is considered a DNS Expert. This fix is meant for me and my particular situation. Therefore, use the "4." at your own risk. I highly recommend not using them unless you clear them first with a DNS Expert of your own. Once again; sorry, I meant no harm honest.


Sincerely,


TimeLord04
Have TARDIS, will travel...
Come along K-9!
Join Calm Chaos
ID: 568421 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 568550 - Posted: 16 May 2007, 14:16:18 UTC - in response to Message 567960.  

Yes, it's not only your clients which can't upload.

No, I am not able to upload either, neither Seti main nor Seti Beta.

Hopefully the rebooting of Bruno will do the trick eventually...



SETI beta shows everything (other than the web page server and the scheduler) down or disconnected. Beta will probably come up once S@H is back online, and perking along nicely... (it's second priority at this time, understandably)
.

Hello, from Albany, CA!...
ID: 568550 · Report as offensive
Profile Demiurg
Volunteer tester
Avatar

Send message
Joined: 2 Jul 02
Posts: 883
Credit: 28,286
RAC: 0
Sweden
Message 568574 - Posted: 16 May 2007, 14:45:48 UTC - in response to Message 568421.  


That said; after explaining my plight and what I've been seeing and experiencing via Time Warner/Road Runner and the vast changes at Berkeley switching from Cogent to Hurricane, my friend helped me to make some changes to the "hosts" file on Excalibur, (my Primary Cruncher), as well as changes to my Router, (changes that correlate to changes suggested by many of you here over the past few months that I couldn't make because they were "locked out" to me by the Router)

You may need to be careful about your HOSTS file.

What this does is completely replace DNS for names that are in the hosts file, so if Berkeley moves a machine to a new IP, your system(s) will not change.

I'd take out the entries in a few days, once things are stable and you can see what changes do.

Also, note that 4.2.2.1 and 4.2.2.2 belong to Verizon, if enough non-Verizon customers use those they'll probably configure them so they don't resolve outside their networks.

You could run your own name server....


I can just see a lot of fellow swedes and other euros doing the "4.2.2.1 and 4.2.2.2" switch and go poot! I would like to say here that this fix is a special for TL and should be avoided by everyone not using the same router and DNS carrier as him...
Carl



Ooops; sorry, Demiurg. I never even thought that anyone else would think to try the "4." addresses for their own DNS replacements/augments... I sincerely was responding to Ned and to give Matt some news about my ongoing issues with Time Warner and now the Berkeley DNS issues since switching from Cogent to Hurricane...

You are absolutely correct. This fix was presented to me by a friend who works for a DNS Software Company called Men and Mice. My friend has been in the DNS area professionally since 1999; and, actually has been dabbling with it and other areas of PCs and MACs for far longer. He is considered a DNS Expert. This fix is meant for me and my particular situation. Therefore, use the "4." at your own risk. I highly recommend not using them unless you clear them first with a DNS Expert of your own. Once again; sorry, I meant no harm honest.
Sincerely,


I never thought you did:-)
But sometimes people try things when they are desperate, that's why I wrote.

Take care!
Carl
It is SEXY to DONATE!
Skype = demiurg2
ID: 568574 · Report as offensive
BarryAZ

Send message
Joined: 1 Apr 01
Posts: 2580
Credit: 16,982,517
RAC: 0
United States
Message 568584 - Posted: 16 May 2007, 14:52:23 UTC - in response to Message 568550.  

Newsflash -- they have a firm date on the resolution of the upload/download problem and the firm date is --- drum roll please --- sometime in the future.

It may well be that the outage inspired datastorm that we are now seeing is simply making the bandwidth constraint issue that surfaces weekly with a 4 hour outage (and that issue has lasted for between 4 and 18 hours), more of an obvious ISSUE which needs fixing. If the upload/download datastorm is something of an hour for hour (or more) problem proportional to the outage (as it is for the weekly backup), then we may be talking of an upload download problem that doesn't resolve until Memorial Day.

So, for now, suspend SETI and let the swamp drain without your contribution. The thing is, in terms of other projects out there, it is actually getting a bit thinner. BBC Climate is ending, Predictor appears to have some severe problems, the change in the work units at Einstein is causing a bunch of issues there and a number of other projects are either scaling down or in alpha/beta mode.



Yes, it's not only your clients which can't upload.

No, I am not able to upload either, neither Seti main nor Seti Beta.

Hopefully the rebooting of Bruno will do the trick eventually...




ID: 568584 · Report as offensive
Profile Demiurg
Volunteer tester
Avatar

Send message
Joined: 2 Jul 02
Posts: 883
Credit: 28,286
RAC: 0
Sweden
Message 568633 - Posted: 16 May 2007, 15:44:48 UTC - in response to Message 568584.  

Newsflash -- they have a firm date on the resolution of the upload/download problem and the firm date is --- drum roll please --- sometime in the future.

It may well be that the outage inspired datastorm that we are now seeing is simply making the bandwidth constraint issue that surfaces weekly with a 4 hour outage (and that issue has lasted for between 4 and 18 hours), more of an obvious ISSUE which needs fixing. If the upload/download datastorm is something of an hour for hour (or more) problem proportional to the outage (as it is for the weekly backup), then we may be talking of an upload download problem that doesn't resolve until Memorial Day.

So, for now, suspend SETI and let the swamp drain without your contribution. The thing is, in terms of other projects out there, it is actually getting a bit thinner. BBC Climate is ending, Predictor appears to have some severe problems, the change in the work units at Einstein is causing a bunch of issues there and a number of other projects are either scaling down or in alpha/beta mode.



Yes, it's not only your clients which can't upload.

No, I am not able to upload either, neither Seti main nor Seti Beta.

Hopefully the rebooting of Bruno will do the trick eventually...





Predictor is actually in a lot of trouble and may have to shut down due to the Scripps pulling the funding after some dubious ethical manouvering lately and threats to sue some of their crunchers that are very well-known and apreciated here at S@H.
So let us help our beloved "Big Crunching Daddy" (S@H) to get well as fast as we can. The best way to get things running is to clink on the link below;

It is SEXY to DONATE!
Skype = demiurg2
ID: 568633 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 568680 - Posted: 16 May 2007, 16:28:54 UTC - in response to Message 568387.  


That said; after explaining my plight and what I've been seeing and experiencing via Time Warner/Road Runner and the vast changes at Berkeley switching from Cogent to Hurricane, my friend helped me to make some changes to the "hosts" file on Excalibur, (my Primary Cruncher), as well as changes to my Router, (changes that correlate to changes suggested by many of you here over the past few months that I couldn't make because they were "locked out" to me by the Router)

You may need to be careful about your HOSTS file.

What this does is completely replace DNS for names that are in the hosts file, so if Berkeley moves a machine to a new IP, your system(s) will not change.

I'd take out the entries in a few days, once things are stable and you can see what changes do.

Also, note that 4.2.2.1 and 4.2.2.2 belong to Verizon, if enough non-Verizon customers use those they'll probably configure them so they don't resolve outside their networks.

You could run your own name server....


I can just see a lot of fellow swedes and other euros doing the "4.2.2.1 and 4.2.2.2" switch and go poot! I would like to say here that this fix is a special for TL and should be avoided by everyone not using the same router and DNS carrier as him...
Carl

I don't want to be critical of TL, as his friends are trying to help him.

As a service provider, when someone else starts using my DNS server as a resolver, I have a name for that: theft of service.

The reason I'm not criticizing TL is very simple: if Verizon/GTEI doesn't want non-customers using their resolvers, they need to secure them against abuse.

... and they have not done that. My resolver won't do full resolution for outsiders -- it has been secured against this kind of abuse.

TL should use his provider (Time Warner) or run his own resolver.

BIND works well, and is available for Windows as well as *nix.
ID: 568680 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 568692 - Posted: 16 May 2007, 16:44:16 UTC - in response to Message 568421.  
Last modified: 16 May 2007, 16:45:47 UTC


Ooops; sorry, Demiurg. I never even thought that anyone else would think to try the "4." addresses for their own DNS replacements/augments... I sincerely was responding to Ned and to give Matt some news about my ongoing issues with Time Warner and now the Berkeley DNS issues since switching from Cogent to Hurricane...

Before anyone touches the HOSTS file, they really should understand what it is.

The HOSTS file goes back to the earliest days of the internet, before DNS was introduced in 1987. If you had one of the few thousand net-connected hosts at that time, you downloaded a new HOSTS file every week or so, and that was your update.

As the net grew, it started taking too much bandwidth, and was too hard to maintain and distribute one file, so DNS was born.

20 years later, the HOSTS file still exists. It's still handy because it overrides DNS.

... and I think your Cogent vs. Hurricane issue might have something to do with your HOSTS file pointing to the old Cogent addresses.

You need to commit fully to the HOSTS file (and go back to TW DNS for everything else) or you need to edit out (or even erase) the HOSTS file.

-- Ned
ID: 568692 · Report as offensive
musikman38

Send message
Joined: 28 Feb 03
Posts: 2
Credit: 33,186
RAC: 0
United States
Message 568804 - Posted: 16 May 2007, 18:45:47 UTC

WAY TO GO you are back online
no more http error messages
ID: 568804 · Report as offensive
RSJR

Send message
Joined: 19 Jan 01
Posts: 12
Credit: 786,583
RAC: 0
United States
Message 568897 - Posted: 16 May 2007, 20:36:09 UTC - in response to Message 567949.  

Sounds like a "Round-robin" setup between two separate NICs attached to totally separate Internet connections for redundancy to me.

Bob

If you do a ping setiathome.berkeley.edu

I do find the server intermittently switching between

208.68.240.11
128.32.18.152

This should be resolved to let us connect/download/upload.

Greetings from Belgium,

HP ;-))



+ 6,198 classic Seti@Home Units (68,265 CPU Hours)
ID: 568897 · Report as offensive
Profile TimeLord04
Volunteer tester
Avatar

Send message
Joined: 9 Mar 06
Posts: 21140
Credit: 33,933,039
RAC: 23
United States
Message 568945 - Posted: 16 May 2007, 21:26:13 UTC - in response to Message 568692.  


Ooops; sorry, Demiurg. I never even thought that anyone else would think to try the "4." addresses for their own DNS replacements/augments... I sincerely was responding to Ned and to give Matt some news about my ongoing issues with Time Warner and now the Berkeley DNS issues since switching from Cogent to Hurricane...

Before anyone touches the HOSTS file, they really should understand what it is.

The HOSTS file goes back to the earliest days of the internet, before DNS was introduced in 1987. If you had one of the few thousand net-connected hosts at that time, you downloaded a new HOSTS file every week or so, and that was your update.

As the net grew, it started taking too much bandwidth, and was too hard to maintain and distribute one file, so DNS was born.

20 years later, the HOSTS file still exists. It's still handy because it overrides DNS.

... and I think your Cogent vs. Hurricane issue might have something to do with your HOSTS file pointing to the old Cogent addresses.

You need to commit fully to the HOSTS file (and go back to TW DNS for everything else) or you need to edit out (or even erase) the HOSTS file.

-- Ned



Actually; Ned, we used the Hurricane address in my Hosts file to completely override the prior Cogent address(es)... It seems that because of Time Warner's faulty DNS practices, (not keen on the details and my friend didn't go into the specifics), as quickly as every 30 Seconds I would see changes in Ping responses. That is, I typed in Command Prompt "Ping setiweb.ssl.berkeley.edu" and would get the Hurricane address. 30 Seconds later, I'd ping again and my PC would show the old Cogent address.

Just prior to these pings and witnessed rotations of IP Addresses my Browser would hang, (and funnily enough, only hang on the open SETI@Home and Beta/Astropulse Tabs), not knowing what to do with them. Eventually this hang would result in the Browser, (Firefox in my case, but it could happen on any browser), displaying the dreaded "Connection Timeout Error" Message. This is one of the things that's been happening to me since Time Warner took over Adelphia here in Southern California.

Time Warner's Customer Service plain SUCKS in getting a response in this matter. It was late in November that I finally got a Tier 3 Service Rep, (with his Supervisor standing over his shoulder telling him what to tell me), to admit that Time Warner indeed is having issues with their DNS Servers especially those in my area that connect me to the Los Angeles Trunk.

Again; while this issue alone is minor, I've had other major issues that have actually lead to full signal loss and thus zero internet connectivity which also takes down my VoIP Phone System that is through Vonage. So, now that my friend helped me to modify my Hosts file to lock out the Cogent address and see only the Hurricane address for both SETI@Home and Beta/Astropulse the Browser hangs are less frequent.

The use of the Verizon "4." DNS addressing, (per my friend's programming and in his words), "...is not theft, they purposely leave these addresses open." So, here I am with a more stable connection. Now doing an end run around Time Warner and so far so good.

Also, apparently my friend isn't the only one to utilize these open Verizon DNS links. My Vonage PAP-2 Device, (that replaces a Vonage Router I had in 2005), now connected to my NetGear Router was set by a Vonage Customer Service Rep who had me program in the "4.2.2.2" DNS Link into the PAP-2's DNS List last year. He didn't even tell me that it was a Verizon DNS Link. My friend did; and that was the first time I'd had knowledge about it.

So; paraphrasing your words, until and unless Verizon considers it an issue and secures these links, and combined with the apparent wide use of these presently open connections, then this is not theft by definition.

In the broader scheme of things; however, I do agree with your ideals. If these links had been hidden, (not been made public domain), or had they been secured but hacked into by outside sources - then yes, use of this to get around Time Warner's poor design would be theft.


Sincerely,


TimeLord04
Have TARDIS, will travel...
Come along K-9!
Join Calm Chaos
ID: 568945 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 568985 - Posted: 16 May 2007, 22:44:30 UTC - in response to Message 568945.  


The use of the Verizon "4." DNS addressing, (per my friend's programming and in his words), "...is not theft, they purposely leave these addresses open." So, here I am with a more stable connection. Now doing an end run around Time Warner and so far so good.

Sorry, but unless you are a Verizon customer, you are using Verizon resources without paying them.

But as you pointed out, it is Verizon's responsibility to secure their stuff.

It's like buying a fancy new mountain bike (titanium frame, trick suspension, etc.) and leaving it unlocked on your front lawn. Sure, the person who stole it broke the law, but what kind of an idiot am I if I didn't lock it up and put it in a safe place?

The danger is finding one day that Verizon woke up and secured their name servers. Whenever I do something like this, it seems to happen just after I forgot what I was doing.

If you have a small network, and you're running SETI all the time on at least one machine, what I'd do is run my own resolver. It isn't hard, instructions are here.

That way, it's your resources, and you can fix it if it breaks.

-- Ned

ID: 568985 · Report as offensive
netwraith

Send message
Joined: 15 Sep 06
Posts: 8
Credit: 61,290
RAC: 0
United States
Message 568986 - Posted: 16 May 2007, 22:47:14 UTC

--

After reading a few of these paragraphs I may have a suggestion or two.

One thing I read was that 'bruno' is dropping a lot of packets. If that is actually what is happening and not that he does not have enough horsepower to respond to thousands of SYN's at once, you should go looking for network issues. Now, I don't know what the specific configurations are, but, I will either make some assumptions or allow for a decision tree based on a couple of options. If 'bruno' is simply getting swamped by the Internet connected interface, you might look closely at the rate of the interface.

Many of my clients purchase Gigabit interfaces to achieve low latency without consideration of the application. Gigabit will help database connections between the database and web-servers due to the limited number of simultaneous connections, but, Gigabit will frequently hurt a web-server when used on the Internet interface.

Remember that ethernet is a variable length packet. It takes a lot of horsepower to parse the packet at Gigabit speeds, to say nothing about the increase in interupt overhead once it's parsed.

Internet connections tend to be different due to the fact that there can be thousands of packets a second hitting the interface. Dropping the connection to 100MBit won't do a lot for a really overloaded server, but, it will keep more packets from being lost and the resulting retransmission load. And if your connection to the Internet is less than a Gigabit, you really don't need a Gigabit interface on your web-server anyway. (There are a few engineering companies around that feel that Gigabit saturates at around 314MBits anyway.. This being a combination of variable frame 1500 byte packets and the interupt overhead (Ethernet and the 1500 byte packet were designed at 1 megabit!!)) If your network guys are up to it, you might consider an etherchannel for your Internet interface.. (Several ethernet ganged together). At any rate, dropping packets should not be tolerated... so get working on this...

As far as NFS being a choke point, you might consider an experiment. Compiling SMB/SAMBA/CIFS for both the server and client and doing a test may yield better throughput on your remote filesystem connections. This is not a promise, but, in more than a few cases, I have been able to greatly exceed NFS throughput with SAMBA. You could also try to solicit donations of newer Network Appliance NetFiler boxen... They might help your bottleneck as well.

If anyone would like to discuss on phone, please send me an email. I will pay the dime and call if needed...

ID: 568986 · Report as offensive
netwraith

Send message
Joined: 15 Sep 06
Posts: 8
Credit: 61,290
RAC: 0
United States
Message 568991 - Posted: 16 May 2007, 22:51:28 UTC - in response to Message 568985.  


The use of the Verizon "4." DNS addressing, (per my friend's programming and in his words), "...is not theft, they purposely leave these addresses open." So, here I am with a more stable connection. Now doing an end run around Time Warner and so far so good.

Sorry, but unless you are a Verizon customer, you are using Verizon resources without paying them.

But as you pointed out, it is Verizon's responsibility to secure their stuff.

It's like buying a fancy new mountain bike (titanium frame, trick suspension, etc.) and leaving it unlocked on your front lawn. Sure, the person who stole it broke the law, but what kind of an idiot am I if I didn't lock it up and put it in a safe place?

The danger is finding one day that Verizon woke up and secured their name servers. Whenever I do something like this, it seems to happen just after I forgot what I was doing.

If you have a small network, and you're running SETI all the time on at least one machine, what I'd do is run my own resolver. It isn't hard, instructions are here.

That way, it's your resources, and you can fix it if it breaks.

-- Ned



The 4. dns servers are public servers. It's part of the social contract that Internet was founded on. They are there to be used by anyone who wants to use them. I used to work on them when I worked for BBN Planet.. Trust me... you are not stealing services by using them to resolve names....


ID: 568991 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 569373 - Posted: 17 May 2007, 12:41:59 UTC - in response to Message 568986.  

Very good helpful comments there...

...There are a few engineering companies around that feel that Gigabit saturates at around 314MBits anyway.. This being a combination of variable frame 1500 byte packets and the interupt overhead (Ethernet and the 1500 byte packet were designed at 1 megabit!!)) If your network guys are up to it, you might consider an etherchannel for your Internet interface.. (Several ethernet ganged together). At any rate, dropping packets should not be tolerated... so get working on this...

A good fix for internal networks is to increase the maximum packet size up to whatever your systems can comfortably take or up to just larger than the usual transaction size. Hence, more data for fewer packets and less overhead. (This is no good for external to the internet!)

Eliminating all the TCP retries should give a big boost.


As far as NFS being a choke point, you might consider an experiment. Compiling SMB/SAMBA/CIFS for both the server and client and doing a test may yield better throughput on your remote filesystem connections. This is not a promise, but, in more than a few cases, I have been able to greatly exceed NFS throughput with SAMBA...


Interesting... Is that true for even the latest NFS?


Regards,
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 569373 · Report as offensive
netwraith

Send message
Joined: 15 Sep 06
Posts: 8
Credit: 61,290
RAC: 0
United States
Message 569646 - Posted: 17 May 2007, 17:06:07 UTC - in response to Message 569373.  

Very good helpful comments there...

...There are a few engineering companies around that feel that Gigabit saturates at around 314MBits anyway.. This being a combination of variable frame 1500 byte packets and the interupt overhead (Ethernet and the 1500 byte packet were designed at 1 megabit!!)) If your network guys are up to it, you might consider an etherchannel for your Internet interface.. (Several ethernet ganged together). At any rate, dropping packets should not be tolerated... so get working on this...

A good fix for internal networks is to increase the maximum packet size up to whatever your systems can comfortably take or up to just larger than the usual transaction size. Hence, more data for fewer packets and less overhead. (This is no good for external to the internet!)

Eliminating all the TCP retries should give a big boost.


As far as NFS being a choke point, you might consider an experiment. Compiling SMB/SAMBA/CIFS for both the server and client and doing a test may yield better throughput on your remote filesystem connections. This is not a promise, but, in more than a few cases, I have been able to greatly exceed NFS throughput with SAMBA...


Interesting... Is that true for even the latest NFS?


Regards,
Martin


Unless they have reworked NFS to no longer be stateless... I think it will remain true. As you know files have a state i.e. NEW, OPEN, CLOSED, TRUNCATED, EXTENDED, etc..... with NFS the design makes it so that all native states are accomplished in a single transaction..

i.e.
LOCK/CHK_XCLUSIVE -> OPEN -> POSITION -> WRITE -> VERIFY -> CLOSE -> SYNC -> UNLOCK ... are all done serially on the server... Of course this is an example for a write of a random access file.. Other types of transactions will differ in components or order.

While this makes NFS very robust as far as enduring crashes.. (it does not acknowledge ANY transaction to the client until the final SYNC is accomplished)... it makes for a slow boat...

Now, I don't really know what the latest NFS entails, nor do I know if SETI is using it or an earlier version (Older is more likely if other O.S. is involved with Solaris), but, I suspect that giving up the stateless nature would create incompatibilities with many applications. I think it is unlikely, but, stranger things have happened.

So.... conclusions beyond what I know may need to be drawn and elucidated by someone more familiar with the latest NFS...

P.S. While a SUN afficionado, I have not been seriously involved with them since Solaris8... My current employment is nearly 100% Linux, so, it's a case of knowing what you need to know. My applications work efficiently with NFS for the most part, and where I use SAMBA, I have built in a loss risk for the application and was able to restructure the flow so that there are backups of transactions that can be replayed if needed. It has not happened yet...

As far as my home systems. I have a few enterprise boxes that stay powered down for the most part. They suck quite a bit of power for what they do and produce quite a bit of heat. Nearly all my RISC stuff is off these days, except for a couple of DEC ALPHA's that crunch SETI...

ID: 569646 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 569876 - Posted: 17 May 2007, 23:23:46 UTC - in response to Message 569646.  

Thanks for the comparison. I'd never considered NFS to have any significant overhead but then again I've never stressed the systems to breaking point.

Meanwhile, SAMBA works well enough also.

To be looked at further when critical!

Regards,
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 569876 · Report as offensive
Previous · 1 . . . 3 · 4 · 5 · 6

Message boards : Technical News : Gasping for Air (May 14 2007)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.