Working as Expected (Jul 13 2009)


log in

Advanced search

Message boards : Technical News : Working as Expected (Jul 13 2009)

Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · Next
Author Message
OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13658
Credit: 31,489,114
RAC: 11,566
United States
Message 920092 - Posted: 21 Jul 2009, 14:26:05 UTC - in response to Message 920042.

As of approximately 1:30am UTC, it looks like there is a nice fat spike in uploaded (probably actually reported) results.

That's just Vistro and TCP Jesus/MC Hammer or whatever he decides to call himself this week pressing the retry button too many times.

Hey! Shame on you... Cynicism doesn't become you.


Then you obviously don't know me very well, nor my sense of humor. Its ok, a lot of people don't get my sense of humor until they really get a chance to know me.
____________

DJStarfox
Send message
Joined: 23 May 01
Posts: 1045
Credit: 568,320
RAC: 353
United States
Message 920093 - Posted: 21 Jul 2009, 14:27:58 UTC - in response to Message 920087.

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads.


Yes, good point. This tells me that downloads are limiting upload speeds. This was discussed earlier. As you know, we should expect upload and download speeds to be highly correlated.

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8567
Credit: 4,232,738
RAC: 1,113
United Kingdom
Message 920095 - Posted: 21 Jul 2009, 14:32:46 UTC - in response to Message 920092.

Hey! Shame on you... Cynicism doesn't become you.


Then you obviously don't know me very well, nor my sense of humor. Its ok, a lot of people don't get my sense of humor until they really get a chance to know me.

Shame on you for missing my pun-ish humour and the pun on a few lines from an old Star Trek episode...

Is it that an Ameri(k)an UK divide/divergence?

;-)

Cheers,
Martin


____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8567
Credit: 4,232,738
RAC: 1,113
United Kingdom
Message 920098 - Posted: 21 Jul 2009, 14:41:10 UTC - in response to Message 920087.
Last modified: 21 Jul 2009, 15:22:45 UTC

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads.
img http://fragment1.berkeley.edu/newcricket/mini-graph.cgi?type=png;target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;inst=0;dslist=ifInUcastPackets%2CifOutUcastPackets;range=151200;rand=252 /img

OUCH! That's a direct link to Cricket. Very antisocial...

I'll capture the latest graphs and link them to my hosting.


Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

More musing...

Regards,
Martin


Bit rate:




And the packet rate (x2 vertical scale):




And the bit rate and packet rate (x2 vertical scale) combined:



Note: Green = downlink, blue line = uplink.

(Note 2: Fixed plots. DO NOT LINK DIRECTLY TO CRICKET!)
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8759
Credit: 52,708,158
RAC: 26,743
United Kingdom
Message 920103 - Posted: 21 Jul 2009, 14:51:26 UTC - in response to Message 920098.

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads.
img http://fragment1.berkeley.edu/newcricket/mini-graph.cgi?type=png;target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;inst=0;dslist=ifInUcastPackets%2CifOutUcastPackets;range=151200;rand=252 /img

OUCH! That's a direct link to Cricket. Very antisocial...

I'll capture the latest graphs and link them to my hosting.

Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

More musing...

Regards,
Martin


If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data.

Surely Andy's point is that some packets (setting up/acknowledging the connection) will have many fewer bits than the data packets? So when the downloads allow a space, many more of the upload packets contain real data, and there's much less administrative chitter-chatter.

* The cricket graphs only update once every four minutes, so the data-rate is much less than, say, streaming.

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8567
Credit: 4,232,738
RAC: 1,113
United Kingdom
Message 920108 - Posted: 21 Jul 2009, 14:56:37 UTC - in response to Message 920103.
Last modified: 21 Jul 2009, 14:58:14 UTC

Aside:

If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data. ...

* The cricket graphs only update once every four minutes, so the data-rate is much less than, say, streaming.

That assumes that Cricket doesn't get touched if the page is cached.

There's previous mention not to directly link to avoid loading the Cricket box. (Not a problem of bandwidth but more that Cricket is not administered by s@h and is supposedly for the benefit of on-campus users only. Abuse it, it will get switched off. [Mods comment here?])

In any case, a direct link is futility. Tomorrow, whatever you're trying to show will have scrolled off the side into oblivion.


Hence, just simply use a link to a static copy hosted somewhere else. Easy enough even for a complete non-webbie such as myself!

Cheers,
Martin
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Profile [seti.international] Dirk Sadowski
Volunteer tester
Avatar
Send message
Joined: 6 Apr 07
Posts: 7115
Credit: 61,255,835
RAC: 7,091
Germany
Message 920111 - Posted: 21 Jul 2009, 15:05:51 UTC


@ current server manage crew ;-)

Matt had let the UL/DL server ONLINE during the weekly maintenance.
Last week the UL server was OFFLINE.

If possible, let them ONLINE during the maintenance time.
It would help to UL stopped results.

____________
BR

SETI@home Needs your Help ... $10 & U get a Star!

Team seti.international

Das Deutsche Cafe. The German Cafe.

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8759
Credit: 52,708,158
RAC: 26,743
United Kingdom
Message 920113 - Posted: 21 Jul 2009, 15:42:05 UTC - in response to Message 920108.

Aside:

If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data. ...

* The cricket graphs only update once every four minutes, so the data-rate is much less than, say, streaming.

That assumes that Cricket doesn't get touched if the page is cached.

There's previous mention not to directly link to avoid loading the Cricket box. (Not a problem of bandwidth but more that Cricket is not administered by s@h and is supposedly for the benefit of on-campus users only. Abuse it, it will get switched off. [Mods comment here?])

In any case, a direct link is futility. Tomorrow, whatever you're trying to show will have scrolled off the side into oblivion.


Hence, just simply use a link to a static copy hosted somewhere else. Easy enough even for a complete non-webbie such as myself!

Cheers,
Martin

If you look further down this very thread (message 918592 - and notice how I took out the nowrap to avoid the excess loading of posts below the "all except the last 75" cutoff), you'll see a static Cricket image hosted off my Imageshack space, again precisely so that we could freeze a moment in time and study it. Been there, done that.

I also had a live Cricket image moderated a couple of weeks ago (I had quoted someone else - I think that two copies of the same image in a thread is no worse for bandwidth than a single copy). The Moderator message said "The Link to the Cricket Graph cause headaches for what happens over the Berkeley wire, everytime a user look at the forum thread it cause a query to the Graph. Which also goes up the hill eating network bandwidth. It is also hosted on a Server that Seti does not own."

Whilst I would not mind honouring a request from the Cricket Admins not to link their resource (though it would be a shame to lose it), the other reasons given don't seem to hold water. So I'm not sure how "official" the embargo on live links is.

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8567
Credit: 4,232,738
RAC: 1,113
United Kingdom
Message 920114 - Posted: 21 Jul 2009, 15:48:02 UTC - in response to Message 920098.
Last modified: 21 Jul 2009, 15:49:40 UTC

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads. ...

... Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

... And the bit rate and packet rate (x2 vertical scale) combined:



Note: Green = downlink, blue line = uplink.

(Note 2: Fixed plots. DO NOT LINK DIRECTLY TO CRICKET!)

(Pale green is bits, dark green is packets. Thin blue is bits, thick blue is packets.)


Looking closely at the period Mon 19:00 - Mon 21:00:

19:00:
Downlink: Both the bit rate and the packet rate sharply ramp downwards;
Uplink: Bit rate jumps stepwise to a maximum, the packet rate sharply ramps downwards;

19:30:
Downlink: Both the bit rate and the packet rate sharply ramp upwards to a maximum;
Uplink: Bit rate jumps stepwise down to a background average level, the packet rate sharply ramps upwards;

19:45:
Downlink: Both the bit rate and the packet rate slowly ramp down;
Uplink: Bit rate slowly ramps down, the packet rate remains steady;

20:20:
Downlink: Bit rate remains at a maximum, the packet rate slowly ramps up;
Uplink: Bit rate slowly ramps up, the packet rate remains steady;

20:50:
Downlink: Bit rate remains at a maximum, the packet rate transitions to a noisy average;
Uplink: Bit rate and packet rate transition to a noisy average.


OK folks: The diagnosis is?

Regards,
Martin
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8567
Credit: 4,232,738
RAC: 1,113
United Kingdom
Message 920119 - Posted: 21 Jul 2009, 15:57:07 UTC - in response to Message 920113.
Last modified: 21 Jul 2009, 15:58:42 UTC

Aside:

... Abuse it, it will get switched off. [Mods comment here?])

In any case, a direct link is futility. Tomorrow, whatever you're trying to show will have scrolled off the side into oblivion.

... you'll see a static Cricket image hosted off my Imageshack space, again precisely so that we could freeze a moment in time and study it. Been there, done that.

... Whilst I would not mind honouring a request from the Cricket Admins not to link their resource (though it would be a shame to lose it) ...

I'm not 'official' on these forums. I'm not Cricket or s@h admin. And you've obviously 'been there. done that'.

Myself, I just consider it to be good sense and good netiquette. Also a "very good idea to avoid spoiling the party".

I'm happy to hand the issue over to the mods for comment.

Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

Regards,
Martin
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

WinterKnight
Volunteer tester
Send message
Joined: 18 May 99
Posts: 8737
Credit: 25,596,566
RAC: 12,620
United Kingdom
Message 920122 - Posted: 21 Jul 2009, 16:03:28 UTC

Have to say I wasn't aware of the request not to link directly, have been fairly busy lately, and not here as much as usual.

If it causes a problem could the mods remove the link.

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8759
Credit: 52,708,158
RAC: 26,743
United Kingdom
Message 920125 - Posted: 21 Jul 2009, 16:13:08 UTC - in response to Message 920119.

I'm happy to hand the issue over to the mods for comment.

My quote - you've obviously noticed we're skating on thin ice here! - was designed to raise the possibility that the mod in question - who I won't name - might have been operating 'freelance', with the best of intentions but ending up blocking a useful tool (sometimes a live version is relevant, unlike this discussion) on the basis of a false premise. I'd prefer it if admin comments, rather than a mod, on technical matters: obviously the mod might have been passing on an official 'admin' request, but that wasn't how it was phrased, even in the bits of the message I've left out.

Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

I've already posted my theory that when the link is less than saturated, a higher proportion of data packets, as opposed to administration packets, get through.

Profile HAL9000
Volunteer tester
Avatar
Send message
Joined: 11 Sep 99
Posts: 4590
Credit: 121,550,217
RAC: 52,464
United States
Message 920129 - Posted: 21 Jul 2009, 16:20:15 UTC - in response to Message 920098.

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads.

OUCH! That's a direct link to Cricket. Very antisocial...

I'll capture the latest graphs and link them to my hosting.


Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

More musing...

Regards,
Martin


Bit rate:




And the packet rate (x2 vertical scale):




And the bit rate and packet rate (x2 vertical scale) combined:



Note: Green = downlink, blue line = uplink.

(Note 2: Fixed plots. DO NOT LINK DIRECTLY TO CRICKET!)


I think I detected a signal from ET in that waveform! :D
____________
SETI@home classic workunits: 93,865 CPU time: 863,447 hours

Join the BP6/VP6 User Group today!

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13658
Credit: 31,489,114
RAC: 11,566
United States
Message 920131 - Posted: 21 Jul 2009, 16:23:27 UTC - in response to Message 920095.

Hey! Shame on you... Cynicism doesn't become you.


Then you obviously don't know me very well, nor my sense of humor. Its ok, a lot of people don't get my sense of humor until they really get a chance to know me.

Shame on you for missing my pun-ish humour and the pun on a few lines from an old Star Trek episode...

Is it that an Ameri(k)an UK divide/divergence?

;-)


Ah, yes. I the humor did go right over my head! I'm not familar with that particular lined, which is why I think I missed it. :)
____________

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8567
Credit: 4,232,738
RAC: 1,113
United Kingdom
Message 920134 - Posted: 21 Jul 2009, 16:26:01 UTC - in response to Message 920125.
Last modified: 21 Jul 2009, 16:28:42 UTC

I'm happy to hand the issue over to the mods for comment.

My quote - you've obviously noticed we're skating on thin ice here!

ONLY in that we are not the authority on the issue.

But then also there is good common sense. You play your ghetto blaster at 95dB and you can expect to get your wires cut.

If I as a busy Cricket admin noticed a higher than normal hit rate from outside that gave cause for note, or just slowed things for when internal people wished to check, then simplest and quickest is to just block the external load and remove an impending headache. We then lose a valuable reference for those that know about it.

It's very easy to avoid such a risk, especially so for something for which s@h may be seen as a headache for the rest of campus... (You can't easily hide a heard of a few thousand desperate crunchers!)


Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

I've already posted my theory that when the link is less than saturated, a higher proportion of data packets, as opposed to administration packets, get through.

Nearly...

My thoughts are that the 'administration' packets are the more important and critical and avoiding those getting dropped avoids needless resends of the associated big data packets. You also get the required resend requests for when a data packet gets dropped.

More interesting is the outlined phases that the link goes through over the 2 hour period. Note also the difference from the presumed server-side outage earlier in the chart.

Regards,
Martin
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13658
Credit: 31,489,114
RAC: 11,566
United States
Message 920137 - Posted: 21 Jul 2009, 16:28:04 UTC - in response to Message 920119.

I'm happy to hand the issue over to the mods for comment.


I believe this was a current hot-button topic recently, and I believe Eric himself chimed in and said that it wasn't an issue linking to the cricket graph.
____________

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 920159 - Posted: 21 Jul 2009, 23:52:19 UTC - in response to Message 920134.

Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

I've already posted my theory that when the link is less than saturated, a higher proportion of data packets, as opposed to administration packets, get through.

Nearly...

My thoughts are that the 'administration' packets are the more important and critical and avoiding those getting dropped avoids needless resends of the associated big data packets. You also get the required resend requests for when a data packet gets dropped.[/quote]
Nearly....

It's also a case of timeouts. Packets come in, and if the ACK can't find its way back to the sender in a reasonable amount of time, the sender will time out and send again. It's possible that the packet being retried is still in a buffer somewhere.

We also seem to forget that each TCP connection, sending no data at all, has a cost. It takes up space in the *NIX kernel for the "handle" and the IP stack has to service it periodically.

We should not forget that, because Eric's "I just increased the bandwidth" works by taking open, stable connection handles, and closes them if they aren't likely to get turned over to the right process(es) on time.

I'm not an expert on Red Hat or Apache, but the same problem exists in other environments.

The simple answer is: you want just enough activity to keep everything at the most efficient speed -- if that's 80mbits (which feels about right) and the average connection can do 250k, then 320 connections is perfect.

You can probably do double that before you're in trouble, but that's the idea.
____________

CAHess-Den
Send message
Joined: 20 May 99
Posts: 21
Credit: 1,618,471
RAC: 795
United States
Message 920161 - Posted: 22 Jul 2009, 0:01:20 UTC



I've been smirking as I read through most of this....

We're frazzled because... this is all being more successful at what it's doing than we ever imagined?!?


ROFL!

Keep up the good work, folks!

And, like is says on the front cover of the manual: "Don't Panic!"


____________

Josef W. SegurProject donor
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4332
Credit: 1,113,463
RAC: 995
United States
Message 920217 - Posted: 22 Jul 2009, 3:22:20 UTC

Observation FWIW.

Several hours after the beginning of the Tuesday outage there was near zero download bandwidth in use and the usual 5 MBits/sec or so on the upload side from hosts trying work requests each hour. Packet rates were low both ways. I had as usual been running with Network activity disabled and had only 5 uploads ready to go, none had been tried. I enabled Network, BOINC tried and tried and tried, eventually one upload was successful before the backoffs became ridiculous.

The obvious conclusion is that saturated download isn't the only cause of difficult uploads. I wonder if the internal 1 GBit network might have been near saturation doing the database backup, and whether that part which is invisible to us may make most of our theorizing moot.

Joe

Richard HaselgroveProject donor
Volunteer tester
Send message
Joined: 4 Jul 99
Posts: 8759
Credit: 52,708,158
RAC: 26,743
United Kingdom
Message 920259 - Posted: 22 Jul 2009, 8:29:39 UTC - in response to Message 920137.
Last modified: 22 Jul 2009, 8:29:58 UTC

I'm happy to hand the issue over to the mods for comment.

I believe this was a current hot-button topic recently, and I believe Eric himself chimed in and said that it wasn't an issue linking to the cricket graph.

I didn't actually read him saying anything, but he did it himself in Panic Mode (20) five days ago, and the Panic Mode threads are pretty heavy traffic - I took that as a strong hint! (Sometimes a picture is worth a thousand words).

But I agree we should exercise restraint in linking live, and just post a reference link if we're just explaining to a new volunteer that things are busy, and a static external image like Martin and I have used if we want to dicuss events at a particular moment in time.

But then - note that if someone follows a link to the full Cricket page, they put more strain on the Cricket server with a request for (usually) two graphs and a page of data, instead of just the image file!

Sorry for delayed post - I had it in preview when maintenance struck!

Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · Next

Message boards : Technical News : Working as Expected (Jul 13 2009)

Copyright © 2014 University of California