Working as Expected (Jul 13 2009)

Message boards : Technical News : Working as Expected (Jul 13 2009)
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · Next

AuthorMessage
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 920095 - Posted: 21 Jul 2009, 14:32:46 UTC - in response to Message 920092.  

Hey! Shame on you... Cynicism doesn't become you.


Then you obviously don't know me very well, nor my sense of humor. Its ok, a lot of people don't get my sense of humor until they really get a chance to know me.

Shame on you for missing my pun-ish humour and the pun on a few lines from an old Star Trek episode...

Is it that an Ameri(k)an UK divide/divergence?

;-)

Cheers,
Martin


See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 920095 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 920098 - Posted: 21 Jul 2009, 14:41:10 UTC - in response to Message 920087.  
Last modified: 21 Jul 2009, 15:22:45 UTC

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads.
img http://fragment1.berkeley.edu/newcricket/mini-graph.cgi?type=png;target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;inst=0;dslist=ifInUcastPackets%2CifOutUcastPackets;range=151200;rand=252 /img

OUCH! That's a direct link to Cricket. Very antisocial...

I'll capture the latest graphs and link them to my hosting.


Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

More musing...

Regards,
Martin


Bit rate:




And the packet rate (x2 vertical scale):




And the bit rate and packet rate (x2 vertical scale) combined:



Note: Green = downlink, blue line = uplink.

(Note 2: Fixed plots. DO NOT LINK DIRECTLY TO CRICKET!)
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 920098 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 920103 - Posted: 21 Jul 2009, 14:51:26 UTC - in response to Message 920098.  

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads.
img http://fragment1.berkeley.edu/newcricket/mini-graph.cgi?type=png;target=%2Frouter-interfaces%2Finr-250%2Fgigabitethernet2_3;inst=0;dslist=ifInUcastPackets%2CifOutUcastPackets;range=151200;rand=252 /img

OUCH! That's a direct link to Cricket. Very antisocial...

I'll capture the latest graphs and link them to my hosting.

Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

More musing...

Regards,
Martin


If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data.

Surely Andy's point is that some packets (setting up/acknowledging the connection) will have many fewer bits than the data packets? So when the downloads allow a space, many more of the upload packets contain real data, and there's much less administrative chitter-chatter.

* The cricket graphs only update once every four minutes, so the data-rate is much less than, say, streaming.
ID: 920103 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 920108 - Posted: 21 Jul 2009, 14:56:37 UTC - in response to Message 920103.  
Last modified: 21 Jul 2009, 14:58:14 UTC

Aside:

If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data. ...

* The cricket graphs only update once every four minutes, so the data-rate is much less than, say, streaming.

That assumes that Cricket doesn't get touched if the page is cached.

There's previous mention not to directly link to avoid loading the Cricket box. (Not a problem of bandwidth but more that Cricket is not administered by s@h and is supposedly for the benefit of on-campus users only. Abuse it, it will get switched off. [Mods comment here?])

In any case, a direct link is futility. Tomorrow, whatever you're trying to show will have scrolled off the side into oblivion.


Hence, just simply use a link to a static copy hosted somewhere else. Easy enough even for a complete non-webbie such as myself!

Cheers,
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 920108 · Report as offensive
Profile Sutaru Tsureku
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 920111 - Posted: 21 Jul 2009, 15:05:51 UTC


@ current server manage crew ;-)

Matt had let the UL/DL server ONLINE during the weekly maintenance.
Last week the UL server was OFFLINE.

If possible, let them ONLINE during the maintenance time.
It would help to UL stopped results.

ID: 920111 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 920113 - Posted: 21 Jul 2009, 15:42:05 UTC - in response to Message 920108.  

Aside:

If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data. ...

* The cricket graphs only update once every four minutes, so the data-rate is much less than, say, streaming.

That assumes that Cricket doesn't get touched if the page is cached.

There's previous mention not to directly link to avoid loading the Cricket box. (Not a problem of bandwidth but more that Cricket is not administered by s@h and is supposedly for the benefit of on-campus users only. Abuse it, it will get switched off. [Mods comment here?])

In any case, a direct link is futility. Tomorrow, whatever you're trying to show will have scrolled off the side into oblivion.


Hence, just simply use a link to a static copy hosted somewhere else. Easy enough even for a complete non-webbie such as myself!

Cheers,
Martin

If you look further down this very thread (message 918592 - and notice how I took out the nowrap to avoid the excess loading of posts below the "all except the last 75" cutoff), you'll see a static Cricket image hosted off my Imageshack space, again precisely so that we could freeze a moment in time and study it. Been there, done that.

I also had a live Cricket image moderated a couple of weeks ago (I had quoted someone else - I think that two copies of the same image in a thread is no worse for bandwidth than a single copy). The Moderator message said "The Link to the Cricket Graph cause headaches for what happens over the Berkeley wire, everytime a user look at the forum thread it cause a query to the Graph. Which also goes up the hill eating network bandwidth. It is also hosted on a Server that Seti does not own."

Whilst I would not mind honouring a request from the Cricket Admins not to link their resource (though it would be a shame to lose it), the other reasons given don't seem to hold water. So I'm not sure how "official" the embargo on live links is.
ID: 920113 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 920114 - Posted: 21 Jul 2009, 15:48:02 UTC - in response to Message 920098.  
Last modified: 21 Jul 2009, 15:49:40 UTC

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads. ...

... Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

... And the bit rate and packet rate (x2 vertical scale) combined:



Note: Green = downlink, blue line = uplink.

(Note 2: Fixed plots. DO NOT LINK DIRECTLY TO CRICKET!)

(Pale green is bits, dark green is packets. Thin blue is bits, thick blue is packets.)


Looking closely at the period Mon 19:00 - Mon 21:00:

19:00:
Downlink: Both the bit rate and the packet rate sharply ramp downwards;
Uplink: Bit rate jumps stepwise to a maximum, the packet rate sharply ramps downwards;

19:30:
Downlink: Both the bit rate and the packet rate sharply ramp upwards to a maximum;
Uplink: Bit rate jumps stepwise down to a background average level, the packet rate sharply ramps upwards;

19:45:
Downlink: Both the bit rate and the packet rate slowly ramp down;
Uplink: Bit rate slowly ramps down, the packet rate remains steady;

20:20:
Downlink: Bit rate remains at a maximum, the packet rate slowly ramps up;
Uplink: Bit rate slowly ramps up, the packet rate remains steady;

20:50:
Downlink: Bit rate remains at a maximum, the packet rate transitions to a noisy average;
Uplink: Bit rate and packet rate transition to a noisy average.


OK folks: The diagnosis is?

Regards,
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 920114 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 920119 - Posted: 21 Jul 2009, 15:57:07 UTC - in response to Message 920113.  
Last modified: 21 Jul 2009, 15:58:42 UTC

Aside:

... Abuse it, it will get switched off. [Mods comment here?])

In any case, a direct link is futility. Tomorrow, whatever you're trying to show will have scrolled off the side into oblivion.

... you'll see a static Cricket image hosted off my Imageshack space, again precisely so that we could freeze a moment in time and study it. Been there, done that.

... Whilst I would not mind honouring a request from the Cricket Admins not to link their resource (though it would be a shame to lose it) ...

I'm not 'official' on these forums. I'm not Cricket or s@h admin. And you've obviously 'been there. done that'.

Myself, I just consider it to be good sense and good netiquette. Also a "very good idea to avoid spoiling the party".

I'm happy to hand the issue over to the mods for comment.

Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

Regards,
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 920119 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19012
Credit: 40,757,560
RAC: 67
United Kingdom
Message 920122 - Posted: 21 Jul 2009, 16:03:28 UTC

Have to say I wasn't aware of the request not to link directly, have been fairly busy lately, and not here as much as usual.

If it causes a problem could the mods remove the link.
ID: 920122 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 920125 - Posted: 21 Jul 2009, 16:13:08 UTC - in response to Message 920119.  

I'm happy to hand the issue over to the mods for comment.

My quote - you've obviously noticed we're skating on thin ice here! - was designed to raise the possibility that the mod in question - who I won't name - might have been operating 'freelance', with the best of intentions but ending up blocking a useful tool (sometimes a live version is relevant, unlike this discussion) on the basis of a false premise. I'd prefer it if admin comments, rather than a mod, on technical matters: obviously the mod might have been passing on an official 'admin' request, but that wasn't how it was phrased, even in the bits of the message I've left out.

Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

I've already posted my theory that when the link is less than saturated, a higher proportion of data packets, as opposed to administration packets, get through.
ID: 920125 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 920129 - Posted: 21 Jul 2009, 16:20:15 UTC - in response to Message 920098.  

Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads.

OUCH! That's a direct link to Cricket. Very antisocial...

I'll capture the latest graphs and link them to my hosting.


Interesting point there!

Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same.

More musing...

Regards,
Martin


Bit rate:




And the packet rate (x2 vertical scale):




And the bit rate and packet rate (x2 vertical scale) combined:



Note: Green = downlink, blue line = uplink.

(Note 2: Fixed plots. DO NOT LINK DIRECTLY TO CRICKET!)


I think I detected a signal from ET in that waveform! :D
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 920129 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 920131 - Posted: 21 Jul 2009, 16:23:27 UTC - in response to Message 920095.  

Hey! Shame on you... Cynicism doesn't become you.


Then you obviously don't know me very well, nor my sense of humor. Its ok, a lot of people don't get my sense of humor until they really get a chance to know me.

Shame on you for missing my pun-ish humour and the pun on a few lines from an old Star Trek episode...

Is it that an Ameri(k)an UK divide/divergence?

;-)


Ah, yes. I the humor did go right over my head! I'm not familar with that particular lined, which is why I think I missed it. :)
ID: 920131 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 920134 - Posted: 21 Jul 2009, 16:26:01 UTC - in response to Message 920125.  
Last modified: 21 Jul 2009, 16:28:42 UTC

I'm happy to hand the issue over to the mods for comment.

My quote - you've obviously noticed we're skating on thin ice here!

ONLY in that we are not the authority on the issue.

But then also there is good common sense. You play your ghetto blaster at 95dB and you can expect to get your wires cut.

If I as a busy Cricket admin noticed a higher than normal hit rate from outside that gave cause for note, or just slowed things for when internal people wished to check, then simplest and quickest is to just block the external load and remove an impending headache. We then lose a valuable reference for those that know about it.

It's very easy to avoid such a risk, especially so for something for which s@h may be seen as a headache for the rest of campus... (You can't easily hide a heard of a few thousand desperate crunchers!)


Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

I've already posted my theory that when the link is less than saturated, a higher proportion of data packets, as opposed to administration packets, get through.

Nearly...

My thoughts are that the 'administration' packets are the more important and critical and avoiding those getting dropped avoids needless resends of the associated big data packets. You also get the required resend requests for when a data packet gets dropped.

More interesting is the outlined phases that the link goes through over the 2 hour period. Note also the difference from the presumed server-side outage earlier in the chart.

Regards,
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 920134 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 920137 - Posted: 21 Jul 2009, 16:28:04 UTC - in response to Message 920119.  

I'm happy to hand the issue over to the mods for comment.


I believe this was a current hot-button topic recently, and I believe Eric himself chimed in and said that it wasn't an issue linking to the cricket graph.
ID: 920137 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 920159 - Posted: 21 Jul 2009, 23:52:19 UTC - in response to Message 920134.  

Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that?

I've already posted my theory that when the link is less than saturated, a higher proportion of data packets, as opposed to administration packets, get through.

Nearly...

My thoughts are that the 'administration' packets are the more important and critical and avoiding those getting dropped avoids needless resends of the associated big data packets. You also get the required resend requests for when a data packet gets dropped.[/quote]
Nearly....

It's also a case of timeouts. Packets come in, and if the ACK can't find its way back to the sender in a reasonable amount of time, the sender will time out and send again. It's possible that the packet being retried is still in a buffer somewhere.

We also seem to forget that each TCP connection, sending no data at all, has a cost. It takes up space in the *NIX kernel for the "handle" and the IP stack has to service it periodically.

We should not forget that, because Eric's "I just increased the bandwidth" works by taking open, stable connection handles, and closes them if they aren't likely to get turned over to the right process(es) on time.

I'm not an expert on Red Hat or Apache, but the same problem exists in other environments.

The simple answer is: you want just enough activity to keep everything at the most efficient speed -- if that's 80mbits (which feels about right) and the average connection can do 250k, then 320 connections is perfect.

You can probably do double that before you're in trouble, but that's the idea.
ID: 920159 · Report as offensive
CAHess-Den

Send message
Joined: 20 May 99
Posts: 21
Credit: 2,575,162
RAC: 0
United States
Message 920161 - Posted: 22 Jul 2009, 0:01:20 UTC



I've been smirking as I read through most of this....

We're frazzled because... this is all being more successful at what it's doing than we ever imagined?!?


ROFL!

Keep up the good work, folks!

And, like is says on the front cover of the manual: "Don't Panic!"


ID: 920161 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 920217 - Posted: 22 Jul 2009, 3:22:20 UTC

Observation FWIW.

Several hours after the beginning of the Tuesday outage there was near zero download bandwidth in use and the usual 5 MBits/sec or so on the upload side from hosts trying work requests each hour. Packet rates were low both ways. I had as usual been running with Network activity disabled and had only 5 uploads ready to go, none had been tried. I enabled Network, BOINC tried and tried and tried, eventually one upload was successful before the backoffs became ridiculous.

The obvious conclusion is that saturated download isn't the only cause of difficult uploads. I wonder if the internal 1 GBit network might have been near saturation doing the database backup, and whether that part which is invisible to us may make most of our theorizing moot.
                                                                Joe
ID: 920217 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 920259 - Posted: 22 Jul 2009, 8:29:39 UTC - in response to Message 920137.  
Last modified: 22 Jul 2009, 8:29:58 UTC

I'm happy to hand the issue over to the mods for comment.

I believe this was a current hot-button topic recently, and I believe Eric himself chimed in and said that it wasn't an issue linking to the cricket graph.

I didn't actually read him saying anything, but he did it himself in Panic Mode (20) five days ago, and the Panic Mode threads are pretty heavy traffic - I took that as a strong hint! (Sometimes a picture is worth a thousand words).

But I agree we should exercise restraint in linking live, and just post a reference link if we're just explaining to a new volunteer that things are busy, and a static external image like Martin and I have used if we want to dicuss events at a particular moment in time.

But then - note that if someone follows a link to the full Cricket page, they put more strain on the Cricket server with a request for (usually) two graphs and a page of data, instead of just the image file!

Sorry for delayed post - I had it in preview when maintenance struck!
ID: 920259 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14649
Credit: 200,643,578
RAC: 874
United Kingdom
Message 920262 - Posted: 22 Jul 2009, 8:35:09 UTC - in response to Message 920217.  

Observation FWIW.

Several hours after the beginning of the Tuesday outage there was near zero download bandwidth in use and the usual 5 MBits/sec or so on the upload side from hosts trying work requests each hour. Packet rates were low both ways. I had as usual been running with Network activity disabled and had only 5 uploads ready to go, none had been tried. I enabled Network, BOINC tried and tried and tried, eventually one upload was successful before the backoffs became ridiculous.

The obvious conclusion is that saturated download isn't the only cause of difficult uploads. I wonder if the internal 1 GBit network might have been near saturation doing the database backup, and whether that part which is invisible to us may make most of our theorizing moot.
                                                                Joe

To add to that - I had about 20 tasks waiting to upload at the beginning of maintenance. I waited maybe 15 minutes, and hit retry - they all went through at the first attempt. Upload traffic was about 10 Mbits at the time. But later in the outage, as more tasks finished, I saw uploads start to stack up again before there were any downloads - must have been some other internal issue, as Joe says.
ID: 920262 · Report as offensive
Invisible Man

Send message
Joined: 24 Jun 01
Posts: 22
Credit: 1,129,336
RAC: 0
United Kingdom
Message 920263 - Posted: 22 Jul 2009, 8:36:47 UTC

Well, this is the 179th reply to Matt's message of the 13th July. It is now the 22nd.

Has he been run over by a bus, has he got swine flu, on holiday or what? Obviously there is a very real problem here to solve, but just a few lines from somebody at Berkeley would be appreciated.

Anyway, keep up the good work!!!
ID: 920263 · Report as offensive
Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · Next

Message boards : Technical News : Working as Expected (Jul 13 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.