Message boards :
Technical News :
Working as Expected (Jul 13 2009)
Message board moderation
Previous · 1 . . . 6 · 7 · 8 · 9 · 10 · 11 · Next
Author | Message |
---|---|
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
Hey! Shame on you... Cynicism doesn't become you. Shame on you for missing my pun-ish humour and the pun on a few lines from an old Star Trek episode... Is it that an Ameri(k)an UK divide/divergence? ;-) Cheers, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads. OUCH! That's a direct link to Cricket. Very antisocial... I'll capture the latest graphs and link them to my hosting. Interesting point there! Note that there are many bits in a packet and so the packet numbers will be a ratio of the bit rate, assuming that the average packet size remains the same. More musing... Regards, Martin Bit rate: And the packet rate (x2 vertical scale): And the bit rate and packet rate (x2 vertical scale) combined: Note: Green = downlink, blue line = uplink. (Note 2: Fixed plots. DO NOT LINK DIRECTLY TO CRICKET!) See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads. If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data. Surely Andy's point is that some packets (setting up/acknowledging the connection) will have many fewer bits than the data packets? So when the downloads allow a space, many more of the upload packets contain real data, and there's much less administrative chitter-chatter. * The cricket graphs only update once every four minutes, so the data-rate is much less than, say, streaming. |
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
Aside: If a live link is OK for Eric Korpela, I think we can survive one in Technical News - remember we're only using Campus bandwidth for the forums (and Cricket bandwidth for the actual semi-static* image), not getting in the way of the Hurricane Electric data. ... That assumes that Cricket doesn't get touched if the page is cached. There's previous mention not to directly link to avoid loading the Cricket box. (Not a problem of bandwidth but more that Cricket is not administered by s@h and is supposedly for the benefit of on-campus users only. Abuse it, it will get switched off. [Mods comment here?]) In any case, a direct link is futility. Tomorrow, whatever you're trying to show will have scrolled off the side into oblivion. Hence, just simply use a link to a static copy hosted somewhere else. Easy enough even for a complete non-webbie such as myself! Cheers, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
Sutaru Tsureku Send message Joined: 6 Apr 07 Posts: 7105 Credit: 147,663,825 RAC: 5 |
@ current server manage crew ;-) Matt had let the UL/DL server ONLINE during the weekly maintenance. Last week the UL server was OFFLINE. If possible, let them ONLINE during the maintenance time. It would help to UL stopped results. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Aside: If you look further down this very thread (message 918592 - and notice how I took out the nowrap to avoid the excess loading of posts below the "all except the last 75" cutoff), you'll see a static Cricket image hosted off my Imageshack space, again precisely so that we could freeze a moment in time and study it. Been there, done that. I also had a live Cricket image moderated a couple of weeks ago (I had quoted someone else - I think that two copies of the same image in a thread is no worse for bandwidth than a single copy). The Moderator message said "The Link to the Cricket Graph cause headaches for what happens over the Berkeley wire, everytime a user look at the forum thread it cause a query to the Graph. Which also goes up the hill eating network bandwidth. It is also hosted on a Server that Seti does not own." Whilst I would not mind honouring a request from the Cricket Admins not to link their resource (though it would be a shame to lose it), the other reasons given don't seem to hold water. So I'm not sure how "official" the embargo on live links is. |
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads. ... (Pale green is bits, dark green is packets. Thin blue is bits, thick blue is packets.) Looking closely at the period Mon 19:00 - Mon 21:00: 19:00: Downlink: Both the bit rate and the packet rate sharply ramp downwards; Uplink: Bit rate jumps stepwise to a maximum, the packet rate sharply ramps downwards; 19:30: Downlink: Both the bit rate and the packet rate sharply ramp upwards to a maximum; Uplink: Bit rate jumps stepwise down to a background average level, the packet rate sharply ramps upwards; 19:45: Downlink: Both the bit rate and the packet rate slowly ramp down; Uplink: Bit rate slowly ramps down, the packet rate remains steady; 20:20: Downlink: Bit rate remains at a maximum, the packet rate slowly ramps up; Uplink: Bit rate slowly ramps up, the packet rate remains steady; 20:50: Downlink: Bit rate remains at a maximum, the packet rate transitions to a noisy average; Uplink: Bit rate and packet rate transition to a noisy average. OK folks: The diagnosis is? Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
Aside: I'm not 'official' on these forums. I'm not Cricket or s@h admin. And you've obviously 'been there. done that'. Myself, I just consider it to be good sense and good netiquette. Also a "very good idea to avoid spoiling the party". I'm happy to hand the issue over to the mods for comment. Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that? Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
W-K 666 Send message Joined: 18 May 99 Posts: 19048 Credit: 40,757,560 RAC: 67 |
Have to say I wasn't aware of the request not to link directly, have been fairly busy lately, and not here as much as usual. If it causes a problem could the mods remove the link. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
I'm happy to hand the issue over to the mods for comment. My quote - you've obviously noticed we're skating on thin ice here! - was designed to raise the possibility that the mod in question - who I won't name - might have been operating 'freelance', with the best of intentions but ending up blocking a useful tool (sometimes a live version is relevant, unlike this discussion) on the basis of a false premise. I'd prefer it if admin comments, rather than a mod, on technical matters: obviously the mod might have been passing on an official 'admin' request, but that wasn't how it was phrased, even in the bits of the message I've left out. Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that? I've already posted my theory that when the link is less than saturated, a higher proportion of data packets, as opposed to administration packets, get through. |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
Just noticed that even though there was a download dip and upload spike in the graph that ML1 displayed, that when looking at the packets there is virtually no variation on the uploads. I think I detected a signal from ET in that waveform! :D SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
Hey! Shame on you... Cynicism doesn't become you. Ah, yes. I the humor did go right over my head! I'm not familar with that particular lined, which is why I think I missed it. :) |
ML1 Send message Joined: 25 Nov 01 Posts: 20258 Credit: 7,508,002 RAC: 20 |
I'm happy to hand the issue over to the mods for comment. ONLY in that we are not the authority on the issue. But then also there is good common sense. You play your ghetto blaster at 95dB and you can expect to get your wires cut. If I as a busy Cricket admin noticed a higher than normal hit rate from outside that gave cause for note, or just slowed things for when internal people wished to check, then simplest and quickest is to just block the external load and remove an impending headache. We then lose a valuable reference for those that know about it. It's very easy to avoid such a risk, especially so for something for which s@h may be seen as a headache for the rest of campus... (You can't easily hide a heard of a few thousand desperate crunchers!) Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that? Nearly... My thoughts are that the 'administration' packets are the more important and critical and avoiding those getting dropped avoids needless resends of the associated big data packets. You also get the required resend requests for when a data packet gets dropped. More interesting is the outlined phases that the link goes through over the 2 hour period. Note also the difference from the presumed server-side outage earlier in the chart. Regards, Martin See new freedom: Mageia Linux Take a look for yourself: Linux Format The Future is what We all make IT (GPLv3) |
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
I'm happy to hand the issue over to the mods for comment. I believe this was a current hot-button topic recently, and I believe Eric himself chimed in and said that it wasn't an issue linking to the cricket graph. |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
Meanwhile, the case in hand is the interesting interaction of downloads poisoning uploads... Your comment on that? Nearly... My thoughts are that the 'administration' packets are the more important and critical and avoiding those getting dropped avoids needless resends of the associated big data packets. You also get the required resend requests for when a data packet gets dropped.[/quote] Nearly.... It's also a case of timeouts. Packets come in, and if the ACK can't find its way back to the sender in a reasonable amount of time, the sender will time out and send again. It's possible that the packet being retried is still in a buffer somewhere. We also seem to forget that each TCP connection, sending no data at all, has a cost. It takes up space in the *NIX kernel for the "handle" and the IP stack has to service it periodically. We should not forget that, because Eric's "I just increased the bandwidth" works by taking open, stable connection handles, and closes them if they aren't likely to get turned over to the right process(es) on time. I'm not an expert on Red Hat or Apache, but the same problem exists in other environments. The simple answer is: you want just enough activity to keep everything at the most efficient speed -- if that's 80mbits (which feels about right) and the average connection can do 250k, then 320 connections is perfect. You can probably do double that before you're in trouble, but that's the idea. |
CAHess-Den Send message Joined: 20 May 99 Posts: 21 Credit: 2,575,162 RAC: 0 |
I've been smirking as I read through most of this.... We're frazzled because... this is all being more successful at what it's doing than we ever imagined?!? ROFL! Keep up the good work, folks! And, like is says on the front cover of the manual: "Don't Panic!" |
Josef W. Segur Send message Joined: 30 Oct 99 Posts: 4504 Credit: 1,414,761 RAC: 0 |
Observation FWIW. Several hours after the beginning of the Tuesday outage there was near zero download bandwidth in use and the usual 5 MBits/sec or so on the upload side from hosts trying work requests each hour. Packet rates were low both ways. I had as usual been running with Network activity disabled and had only 5 uploads ready to go, none had been tried. I enabled Network, BOINC tried and tried and tried, eventually one upload was successful before the backoffs became ridiculous. The obvious conclusion is that saturated download isn't the only cause of difficult uploads. I wonder if the internal 1 GBit network might have been near saturation doing the database backup, and whether that part which is invisible to us may make most of our theorizing moot. Joe |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
I'm happy to hand the issue over to the mods for comment. I didn't actually read him saying anything, but he did it himself in Panic Mode (20) five days ago, and the Panic Mode threads are pretty heavy traffic - I took that as a strong hint! (Sometimes a picture is worth a thousand words). But I agree we should exercise restraint in linking live, and just post a reference link if we're just explaining to a new volunteer that things are busy, and a static external image like Martin and I have used if we want to dicuss events at a particular moment in time. But then - note that if someone follows a link to the full Cricket page, they put more strain on the Cricket server with a request for (usually) two graphs and a page of data, instead of just the image file! Sorry for delayed post - I had it in preview when maintenance struck! |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14650 Credit: 200,643,578 RAC: 874 |
Observation FWIW. To add to that - I had about 20 tasks waiting to upload at the beginning of maintenance. I waited maybe 15 minutes, and hit retry - they all went through at the first attempt. Upload traffic was about 10 Mbits at the time. But later in the outage, as more tasks finished, I saw uploads start to stack up again before there were any downloads - must have been some other internal issue, as Joe says. |
Invisible Man Send message Joined: 24 Jun 01 Posts: 22 Credit: 1,129,336 RAC: 0 |
Well, this is the 179th reply to Matt's message of the 13th July. It is now the 22nd. Has he been run over by a bus, has he got swine flu, on holiday or what? Obviously there is a very real problem here to solve, but just a few lines from somebody at Berkeley would be appreciated. Anyway, keep up the good work!!! |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.