Echoes of Classic (Mar 20 2007)


log in

Advanced search

Message boards : Technical News : Echoes of Classic (Mar 20 2007)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 534260 - Posted: 20 Mar 2007, 21:50:39 UTC
Last modified: 20 Mar 2007, 21:51:03 UTC

Regular backup outage today. Everything was normal except we bounced the replica database to change one buffer size setting and now nobody can connect to it - even to shut it down! Seems like we lost all our connection permission info somehow. From what we can tell it is still acting as a replica and making updates, but we can't access the data at all. We're stumped. Bob's looking into it.

On the plus side, we got all the pieces in place to move another function off kryten and onto bruno: file deletion. I just fired this off, and at first glance it seems faster. Time will tell if this is an improvement. Bruno is a faster machine in general, but kryten had a gigabit connection to the workunit file server, while due to lab infrastructure bruno can currently only have 100 Mbit. So we shall see. Hopefully queues will drain after we recover from the outage backlog.

Here's a fun one: Since the switchover to using Hurricane Electric as our main ISP I noticed lingering traffic on the campus router which served our Cogent link. We're talking as much as 1 Mbit/sec. Today while updating lab-wide DNS records I noticed shserver2 was still there. This was the DNS alias for our SETI@home classic data server. I removed this entry, and check out the dip in traffic:



So, well over a year since unplugging the classic data server, there are still enough SETI@home classic clients around the world trying to access a missing server to account for almost 1 Mbit/sec of traffic on UC Berkeley campus routers. Not sure how to exactly explain the shape of this graph (and why incoming = outgoing). The diurnal shape and hourly ridges look like scripts or cronjobs running on machines that haven't been checked in ages.

A lot of BOINC naysayers like to point out how many classic users "quit" last year after the big transition. But this graph adds some meat to my theory that a large chunk of the SETI@home classic users actually left the project many ages ago, and their old clients simply continued to run unattended. Mind you, this is 1 MBit/sec of traffic without actual workunit data being sent - just SYNs, basically. I think. Somebody break out a calculator and determine how many SETI@home classic clients this represents.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 534274 - Posted: 20 Mar 2007, 22:16:29 UTC - in response to Message 534260.
Last modified: 20 Mar 2007, 22:16:56 UTC


Here's a fun one: Since the switchover to using Hurricane Electric as our main ISP I noticed lingering traffic on the campus router which served our Cogent link. We're talking as much as 1 Mbit/sec. Today while updating lab-wide DNS records I noticed shserver2 was still there. This was the DNS alias for our SETI@home classic data server. I removed this entry, and check out the dip in traffic:

Matt,

There is a flaw in most resolvers dealing with hosts that don't exist: if a server says "I don't know shserver2" (rcode=3) the resolver will ask another one of your DNS servers.

It'll reduce DNS load if you put something in your zone for shserver2, I'd put 127.0.0.1.

-- Ned
____________

Profile Dr. C.E.T.I.
Avatar
Send message
Joined: 29 Feb 00
Posts: 15993
Credit: 690,597
RAC: 0
United States
Message 534303 - Posted: 20 Mar 2007, 22:58:39 UTC


. . . Nice work there Matt / Berkeley yo 4 bruno ;)

Somebody break out a calculator and determine how many SETI@home classic clients this represents


. . . callin' Esme' ;)

zoom314Project donor
Avatar
Send message
Joined: 30 Nov 03
Posts: 46120
Credit: 36,593,924
RAC: 5,358
Message 534393 - Posted: 21 Mar 2007, 2:21:19 UTC

So that is what was using the little bit of bandwisth, Classic just won't die. Interesting though. Facinating.
____________
My Facebook, War Commander, 2015

Profile mikey
Volunteer tester
Avatar
Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 534403 - Posted: 21 Mar 2007, 2:33:09 UTC - in response to Message 534260.

Here's a fun one: Since the switchover to using Hurricane Electric as our main ISP I noticed lingering traffic on the campus router which served our Cogent link. We're talking as much as 1 Mbit/sec. Today while updating lab-wide DNS records I noticed shserver2 was still there. This was the DNS alias for our SETI@home classic data server. I removed this entry, and check out the dip in traffic:

- Matt


Don't have a clue if it is relevant but waaaay back when, when you were having server issues ALOT of us put in actual server addresses in the "http proxy" section of Boinc. It worked until you caught on, seems we were using a connection line that cost you a fortune instead of a cheap one. I wonder if some hosts are still using that "technique".

____________

Odysseus
Volunteer tester
Avatar
Send message
Joined: 26 Jul 99
Posts: 1786
Credit: 3,807,823
RAC: 392
Canada
Message 534404 - Posted: 21 Mar 2007, 2:34:03 UTC - in response to Message 534393.

So that is what was using the little bit of bandwisth, Classic just won't die. Interesting though. Facinating.

There’s still the occasional posting in the Q&A boards from people asking why their S@h v3.x doesn’t seem to be making any progress …
____________

Profile Misfit
Volunteer tester
Avatar
Send message
Joined: 21 Jun 01
Posts: 21790
Credit: 2,510,901
RAC: 0
United States
Message 534426 - Posted: 21 Mar 2007, 2:53:10 UTC - in response to Message 534403.

It worked until you caught on, seems we were using a connection line that cost you a fortune instead of a cheap one.

Another reason for the lot of you to make a donation. Payback.
____________

Profile mikey
Volunteer tester
Avatar
Send message
Joined: 17 Dec 99
Posts: 4215
Credit: 3,474,603
RAC: 0
United States
Message 534553 - Posted: 21 Mar 2007, 10:12:25 UTC - in response to Message 534426.

It worked until you caught on, seems we were using a connection line that cost you a fortune instead of a cheap one.

Another reason for the lot of you to make a donation. Payback.

Did so a month ago, sent a $100.00 donation thru the new paypal way.
____________

Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar
Send message
Joined: 20 Dec 05
Posts: 1923
Credit: 9,755,933
RAC: 16,899
United States
Message 534641 - Posted: 21 Mar 2007, 16:57:30 UTC - in response to Message 534260.

[snip]

On the plus side, we got all the pieces in place to move another function off kryten and onto bruno: file deletion. I just fired this off, and at first glance it seems faster. Time will tell if this is an improvement. Bruno is a faster machine in general, but kryten had a gigabit connection to the workunit file server, while due to lab infrastructure bruno can currently only have 100 Mbit. So we shall see. Hopefully queues will drain after we recover from the outage backlog.

[snip]
- Matt


Matt, now that Bruno is actually showing up on the "server" page, the "hosts" section of that page should be updated to include it, don't ya think?
____________
.

Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 534643 - Posted: 21 Mar 2007, 17:05:48 UTC - in response to Message 534641.

Matt, now that Bruno is actually showing up on the "server" page, the "hosts" section of that page should be updated to include it, don't ya think?


Yes I do, which is why I did included it before I even started this thread. Fourth one down in the list. The list is in no specific order.

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Michael Roberts
Volunteer tester
Avatar
Send message
Joined: 20 Aug 99
Posts: 1897
Credit: 309,263
RAC: 617
Germany
Message 535269 - Posted: 22 Mar 2007, 23:11:42 UTC - in response to Message 534303.
Last modified: 22 Mar 2007, 23:13:29 UTC

Somebody break out a calculator and determine how many SETI@home classic clients this represents


Here's my response from the Café - can anyone improve the estimated input values or suggest a better model?
Well I'll start the ball rolling. There are two issues here - first a suitable model of the traffic origins and second, sensible input numbers.

From inspection of the graph I assume 0.7Mbit/S average shserver2 traffic, with 33% generated hourly and the rest generated daily. Reasonable choices of such values could affect the answer by factors of 2 or so and are thus not too critical for a first estimate.

From this conversation I will take an average packet size of 40 bytes (given that no actual data is being sent anywhere) This gives a little over 2000 packets per second and just under 200 million per day. I also assume that each contact results in 4 packets. But I really don't know in either case. The equality of input and output traffic rates suggests a strong minimum packet size effect.

Using a formula

numberofhosts = packetsperday/(hourlyfraction*24+(1-hourlyfraction))/packetspercontact


we then get 5768000 hosts (rounded a bit,) which at just under 6 million seems rather high, but it is more than a thousand and less than a billion, so it gives us something to talk about.

Profile Es99Project donor
Volunteer moderator
Volunteer tester
Avatar
Send message
Joined: 23 Aug 05
Posts: 8842
Credit: 250,374
RAC: 121
Canada
Message 535686 - Posted: 23 Mar 2007, 21:20:54 UTC - in response to Message 534303.


. . . Nice work there Matt / Berkeley yo 4 bruno ;)

Somebody break out a calculator and determine how many SETI@home classic clients this represents


. . . callin' Esme' ;)


What am I? The secretary? :p
____________
Are you a feminist? Take the test

Profile Dr. C.E.T.I.
Avatar
Send message
Joined: 29 Feb 00
Posts: 15993
Credit: 690,597
RAC: 0
United States
Message 535702 - Posted: 23 Mar 2007, 22:40:25 UTC - in response to Message 535686.
Last modified: 23 Mar 2007, 22:41:59 UTC


. . . Nice work there Matt / Berkeley yo 4 bruno ;)

Somebody break out a calculator and determine how many SETI@home classic clients this represents


. . . callin' Esme' ;)


What am I? The secretary? :p


Secretary? NOPE - 'NOT' what i was thinkin' - i refer to Matt's Request which i have hope / belief - in the Positive Sense of the Word - that you may be able to assist with some 'Math' related (see Matt's Post) . . . Michael Roberts has started the 'ball rollin' per se . . . see IF it's something YOU could help out on . . . that's all (after all - YOU are GOOD with Math aren't you?) mi thinks so ;)

With Sincerity,

richard

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 535722 - Posted: 23 Mar 2007, 23:35:48 UTC - in response to Message 535686.


. . . Nice work there Matt / Berkeley yo 4 bruno ;)

Somebody break out a calculator and determine how many SETI@home classic clients this represents


. . . callin' Esme' ;)


What am I? The secretary? :p

I was thinking "Executive Assistant".... <ducking>
____________

Profile Clyde C. Phillips, III
Send message
Joined: 2 Aug 00
Posts: 1851
Credit: 5,955,047
RAC: 0
United States
Message 536128 - Posted: 24 Mar 2007, 18:22:47 UTC

What's the significance of "Esme"? I see the green graph. Maybe somebody in this thread likes a Hispanic girl named "Esmeralda", which is Spanish for "emerald". Back in 1998 I saw parts of a Mexican soap opera "telenovela" on the TV channel "Univisión" with that title, starring Fernando Colunga y Leticia Calderón.
____________

Profile Dr. C.E.T.I.
Avatar
Send message
Joined: 29 Feb 00
Posts: 15993
Credit: 690,597
RAC: 0
United States
Message 536268 - Posted: 24 Mar 2007, 20:16:51 UTC - in response to Message 536128.

What's the significance of "Esme"? I see the green graph. Maybe somebody in this thread likes a Hispanic girl named "Esmeralda", which is Spanish for "emerald". Back in 1998 I saw parts of a Mexican soap opera "telenovela" on the TV channel "Univisión" with that title, starring Fernando Colunga y Leticia Calderón.


Clyde - it's Es99 - 'er name is Esme' (i believe i 'ave the spellin' correct?)

John McLeod VII
Volunteer developer
Volunteer tester
Avatar
Send message
Joined: 15 Jul 99
Posts: 24381
Credit: 519,750
RAC: 37
United States
Message 536548 - Posted: 25 Mar 2007, 3:39:00 UTC - in response to Message 535269.

Somebody break out a calculator and determine how many SETI@home classic clients this represents


Here's my response from the Café - can anyone improve the estimated input values or suggest a better model?
Well I'll start the ball rolling. There are two issues here - first a suitable model of the traffic origins and second, sensible input numbers.

From inspection of the graph I assume 0.7Mbit/S average shserver2 traffic, with 33% generated hourly and the rest generated daily. Reasonable choices of such values could affect the answer by factors of 2 or so and are thus not too critical for a first estimate.

From this conversation I will take an average packet size of 40 bytes (given that no actual data is being sent anywhere) This gives a little over 2000 packets per second and just under 200 million per day. I also assume that each contact results in 4 packets. But I really don't know in either case. The equality of input and output traffic rates suggests a strong minimum packet size effect.

Using a formula

numberofhosts = packetsperday/(hourlyfraction*24+(1-hourlyfraction))/packetspercontact


we then get 5768000 hosts (rounded a bit,) which at just under 6 million seems rather high, but it is more than a thousand and less than a billion, so it gives us something to talk about.

OK, call it 5 million connections. That does not equate to 5 million hosts. Assume that each host contacts the server 10 times per hour and you get around 25000 hosts instead. The 10 contacts per hour was a left handed extraction.
____________


BOINC WIKI

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 536565 - Posted: 25 Mar 2007, 5:23:44 UTC - in response to Message 536548.

Somebody break out a calculator and determine how many SETI@home classic clients this represents


Here's my response from the Café - can anyone improve the estimated input values or suggest a better model?
Well I'll start the ball rolling. There are two issues here - first a suitable model of the traffic origins and second, sensible input numbers.

From inspection of the graph I assume 0.7Mbit/S average shserver2 traffic, with 33% generated hourly and the rest generated daily. Reasonable choices of such values could affect the answer by factors of 2 or so and are thus not too critical for a first estimate.

From this conversation I will take an average packet size of 40 bytes (given that no actual data is being sent anywhere) This gives a little over 2000 packets per second and just under 200 million per day. I also assume that each contact results in 4 packets. But I really don't know in either case. The equality of input and output traffic rates suggests a strong minimum packet size effect.

Using a formula

numberofhosts = packetsperday/(hourlyfraction*24+(1-hourlyfraction))/packetspercontact


we then get 5768000 hosts (rounded a bit,) which at just under 6 million seems rather high, but it is more than a thousand and less than a billion, so it gives us something to talk about.

OK, call it 5 million connections. That does not equate to 5 million hosts. Assume that each host contacts the server 10 times per hour and you get around 25000 hosts instead. The 10 contacts per hour was a left handed extraction.

If there is a host at the IP, listening on the right port, you'd have six packets minimum per connection.

If nothing is listening, you'd have several "syn" packets per host that would go unanswered. I'd have to read a couple of RFCs to figure out the likely number.
____________

Profile Uioped1
Volunteer tester
Avatar
Send message
Joined: 17 Sep 03
Posts: 50
Credit: 1,179,926
RAC: 0
United States
Message 540930 - Posted: 4 Apr 2007, 18:05:49 UTC - in response to Message 534260.

Well, here's some anecdotal evidence:
I probably have somewhere between 50 and 100 hosts still trying to get some classic workunits.

Those were from previous jobs, where I had SAH running as a windows service in the standard workstation build. I had permission to install those clients, but I was the helpdesk and I didn't want to receive complaints from users about their processor utilization, so I hid the process. It wouldn't suprise me if whoever is running IT at those places doesn't know or doesn't care about it. Those workstations will likely keep attempting to pull work until they're finally retired or rebuilt.

Message boards : Technical News : Echoes of Classic (Mar 20 2007)

Copyright © 2014 University of California