Tweenday One (Dec 26 2007)


log in

Advanced search

Message boards : Technical News : Tweenday One (Dec 26 2007)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 694838 - Posted: 27 Dec 2007, 0:05:28 UTC

The weekend was a difficult as we kept splitting noisy/fast work, so our back-end production was running full speed most of the time, clogging several pipes, filling some queues, emptying others, etc. We were able to keep reaching our current outbound ceiling of 60 Mbits/sec, so despite the problems we were sending out work as fast as we could otherwise. That's good, but bigger pipes would be better. Also one of the assimilators was failing on a particular result. We're not sure why, but I deleted that one result and that particular dam broke. Some untested forum code was put on line which also wreaked minor havoc. Not my fault.

Anyway.. this is a short mini week for us in between Xmas/New Year's. Since we weren't around yesterday, we had our normal weekly outage today. Also took care of cleaning some extra "bloat" in our database. About 20% of the rows in the host table were hosts that last connected over a year ago and ultimately never got any credit. We blitzed all those.

Upon restarting everything this afternoon after the outage I noticed the feeder executables had disappeared sometime around 3-4 days ago (luckily images of the executables remained in memory since we had no downtime over the weekend). We have snapshots on that filesystem so recovery was instantaneous, but the initial disappearance is mysterious and a bit troubling.

- Matt

____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Dr. C.E.T.I.
Avatar
Send message
Joined: 29 Feb 00
Posts: 15993
Credit: 690,597
RAC: 0
United States
Message 694858 - Posted: 27 Dec 2007, 1:29:29 UTC


A Job Well Done Sir . . . nice going Matt (and the others @ the lab)

note: Favourite day of the week:

Tweenday, the magical day between Tuesday and Wednesday, when those of us with faerie blood flit around the sleeping earth righting wrongs and playing pranks. We give leg-cramps to the wicked, lovely dreams to the virtuous but unhappy, and stifle in the crib those babies who are likely to grow up to send e-mail questionnaires asking things like 'What is your favourite day of the week.' © 2007 Michael Horatio Kelly

;)



____________
BOINC Wiki . . .

Science Status Page . . .

Gene Barnhardt
Send message
Joined: 27 May 00
Posts: 3
Credit: 255,307
RAC: 0
United States
Message 694876 - Posted: 27 Dec 2007, 2:20:52 UTC

Looks like project server may be down - have been receiving the following msg for quite some time now. 12/26/2007 9:13:50 PM||Access to reference site succeeded - project servers may be temporarily down.
Help......Running out of work.

Profile Neil Blaikie
Volunteer tester
Avatar
Send message
Joined: 17 May 99
Posts: 142
Credit: 6,466,200
RAC: 0
Canada
Message 694879 - Posted: 27 Dec 2007, 2:26:50 UTC

Excellent job as always.

Have a shot of my large Jack Daniels bottle that Santa deposited in my stocking.

You have earned it guys!

Thank you.
____________

WinterKnight
Volunteer tester
Send message
Joined: 18 May 99
Posts: 8635
Credit: 23,739,815
RAC: 18,940
United Kingdom
Message 694881 - Posted: 27 Dec 2007, 2:29:49 UTC - in response to Message 694876.

Looks like project server may be down - have been receiving the following msg for quite some time now. 12/26/2007 9:13:50 PM||Access to reference site succeeded - project servers may be temporarily down.
Help......Running out of work.

The servers are just getting back up to speed, after the 24hr delayed Tuesday maintenance period, see cricket graphs

DJStarfox
Send message
Joined: 23 May 01
Posts: 1040
Credit: 544,758
RAC: 239
United States
Message 695085 - Posted: 27 Dec 2007, 17:57:31 UTC - in response to Message 694838.

Upon restarting everything this afternoon after the outage I noticed the feeder executables had disappeared sometime around 3-4 days ago (luckily images of the executables remained in memory since we had no downtime over the weekend). We have snapshots on that filesystem so recovery was instantaneous, but the initial disappearance is mysterious and a bit troubling.


Missing files like that prompt me to make an immediate fsck on the filesystem. Too many crashes under my belt.

I hope you had a Merry Christmas.

PhonAcq
Send message
Joined: 14 Apr 01
Posts: 1622
Credit: 22,108,511
RAC: 3,811
United States
Message 695095 - Posted: 27 Dec 2007, 18:12:07 UTC - in response to Message 694881.

Looks like project server may be down - have been receiving the following msg for quite some time now. 12/26/2007 9:13:50 PM||Access to reference site succeeded - project servers may be temporarily down.
Help......Running out of work.

The servers are just getting back up to speed, after the 24hr delayed Tuesday maintenance period, see cricket graphs


Why are the bits 'in' larger than the bits 'out'? I would have thought it would be the other way. Isn't the downloading of wu's the greatest demand on the network?

msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38925
Credit: 579,126,052
RAC: 512,115
United States
Message 695099 - Posted: 27 Dec 2007, 18:17:35 UTC - in response to Message 695095.

Looks like project server may be down - have been receiving the following msg for quite some time now. 12/26/2007 9:13:50 PM||Access to reference site succeeded - project servers may be temporarily down.
Help......Running out of work.

The servers are just getting back up to speed, after the 24hr delayed Tuesday maintenance period, see cricket graphs


Why are the bits 'in' larger than the bits 'out'? I would have thought it would be the other way. Isn't the downloading of wu's the greatest demand on the network?

Dunno.....but there are different versions of the Cricket Graph..try this one.
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Josef W. SegurProject donor
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4230
Credit: 1,043,255
RAC: 311
United States
Message 695213 - Posted: 28 Dec 2007, 0:12:35 UTC - in response to Message 695095.

...see cricket graphs


Why are the bits 'in' larger than the bits 'out'? I would have thought it would be the other way. Isn't the downloading of wu's the greatest demand on the network?

Because the monitoring point is the campus end of the link from Space Sciences Lab. The bits 'in' to your computer (WU downloads) is also larger than the bits 'out' (result uploads) for the same reason.
Joe

SockGap
Send message
Joined: 16 Apr 07
Posts: 13
Credit: 5,819,043
RAC: 2,768
Australia
Message 695215 - Posted: 28 Dec 2007, 0:17:09 UTC - in response to Message 695095.

Looks like project server may be down - have been receiving the following msg for quite some time now. 12/26/2007 9:13:50 PM||Access to reference site succeeded - project servers may be temporarily down.
Help......Running out of work.

The servers are just getting back up to speed, after the 24hr delayed Tuesday maintenance period, see cricket graphs


Why are the bits 'in' larger than the bits 'out'? I would have thought it would be the other way. Isn't the downloading of wu's the greatest demand on the network?


Because the graph is taken from the point of view of the router. So for that interface the "bits in" are coming "in" from the SETI servers and the "bits out" are the ones being sent back to the SETI servers. If they were monitoring the port on the switch on the other end of that piece of cable (or fibre) then the graph would be reversed.

(Long time reader - first time poster. Finally I can add something to the forum!)

____________

seti@elrcastor.com
Volunteer tester
Send message
Joined: 30 Jan 00
Posts: 35
Credit: 4,868,442
RAC: 0
United States
Message 695369 - Posted: 28 Dec 2007, 17:03:33 UTC

the interface gigabitethernet2_3 is looking towards the seti servers so packets in and packets out are reversed

the interface gigabitethernet5_1 is looking at the internet so packets in and packets out are normal
____________

msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38925
Credit: 579,126,052
RAC: 512,115
United States
Message 695396 - Posted: 28 Dec 2007, 19:08:28 UTC - in response to Message 695369.

the interface gigabitethernet2_3 is looking towards the seti servers so packets in and packets out are reversed

the interface gigabitethernet5_1 is looking at the internet so packets in and packets out are normal

Thank you very much for the new Cricket Graph reference.....
Much easier to make sense of. I may start to link to that page from now on.
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Josef W. SegurProject donor
Volunteer developer
Volunteer tester
Send message
Joined: 30 Oct 99
Posts: 4230
Credit: 1,043,255
RAC: 311
United States
Message 695442 - Posted: 28 Dec 2007, 21:58:43 UTC - in response to Message 695396.

the interface gigabitethernet2_3 is looking towards the seti servers so packets in and packets out are reversed

the interface gigabitethernet5_1 is looking at the internet so packets in and packets out are normal

Thank you very much for the new Cricket Graph reference.....
Much easier to make sense of. I may start to link to that page from now on.

Don't be surprised if after the holidays there's a lot of non-SETI traffic on that page.
Joe

msattlerProject donor
Volunteer tester
Avatar
Send message
Joined: 9 Jul 00
Posts: 38925
Credit: 579,126,052
RAC: 512,115
United States
Message 695450 - Posted: 28 Dec 2007, 22:16:53 UTC - in response to Message 695442.


Don't be surprised if after the holidays there's a lot of non-SETI traffic on that page.
Joe

Thanx for the heads up. I'll compare the two next year and see what happens....
____________
*********************************************
Embrace your inner kitty...ya know ya wanna!

I have met a few friends in my life.
Most were cats.

Message boards : Technical News : Tweenday One (Dec 26 2007)

Copyright © 2014 University of California