Oopsie (Oct 06 2009)

Message boards : Technical News : Oopsie (Oct 06 2009)

Author	Message
Matt Lebofsky Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 1 Mar 99 Posts: 1375 Credit: 74,079 RAC: 0	Message 938002 - Posted: 6 Oct 2009, 22:43:01 UTC
	Quick post-weekly-outage wrapup: everything went fine, albeit a little slow given recent events. The replica recovery is going on now. Hopefully it'll continue along safely overnight and we can turn the replica back on sometime tomorrow. One hilarious note. All our server reboots over the weekend dislodged several instances of sendmail, which then went on to send forth unexpectedly large queues of cronjob/server related e-mails to me, Jeff, and Eric. We're talking about 35,000 e-mails, all of which went through the lab spam firewall first, thus clobbering everybody's e-mail in the entire Space Lab for about 24 hours. Fun. - Matt ____________ -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
	ID: 938002 ·

Pablo_ARG Send message Joined: 13 Aug 03 Posts: 12 Credit: 2,041,544 RAC: 0	Message 938008 - Posted: 6 Oct 2009, 23:07:58 UTC
	The weekend outage is finally repaired. Thanks for the update. PD: 35.000 e-mails in two days ? ... servers cries loud :) ____________
	ID: 938008 ·

LiliKrist Volunteer tester Send message Joined: 12 Aug 09 Posts: 333 Credit: 143,167 RAC: 0	Message 938026 - Posted: 7 Oct 2009, 1:20:48 UTC
	^U^ ____________ N = R x fp x ne x fl x fi x fc x L
	ID: 938026 ·

Johnney Guinness Volunteer tester Send message Joined: 11 Sep 06 Posts: 2944 Credit: 2,305,966 RAC: 330	Message 938035 - Posted: 7 Oct 2009, 2:19:15 UTC Last modified: 7 Oct 2009, 2:20:04 UTC
	I start sweating if my mail box says "You have 5 new messages". With 35,000 emails, you must have thought you were Mr. Popular for a few moments Matt...LOL John. ____________
	ID: 938035 ·

uli Volunteer tester Send message Joined: 6 Feb 00 Posts: 8029 Credit: 3,764,138 RAC: 3,747	Message 938041 - Posted: 7 Oct 2009, 2:54:05 UTC
	Thanks Matt, that explains a few things. ____________ Pluto will always be a planet to me. Cash Donation Specialist
	ID: 938041 ·

John McLeod VII Volunteer developer Volunteer tester Send message Joined: 15 Jul 99 Posts: 22343 Credit: 439,209 RAC: 396	Message 938042 - Posted: 7 Oct 2009, 2:55:14 UTC - in response to Message 938002.
	Quick post-weekly-outage wrapup: everything went fine, albeit a little slow given recent events. The replica recovery is going on now. Hopefully it'll continue along safely overnight and we can turn the replica back on sometime tomorrow. One hilarious note. All our server reboots over the weekend dislodged several instances of sendmail, which then went on to send forth unexpectedly large queues of cronjob/server related e-mails to me, Jeff, and Eric. We're talking about 35,000 e-mails, all of which went through the lab spam firewall first, thus clobbering everybody's e-mail in the entire Space Lab for about 24 hours. Fun. - Matt THAT explains the delayed emails to the BOINC Help Desk and the S@H moderators email list. ____________ BOINC WIKI
	ID: 938042 ·

Arvid Almstrom Send message Joined: 23 Mar 00 Posts: 86 Credit: 126,111,595 RAC: 78,608	Message 938053 - Posted: 7 Oct 2009, 3:48:52 UTC
	I do like the 35,000 e-mails, no doubt causing all sorts of problems for the administrators, maybe your were not the only once dragged back to work over the weekend. One another note, the main page for technical notes is getting quite large. I was on a slow wireless broadband link stuck on 2G, when I discovered that this page is huge on a slow connection. Any chance of truncating say the first couple years. If anyone would like access to the older stuff maybe a second page which includes the removed comments. Arvid ____________ Arvid Almstrom
	ID: 938053 ·

Josef W. Segur Volunteer developer Volunteer tester Send message Joined: 30 Oct 99 Posts: 3932 Credit: 934,327 RAC: 15	Message 938068 - Posted: 7 Oct 2009, 5:24:48 UTC - in response to Message 938053.
	... One another note, the main page for technical notes is getting quite large. I was on a slow wireless broadband link stuck on 2G, when I discovered that this page is huge on a slow connection. Any chance of truncating say the first couple years. If anyone would like access to the older stuff maybe a second page which includes the removed comments. Arvid If you get to the bottom of that page, you discover that several older years have already been split off into separate archives. Being on dial-up, I'd also find it easier to handle if it were split again. But I'd put it in the "nice to have" class, something for a day when more important work is blocked for some reason. Joe
	ID: 938068 ·

Speedy Volunteer tester Send message Joined: 26 Jun 04 Posts: 579 Credit: 3,499,656 RAC: 8,914	Message 938085 - Posted: 7 Oct 2009, 8:27:48 UTC Last modified: 7 Oct 2009, 8:32:20 UTC
	Thank you to the people came up to the lab to try and the servers working again. Am I correct in saying that all the fresh tapes full of data that are left have been loaded into the splitters? Thanks for reading ____________ Live in NZ y not join Smile City?
	ID: 938085 ·

John McCallum Send message Joined: 5 Dec 04 Posts: 793 Credit: 216,619 RAC: 182	Message 938126 - Posted: 7 Oct 2009, 14:19:27 UTC
	'We're talking about 35,000 e-mails'and every one or most will have to be checked,at least the message titles to ensure that nothing important is missed some one is going to be very popular lol ____________ To get back my youth I would do any thing in the world EXCEPT take Exercize,GET up early or be RESPECTABLE.
	ID: 938126 ·

Speedy Volunteer tester Send message Joined: 26 Jun 04 Posts: 579 Credit: 3,499,656 RAC: 8,914	Message 938184 - Posted: 7 Oct 2009, 20:50:46 UTC
	I was taking a look at http://setiathome.berkeley.edu/sah_status.html & noticed that a few ap & mb splitters are disabled. I also noticed that there's a new Program running on bruno called Vote_Monitor. Does anyone know what the vote monitor does? I couldn't ever imagine e-mails in my in box were the e-mails telling Matt Jeff & Eric that the the servers were looking up? ____________ Live in NZ y not join Smile City?
	ID: 938184 ·

Matt Lebofsky Volunteer moderator Project administrator Project developer Project scientist Send message Joined: 1 Mar 99 Posts: 1375 Credit: 74,079 RAC: 0	Message 938187 - Posted: 7 Oct 2009, 20:56:26 UTC - in response to Message 938184.
	We have a lot of splitters listed in our back-end configuration file - more than we usually need (or can handle) - which is why several are disabled at any given time. If we need to start more (or less), it's an easy toggle within the config file. The vote monitor is a script that regularly check the state of and handles elections for user banishment on the forums. I was taking a look at http://setiathome.berkeley.edu/sah_status.html & noticed that a few ap & mb splitters are disabled. I also noticed that there's a new Program running on bruno called Vote_Monitor. Does anyone know what the vote monitor does? I couldn't ever imagine e-mails in my in box were the e-mails telling Matt Jeff & Eric that the the servers were looking up? - Matt ____________ -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
	ID: 938187 ·

52 Aces Send message Joined: 7 Jan 02 Posts: 494 Credit: 11,304,102 RAC: 3,565	Message 938192 - Posted: 7 Oct 2009, 21:25:03 UTC - in response to Message 938187.
	The vote monitor is a script that regularly check the state of and handles elections for user banishment on the forums. Cool, how many fake logons under spoofed IP & MACs must I create to ban someone ? j/k ;-)
	ID: 938192 ·

Volunteer tester Send message Joined: 9 Apr 02 Posts: 11987 Credit: 18,237,262 RAC: 57,346	Message 938209 - Posted: 7 Oct 2009, 21:55:05 UTC - in response to Message 938192.
	The vote monitor is a script that regularly check the state of and handles elections for user banishment on the forums. Cool, how many fake logons under spoofed IP & MACs must I create to ban someone ? j/k ;-) LOL It won't work though. In order to vote you must have a bit set on your account indicating that you are a moderator for SETI@Home. You'd have to hack the server where user accounts are stored, find the appropriate flag and set it. Even then, the Admin(s) can override a ban vote and reverse it immediately, then quickly remove the flagged bit from the offending account. At that point, there's a strong chance that the account with the bit set would be reviewed and possibly perma-banned for such an infraction. ____________
	ID: 938209 ·

52 Aces Send message Joined: 7 Jan 02 Posts: 494 Credit: 11,304,102 RAC: 3,565	Message 938224 - Posted: 7 Oct 2009, 22:46:19 UTC - in response to Message 938209.
	Thx for the tech-splanation. That said, you have software & a thread running just for ban concensus !! ;-) Good thing I don't have a vote, cuz I'd vote to turn off the vote_monitor and run another instance of upload_server ;-) (I know, that one is probably wire bound anyhow)
	ID: 938224 ·

Volunteer tester Send message Joined: 9 Apr 02 Posts: 11987 Credit: 18,237,262 RAC: 57,346	Message 938225 - Posted: 7 Oct 2009, 22:58:42 UTC - in response to Message 938224.
	Thx for the tech-splanation. That said, you have software & a thread running just for ban concensus !! ;-) Good thing I don't have a vote, cuz I'd vote to turn off the vote_monitor and run another instance of upload_server ;-) (I know, that one is probably wire bound anyhow) If there are no active bans pending, then the process doesn't use any CPU cycles, thus negating a need to turn it off and create another upload_server instance. ;-) ____________
	ID: 938225 ·

Message boards : Technical News : Oopsie (Oct 06 2009)