Retreat! (Jun 24 2009)


log in

Advanced search

Message boards : Technical News : Retreat! (Jun 24 2009)

Previous · 1 · 2
Author Message
HAL
Send message
Joined: 28 Mar 03
Posts: 704
Credit: 870,617
RAC: 0
United States
Message 911331 - Posted: 25 Jun 2009, 18:31:48 UTC

I guess there are two ways to look at the events of the past few days
#1 - Into Every Life some rain must fall.
#2 - How long does typhoon season last?

____________

Classic WU= 7,237 Classic Hours= 42,079

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 911344 - Posted: 25 Jun 2009, 19:21:29 UTC - in response to Message 911331.

I guess there are two ways to look at the events of the past few days
#1 - Into Every Life some rain must fall.
#2 - How long does typhoon season last?

To push an analogy....

If you live in a desert, a light sprinkling of rain is a huge event. If you live in the Pacific Northwest (Washington/Oregon) on the coast, a little rain is expected most every day.
____________

Profile [seti.international] Dirk Sadowski
Volunteer tester
Avatar
Send message
Joined: 6 Apr 07
Posts: 6972
Credit: 57,200,853
RAC: 22,381
Germany
Message 911346 - Posted: 25 Jun 2009, 19:28:18 UTC
Last modified: 25 Jun 2009, 19:32:44 UTC


I had ~ 1,000 WUs @ download.

All with errors, wrong size, http, or what ever error.

A restart BOINC/reboot PC didn't helped.
After ~ 24 hours.. and after the 10. reboot of the PC.. ~ 50 WUs downloaded.

Then again.. errors.. errors.. errors..

Next reboot of PC.. again ~ 50 WUs downloaded.

Then again.. errors.. errors.. errors..

Now it's not possible to upload..
Also all the last ~ 900 WUs are in the download overview with errors.

BTW.
I have only set a cache of 3 days.
The GPU cruncher would need ~ 800 MB AR=0.4x WUs / day.

____________
BR



>Das Deutsche Cafe. The German Cafe.<

Berserker
Volunteer tester
Send message
Joined: 2 Jun 99
Posts: 105
Credit: 5,386,463
RAC: 0
United Kingdom
Message 911363 - Posted: 25 Jun 2009, 20:17:20 UTC

Re: The download situation

Apache is notoriously heavyweight when it comes to system resources. If/when the current firestorm blows through, it might be worth looking around to see if something else might be capable of filling Apache's boots, so to speak.

My suggestion - lighttpd.

PS - Yes, I do realize this might not be an easy transition to make, and you may even have looked at it before - so maybe just another entry on the ever-growing "to do" list. :)
____________
Stats site - http://www.teamocuk.co.uk - still alive and (just about) kicking.

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 7965
Credit: 4,014,392
RAC: 844
United Kingdom
Message 911364 - Posted: 25 Jun 2009, 20:17:25 UTC - in response to Message 910837.

... going to one download server wasn't a good thing. It seems impossible to me that apache can't handle all the downloads on one system - especially given the abundance of free resources. It drops connections regardless of how much network/httpd.conf tweaking I do. So we fell back to using two download servers, and that immediately solved everything. ...

OK, folks:

What suggestions for what should be tweaked to at least double the number of connections that can be serviced?

Or where to look for what limits might be getting hit?


My first guesses:

In the OS itself, set ulimit and nofile (number of file descriptors) for apache to x4 their present values and see if that helps without overloading anything? nproc also? Also check whether net.ipv4.tcp_keepalive_time is short enough so as not to strangle the server with old stale dropped connections for too long?


Others?

Regards,
Martin

____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13326
Credit: 27,943,019
RAC: 16,069
United States
Message 911372 - Posted: 25 Jun 2009, 20:53:18 UTC - in response to Message 911325.
Last modified: 25 Jun 2009, 21:06:18 UTC

>O'Reilly open source convention

WOW! Seti/OpenSource are sponsoring an O'Reily race car? Kewl! Does it run really, really fast but require a complete engine replacement every lap?


Nah, that would be if it were running Windows. You'd have to replace the
engine and call Microsoft to make sure you're authorized to activate/restart the engine.

With open source, you have to stop in the pitstop every lap, have a team of about 2,000 people check the engine, make contributions, call Linus Torvalds if you want to make any changes to the engine's kernel, recompile the engine, find the conflicting installed libraries, suggest running the engine without the fancy hood (GUI), argue over what type of hood to run on the engine: the one that looks like Microsoft's car or the one that looks like Mac's car, each of the 2,000 technicians would have to hurry to the stands looking for additional support on the arcane structure of the Open Source Engine (OSE), give up the entire process, slap some bigger tires on the car and tell the driver to take off. Then, the OSE is only allowed to run on the outer track of the racetrack unless the technicians installed WINE, which doesn't technically "emulate" the Microsoft car, but allows you to use the third-inner track (because the second inner track is used by Mac). Changing the tires can be done while the OSE is still running so that you can brag about your "uptime".

By the time the OSE car gets out of the pitstop for the second time, the race is over and the Microsoft car has won because it simply works and does everything you need it to do.

The Mac car only loses because they are too snobbish to care about winning. They don't want to win; they don't need to win. Winning is so passe when you have such a fine machine that's obviously better than everyone else's, even if it is now using the same framework as the Microsoft car, its still better because of its more stylish hood (GUI), which was borrowed from a previous iteration of the OSE car. And every time someone puts a really neat gadget inside the car for the driver, Steve Jobs gets angry and has it removed and crushed on the concrete right before the owner's eyes as they are thrown out of the raceway by very stylish bodyguards who are clearly Macs and not those dorky PCs. In fact, the Mac car even hires two people to stand out front and try to convince everyone how second place is the new first place, and that only dorky looking people win first place while all the cool people take second and rip on first.
____________

Profile Kibble (KB7TIB)
Avatar
Send message
Joined: 6 Dec 99
Posts: 21
Credit: 1,006,434
RAC: 1,743
United States
Message 911375 - Posted: 25 Jun 2009, 20:56:56 UTC

"Of course, we've been offline for 24 hours, so there's gonna be lots of traffic for a while making it hard to upload/download anything."

All right! For those of us who ran out of work and/or turned off our computers but are now getting work units downloaded. I noticed that a fair amount of the incoming wu's were going to be due on July 1st. Will there be an extension of time for those?

Great work on bringing the system to heel, Matt. Hopefully there won't be any outage over the long July 4th holiday.

Kibble

Garry Webb
Send message
Joined: 25 Aug 99
Posts: 40
Credit: 12,982,736
RAC: 1,492
United States
Message 911404 - Posted: 25 Jun 2009, 21:38:30 UTC - in response to Message 911260.

Worked for one time through the waiting uploads for me! Now it has a retry on uploads on all machines.

Even updated to latest edition of BOINC. Oh Well! Previous paragraph got my machines the max in downloads, so I'll just wait for the uploads to cycle through automatically.
____________

Profile Robi
Send message
Joined: 24 Oct 00
Posts: 31
Credit: 236,867
RAC: 399
United States
Message 911445 - Posted: 25 Jun 2009, 22:54:59 UTC - in response to Message 911404.

Worked for one time through the waiting uploads for me! Now it has a retry on uploads on all machines.

Even updated to latest edition of BOINC. Oh Well! Previous paragraph got my machines the max in downloads, so I'll just wait for the uploads to cycle through automatically.


I used to have the problem with the downloads, now it's the uploads :(
ah, well, it'll eventually clear up (I hope :)
____________
Robi

Berserker
Volunteer tester
Send message
Joined: 2 Jun 99
Posts: 105
Credit: 5,386,463
RAC: 0
United Kingdom
Message 911465 - Posted: 26 Jun 2009, 0:09:12 UTC - in response to Message 911364.

What suggestions for what should be tweaked to at least double the number of connections that can be serviced?

...

Others?

I posted about five seconds before you wish a likely candidate. Apache itself is a bit of a "jack of all trades and master of none" - there are more efficient open source web servers.
____________
Stats site - http://www.teamocuk.co.uk - still alive and (just about) kicking.

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13326
Credit: 27,943,019
RAC: 16,069
United States
Message 912694 - Posted: 29 Jun 2009, 20:15:01 UTC

I find it painfully ridiculous that people complain about off topic posts when it comes to Martin and I sparring over Microsoft and Open Source, yet no one complains when the topic is "Old IBM Mainfraim computers from the 1960's" or "Printers that are older than Matt Lebofsky", which is just as off topic in the Technical News threads as anything else.
____________

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 912696 - Posted: 29 Jun 2009, 20:22:41 UTC - in response to Message 912694.

I find it painfully ridiculous that people complain about off topic posts when it comes to Martin and I sparring over Microsoft and Open Source, yet no one complains when the topic is "Old IBM Mainfraim computers from the 1960's" or "Printers that are older than Matt Lebofsky", which is just as off topic in the Technical News threads as anything else.

... and I'm deeply bothered by the fact that my posts were moved to "Politics" which is an area that I try very hard to stay as far away from as possible.
____________

OzzFan
Volunteer tester
Avatar
Send message
Joined: 9 Apr 02
Posts: 13326
Credit: 27,943,019
RAC: 16,069
United States
Message 912702 - Posted: 29 Jun 2009, 20:41:40 UTC - in response to Message 912696.

I moved the thread to Number Crunching after a general consensus from the other Mods.
____________

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 912718 - Posted: 29 Jun 2009, 21:42:51 UTC - in response to Message 912702.

I moved the thread to Number Crunching after a general consensus from the other Mods.

I appreciate it. It didn't belong with "Cannabis vs. Tobacco" and "Single Payer Health Care."

If there was a section for "Religion" it would have been appropriate there.
____________

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 7965
Credit: 4,014,392
RAC: 844
United Kingdom
Message 912720 - Posted: 29 Jun 2009, 22:05:41 UTC - in response to Message 912694.

... yet no one complains when the topic is ... "Printers that are older than Matt Lebofsky" ...

Oooer... You don't mean old printers and computers that can play music better than Matt?...

zx80 music

EEeeeeeOoooooops OT!

Poof!

;-)

(Special note: This is very ON-topic being as it includes Matt and his music :-p )

Cheers,
Martin


ps: Sorry about the "#include" pun!


____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 7965
Credit: 4,014,392
RAC: 844
United Kingdom
Message 912721 - Posted: 29 Jun 2009, 22:07:55 UTC - in response to Message 912718.

If there was a section for "Religion" it would have been appropriate there.

Unfortunately I'd have to oncemore agree...

:-(

Cheers,
Martin

____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Previous · 1 · 2

Message boards : Technical News : Retreat! (Jun 24 2009)

Copyright © 2014 University of California