Retreat! (Jun 24 2009)

Message boards : Technical News : Retreat! (Jun 24 2009)
Message board moderation

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
HAL

Send message
Joined: 28 Mar 03
Posts: 704
Credit: 870,617
RAC: 0
United States
Message 911331 - Posted: 25 Jun 2009, 18:31:48 UTC

I guess there are two ways to look at the events of the past few days
#1 - Into Every Life some rain must fall.
#2 - How long does typhoon season last?


Classic WU= 7,237 Classic Hours= 42,079
ID: 911331 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 911344 - Posted: 25 Jun 2009, 19:21:29 UTC - in response to Message 911331.  

I guess there are two ways to look at the events of the past few days
#1 - Into Every Life some rain must fall.
#2 - How long does typhoon season last?

To push an analogy....

If you live in a desert, a light sprinkling of rain is a huge event. If you live in the Pacific Northwest (Washington/Oregon) on the coast, a little rain is expected most every day.
ID: 911344 · Report as offensive
Profile Dirk Sadowski
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 911346 - Posted: 25 Jun 2009, 19:28:18 UTC
Last modified: 25 Jun 2009, 19:32:44 UTC


I had ~ 1,000 WUs @ download.

All with errors, wrong size, http, or what ever error.

A restart BOINC/reboot PC didn't helped.
After ~ 24 hours.. and after the 10. reboot of the PC.. ~ 50 WUs downloaded.

Then again.. errors.. errors.. errors..

Next reboot of PC.. again ~ 50 WUs downloaded.

Then again.. errors.. errors.. errors..

Now it's not possible to upload..
Also all the last ~ 900 WUs are in the download overview with errors.

BTW.
I have only set a cache of 3 days.
The GPU cruncher would need ~ 800 MB AR=0.4x WUs / day.

ID: 911346 · Report as offensive
Berserker
Volunteer tester

Send message
Joined: 2 Jun 99
Posts: 105
Credit: 5,440,087
RAC: 0
United Kingdom
Message 911363 - Posted: 25 Jun 2009, 20:17:20 UTC

Re: The download situation

Apache is notoriously heavyweight when it comes to system resources. If/when the current firestorm blows through, it might be worth looking around to see if something else might be capable of filling Apache's boots, so to speak.

My suggestion - lighttpd.

PS - Yes, I do realize this might not be an easy transition to make, and you may even have looked at it before - so maybe just another entry on the ever-growing "to do" list. :)
Stats site - http://www.teamocuk.co.uk - still alive and (just about) kicking.
ID: 911363 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21227
Credit: 7,508,002
RAC: 20
United Kingdom
Message 911364 - Posted: 25 Jun 2009, 20:17:25 UTC - in response to Message 910837.  

... going to one download server wasn't a good thing. It seems impossible to me that apache can't handle all the downloads on one system - especially given the abundance of free resources. It drops connections regardless of how much network/httpd.conf tweaking I do. So we fell back to using two download servers, and that immediately solved everything. ...

OK, folks:

What suggestions for what should be tweaked to at least double the number of connections that can be serviced?

Or where to look for what limits might be getting hit?


My first guesses:

In the OS itself, set ulimit and nofile (number of file descriptors) for apache to x4 their present values and see if that helps without overloading anything? nproc also? Also check whether net.ipv4.tcp_keepalive_time is short enough so as not to strangle the server with old stale dropped connections for too long?


Others?

Regards,
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 911364 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 911372 - Posted: 25 Jun 2009, 20:53:18 UTC - in response to Message 911325.  
Last modified: 25 Jun 2009, 21:06:18 UTC

>O'Reilly open source convention

WOW! Seti/OpenSource are sponsoring an O'Reily race car? Kewl! Does it run really, really fast but require a complete engine replacement every lap?


Nah, that would be if it were running Windows. You'd have to replace the
engine and call Microsoft to make sure you're authorized to activate/restart the engine.

With open source, you have to stop in the pitstop every lap, have a team of about 2,000 people check the engine, make contributions, call Linus Torvalds if you want to make any changes to the engine's kernel, recompile the engine, find the conflicting installed libraries, suggest running the engine without the fancy hood (GUI), argue over what type of hood to run on the engine: the one that looks like Microsoft's car or the one that looks like Mac's car, each of the 2,000 technicians would have to hurry to the stands looking for additional support on the arcane structure of the Open Source Engine (OSE), give up the entire process, slap some bigger tires on the car and tell the driver to take off. Then, the OSE is only allowed to run on the outer track of the racetrack unless the technicians installed WINE, which doesn't technically "emulate" the Microsoft car, but allows you to use the third-inner track (because the second inner track is used by Mac). Changing the tires can be done while the OSE is still running so that you can brag about your "uptime".

By the time the OSE car gets out of the pitstop for the second time, the race is over and the Microsoft car has won because it simply works and does everything you need it to do.

The Mac car only loses because they are too snobbish to care about winning. They don't want to win; they don't need to win. Winning is so passe when you have such a fine machine that's obviously better than everyone else's, even if it is now using the same framework as the Microsoft car, its still better because of its more stylish hood (GUI), which was borrowed from a previous iteration of the OSE car. And every time someone puts a really neat gadget inside the car for the driver, Steve Jobs gets angry and has it removed and crushed on the concrete right before the owner's eyes as they are thrown out of the raceway by very stylish bodyguards who are clearly Macs and not those dorky PCs. In fact, the Mac car even hires two people to stand out front and try to convince everyone how second place is the new first place, and that only dorky looking people win first place while all the cool people take second and rip on first.
ID: 911372 · Report as offensive
Profile Kibble (KB7TIB)
Avatar

Send message
Joined: 6 Dec 99
Posts: 27
Credit: 10,121,469
RAC: 2
United States
Message 911375 - Posted: 25 Jun 2009, 20:56:56 UTC

"Of course, we've been offline for 24 hours, so there's gonna be lots of traffic for a while making it hard to upload/download anything."

All right! For those of us who ran out of work and/or turned off our computers but are now getting work units downloaded. I noticed that a fair amount of the incoming wu's were going to be due on July 1st. Will there be an extension of time for those?

Great work on bringing the system to heel, Matt. Hopefully there won't be any outage over the long July 4th holiday.

Kibble
ID: 911375 · Report as offensive
Garry Webb

Send message
Joined: 25 Aug 99
Posts: 40
Credit: 13,561,408
RAC: 0
United States
Message 911404 - Posted: 25 Jun 2009, 21:38:30 UTC - in response to Message 911260.  

Worked for one time through the waiting uploads for me! Now it has a retry on uploads on all machines.

Even updated to latest edition of BOINC. Oh Well! Previous paragraph got my machines the max in downloads, so I'll just wait for the uploads to cycle through automatically.
ID: 911404 · Report as offensive
Profile Robi

Send message
Joined: 24 Oct 00
Posts: 33
Credit: 886,890
RAC: 1
United States
Message 911445 - Posted: 25 Jun 2009, 22:54:59 UTC - in response to Message 911404.  

Worked for one time through the waiting uploads for me! Now it has a retry on uploads on all machines.

Even updated to latest edition of BOINC. Oh Well! Previous paragraph got my machines the max in downloads, so I'll just wait for the uploads to cycle through automatically.


I used to have the problem with the downloads, now it's the uploads :(
ah, well, it'll eventually clear up (I hope :)
Robi
ID: 911445 · Report as offensive
Berserker
Volunteer tester

Send message
Joined: 2 Jun 99
Posts: 105
Credit: 5,440,087
RAC: 0
United Kingdom
Message 911465 - Posted: 26 Jun 2009, 0:09:12 UTC - in response to Message 911364.  

What suggestions for what should be tweaked to at least double the number of connections that can be serviced?

...

Others?

I posted about five seconds before you wish a likely candidate. Apache itself is a bit of a "jack of all trades and master of none" - there are more efficient open source web servers.
Stats site - http://www.teamocuk.co.uk - still alive and (just about) kicking.
ID: 911465 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 912694 - Posted: 29 Jun 2009, 20:15:01 UTC

I find it painfully ridiculous that people complain about off topic posts when it comes to Martin and I sparring over Microsoft and Open Source, yet no one complains when the topic is "Old IBM Mainfraim computers from the 1960's" or "Printers that are older than Matt Lebofsky", which is just as off topic in the Technical News threads as anything else.
ID: 912694 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 912696 - Posted: 29 Jun 2009, 20:22:41 UTC - in response to Message 912694.  

I find it painfully ridiculous that people complain about off topic posts when it comes to Martin and I sparring over Microsoft and Open Source, yet no one complains when the topic is "Old IBM Mainfraim computers from the 1960's" or "Printers that are older than Matt Lebofsky", which is just as off topic in the Technical News threads as anything else.

... and I'm deeply bothered by the fact that my posts were moved to "Politics" which is an area that I try very hard to stay as far away from as possible.
ID: 912696 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 912702 - Posted: 29 Jun 2009, 20:41:40 UTC - in response to Message 912696.  

I moved the thread to Number Crunching after a general consensus from the other Mods.
ID: 912702 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 912718 - Posted: 29 Jun 2009, 21:42:51 UTC - in response to Message 912702.  

I moved the thread to Number Crunching after a general consensus from the other Mods.

I appreciate it. It didn't belong with "Cannabis vs. Tobacco" and "Single Payer Health Care."

If there was a section for "Religion" it would have been appropriate there.
ID: 912718 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21227
Credit: 7,508,002
RAC: 20
United Kingdom
Message 912720 - Posted: 29 Jun 2009, 22:05:41 UTC - in response to Message 912694.  

... yet no one complains when the topic is ... "Printers that are older than Matt Lebofsky" ...

Oooer... You don't mean old printers and computers that can play music better than Matt?...

zx80 music

EEeeeeeOoooooops OT!

Poof!

;-)

(Special note: This is very ON-topic being as it includes Matt and his music :-p )

Cheers,
Martin


ps: Sorry about the "#include" pun!


See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 912720 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21227
Credit: 7,508,002
RAC: 20
United Kingdom
Message 912721 - Posted: 29 Jun 2009, 22:07:55 UTC - in response to Message 912718.  

If there was a section for "Religion" it would have been appropriate there.

Unfortunately I'd have to oncemore agree...

:-(

Cheers,
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 912721 · Report as offensive
Previous · 1 · 2

Message boards : Technical News : Retreat! (Jun 24 2009)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.