Thursday Thunder (Jun 26 2008)

Message boards : Technical News : Thursday Thunder (Jun 26 2008)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 773948 - Posted: 26 Jun 2008, 21:07:44 UTC

The new scheduler continues to be handling its new duties just fine. Slowly but surely people are moving their connections over to this new server, but I'm not convinced the change rate is fast enough to do a whole sale cutover by next week. We shall see.

Funny aside: while getting new-ish donated server "clarke" up yesterday I was annoyed to find that Fedora Core 9 was booting to run level 5 (where it loads the X windowing environment). We don't need X on these servers, so we typically set our servers to boot to run level 3 via a change in /etc/inittab. In doing so, I'd comment out the old line with a "#" and enter in a new line with the adjusted run level. It was still booting up in X. Why? Turns out the latest inittab parser (new with FC9, I guess) ignores "#" comments in inittab, and just looks for lines containing the string "initdefault" and parses the first one it finds. Since I left the old line in there commented out (or so I thought) it was superseding the line I wanted. So much for standards (and clear documentation stating when/how standards change).

Nitpicker weirdness: While finally getting around to testing the few optimizations I made to Jeff's code I found that multiple runs of the nitpicker on the same pixel were producing slightly different results each time. We believe this is due to the order which the database pulls out rows - unless requested otherwise databases generally pull things out in random order, i.e. the order which requires the least I/O at that exact point in time (mostly due to page caching or where the many drive arms are currently located in our RAID set). Sorting query output adds significant (and usually unnecessary) overhead. But there are a lot of "fuzzy compares" in the nitpicker (due to floating point computations on different chips you can't expect decimal values to be "exactly exact"). When two items are close enough to be called "duplicates" you only need one, but which one you pick may cause different results down the road. So Jeff is elbow deep in this problem right now.

Apropos of nothing, the entire northern half of state of California is on fire. The smoke ending up here in the Bay Area is intense. I feel like I'm smoking a couple packs a day just walking around outside. I can smell it sitting here at my desk.

- Matt

-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 773948 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 774010 - Posted: 26 Jun 2008, 22:31:30 UTC


. . . was wonderin' IF that smoke was effectin' you @ Berkeley - that's too bad Sir!

Thanks for Posting Matt - we'll all keep our fingers 'crossed' for y'all


BOINC Wiki . . .

Science Status Page . . .
ID: 774010 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30608
Credit: 53,134,872
RAC: 32
United States
Message 774152 - Posted: 27 Jun 2008, 4:34:27 UTC - in response to Message 773948.  

The new scheduler continues to be handling its new duties just fine. Slowly but surely people are moving their connections over to this new server, but I'm not convinced the change rate is fast enough to do a whole sale cutover by next week. We shall see.

Funny aside: while getting new-ish donated server "clarke" up yesterday I was annoyed to find that Fedora Core 9 was booting to run level 5 (where it loads the X windowing environment). We don't need X on these servers, so we typically set our servers to boot to run level 3 via a change in /etc/inittab. In doing so, I'd comment out the old line with a "#" and enter in a new line with the adjusted run level. It was still booting up in X. Why? Turns out the latest inittab parser (new with FC9, I guess) ignores "#" comments in inittab, and just looks for lines containing the string "initdefault" and parses the first one it finds. Since I left the old line in there commented out (or so I thought) it was superseding the line I wanted. So much for standards (and clear documentation stating when/how standards change).

Nitpicker weirdness: While finally getting around to testing the few optimizations I made to Jeff's code I found that multiple runs of the nitpicker on the same pixel were producing slightly different results each time. We believe this is due to the order which the database pulls out rows - unless requested otherwise databases generally pull things out in random order, i.e. the order which requires the least I/O at that exact point in time (mostly due to page caching or where the many drive arms are currently located in our RAID set). Sorting query output adds significant (and usually unnecessary) overhead. But there are a lot of "fuzzy compares" in the nitpicker (due to floating point computations on different chips you can't expect decimal values to be "exactly exact"). When two items are close enough to be called "duplicates" you only need one, but which one you pick may cause different results down the road. So Jeff is elbow deep in this problem right now.

Apropos of nothing, the entire northern half of state of California is on fire. The smoke ending up here in the Bay Area is intense. I feel like I'm smoking a couple packs a day just walking around outside. I can smell it sitting here at my desk.

- Matt


Hey, in all seriousness if you can smell it, it can be doing damage. Just got over a bout of eating a bit too much smoke from a brush fire down in the southern part of the state a couple months ago. Worst part is if you are down wind and it burns through poison oak. Oh and it looks like a lot of the smoke from up there has blown down here!

As for that fuzzy logic lesson, just remember it all in Astropulse. I suspect it will be the very same problem.

Oh, thanks for the update.


ID: 774152 · Report as offensive
Profile SMW

Send message
Joined: 16 May 99
Posts: 22
Credit: 29,285,238
RAC: 16
United States
Message 774366 - Posted: 27 Jun 2008, 18:28:56 UTC

Anyone else having issues with the server responding back as:
[error] No start tag in scheduler reply

I have been getting this all day, even after a reboot and a restart of the program:( BOINC then sets it up to try again in a bit, with each but getting longer and longer between trys.
"It is better to be hated for what you are then to be loved for what you are not"
- Andre Gide (1869-1951)
ID: 774366 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 774478 - Posted: 27 Jun 2008, 22:39:44 UTC - in response to Message 774366.  

Anyone else having issues with the server responding back as:
[error] No start tag in scheduler reply

I have been getting this all day, even after a reboot and a restart of the program:( BOINC then sets it up to try again in a bit, with each but getting longer and longer between trys.

Have you read Wednesday Words (Jun 25 2008)? Trying resetting the project to change the download servers.

Cheers
Speedy
ID: 774478 · Report as offensive
Profile speedimic
Volunteer tester
Avatar

Send message
Joined: 28 Sep 02
Posts: 362
Credit: 16,590,653
RAC: 0
Germany
Message 775438 - Posted: 29 Jun 2008, 19:47:39 UTC

If you don't need it why install it?

Funny aside: while getting new-ish donated server "clarke" up yesterday I was annoyed to find that Fedora Core 9 was booting to run level 5 (where it loads the X windowing environment). We don't need X on these servers, so we typically set our servers to boot to run level 3 via a change in /etc/inittab. In doing so, I'd comment out the old line with a "#" and enter in a new line with the adjusted run level. It was still booting up in X. Why? Turns out the latest inittab parser (new with FC9, I guess) ignores "#" comments in inittab, and just looks for lines containing the string "initdefault" and parses the first one it finds. Since I left the old line in there commented out (or so I thought) it was superseding the line I wanted. So much for standards (and clear documentation stating when/how standards change).

mic.


ID: 775438 · Report as offensive
Profile RandyC
Avatar

Send message
Joined: 20 Oct 99
Posts: 714
Credit: 1,704,345
RAC: 0
United States
Message 775510 - Posted: 29 Jun 2008, 21:05:45 UTC - in response to Message 775438.  

Apparently it is a default part of the install package. Matt was trying to deactivate it but they changed standards on him and gotta gotcha!

If you don't need it why install it?

Funny aside: while getting new-ish donated server "clarke" up yesterday I was annoyed to find that Fedora Core 9 was booting to run level 5 (where it loads the X windowing environment). We don't need X on these servers, so we typically set our servers to boot to run level 3 via a change in /etc/inittab. In doing so, I'd comment out the old line with a "#" and enter in a new line with the adjusted run level. It was still booting up in X. Why? Turns out the latest inittab parser (new with FC9, I guess) ignores "#" comments in inittab, and just looks for lines containing the string "initdefault" and parses the first one it finds. Since I left the old line in there commented out (or so I thought) it was superseding the line I wanted. So much for standards (and clear documentation stating when/how standards change).

ID: 775510 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 775517 - Posted: 29 Jun 2008, 21:11:09 UTC - in response to Message 775510.  

Apparently it is a default part of the install package. Matt was trying to deactivate it but they changed standards on him and gotta gotcha!

If you don't need it why install it?

Also, much easier to leave it in place for keeping dependencies happy and for if you ever might want to play on a graphical display.

Setting the init level is simply easy and minimum maintenance.


In Linux, there are always many ways to do things. You can take your choice.

Happy crunchin',
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 775517 · Report as offensive
Profile speedimic
Volunteer tester
Avatar

Send message
Joined: 28 Sep 02
Posts: 362
Credit: 16,590,653
RAC: 0
Germany
Message 775567 - Posted: 29 Jun 2008, 21:50:43 UTC

Also, much easier to leave it in place for keeping dependencies happy and for if you ever might want to play on a graphical display.


Nothing you need for a server is depending on any of the stuff you get installing a graphical desktop.
...and you never wanna "play" on a "hot" server - there are testing systems for that.

Setting the init level is simply easy and minimum maintenance.


Yea, 500 more packages to be updated....


mic.


ID: 775567 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 775622 - Posted: 29 Jun 2008, 23:50:37 UTC

Why install X if we don't end up booting the system in run level 5? It does sound silly, but there are a couple reasons: First - we have a policy to install everything possible - on a 100 Mbit network it's no big deal to yum update a few hundred packages at a time. Second - we have many servers which do run in level 5, and we like to keep all the packages on all the servers for general sanity. Third - after installing the OS, doing all the initial SETI-specific configuration is far easier in X (cutting and pasting text, having several windows/browsers open, etc) and then we turn it off.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 775622 · Report as offensive
Profile Dr. C.E.T.I.
Avatar

Send message
Joined: 29 Feb 00
Posts: 16019
Credit: 794,685
RAC: 0
United States
Message 775634 - Posted: 30 Jun 2008, 0:29:32 UTC


. . . Thanks for All your work @ Berkeley Matt - iT's much appreciated Sir!

and the Thanks goes out to each of You @ the Berkeley Labs also . . .


BOINC Wiki . . .

Science Status Page . . .
ID: 775634 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20140
Credit: 7,508,002
RAC: 20
United Kingdom
Message 775757 - Posted: 30 Jun 2008, 8:38:06 UTC - in response to Message 775567.  
Last modified: 30 Jun 2008, 8:39:13 UTC

Nothing you need for a server is depending on any of the stuff you get installing a graphical desktop.
...and you never wanna "play" on a "hot" server - there are testing systems for that.

For a hardened and locked down dedicated 'production' server serving stable standard functions, that is very true. Expected even. I agree.

Then again...

If the server functions are more fluid, or experimental even, then a few extra bits o' fluff likely won't hurt.

OK, I'll admit that a cluster of servers serving hundreds of thousands of users isn't normally 'experimental'!... But then, s@h is a long way from 'normal'.

Also, BOINC is designed to gracefully accept downtime of the main servers. Hence the WU caches on the clients.

Yea, 500 more packages to be updated....

I mainly work on the command-line, but in a terminal window (or few) on a nice Xorg served display! There's even pretty (convenient) GUI config tools for some things also.

A few other bits of GUI 'fluff' such as music, email, and the web are good also... :-) Then again, this thing I'm on might be called a 'test' and development server...


OK, that's my spin!

Happy crunchin',
Martin

(Note: I'm nothing to do with s@h other than merely being one of the millions of participants.)
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 775757 · Report as offensive

Message boards : Technical News : Thursday Thunder (Jun 26 2008)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.