Foam (Feb 15 2007)

Message boards : Technical News : Foam (Feb 15 2007)
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 517954 - Posted: 15 Feb 2007, 23:16:50 UTC

We have a lot planned for next week.

First, we are going to finally upgrade isaac (the boinc.berkeley.edu web server, among other things) to increase disk space and put on a more modern linux OS. I just did some testing this afternoon - thanks to a DNS fake users were forwarded to a "we're down temporarily" web site. The bulk of this process will take place on Tuesday, spilling over into Wednesday if need be. During so, BOINC core client downloads will still be available. Monday is a holiday.

Second, unless THEMIS slips again, we're going to do the big network cut-over on Wednesday. More details will come once we have everything working.

Third, we got new drives for sidious (our new database replica server). We've been itching to get this machine on-line for months now. We'll simultaneously add these drives and do some surgery on ewen to add recently donated memory either Tuesday or Wednesday, depending on the timing of various things.

What else is new..? Well, per user suggestion I'm going to make the most recent threads here sticky. Seems like a perfectly good idea. We also just got some specially made foam/boxes for shipping of drives to/from Arecibo. Hopefully that will reduce drive failures in shipping.

We'll have a writeup on Bob's observations regarding recently database changes which hopefully fixed our slow query issues. Turns out Einstein@home was starting to get similar problems, so we pushed through some new BOINC server back end code. We'll observe closely to make sure this didn't break anything, and perhaps make more changes. We're not gaining anything positive as much as losing something negative.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 517954 · Report as offensive
Profile Crunch3r
Volunteer tester
Avatar

Send message
Joined: 15 Apr 99
Posts: 1546
Credit: 3,438,823
RAC: 0
Germany
Message 517996 - Posted: 16 Feb 2007, 0:19:56 UTC - in response to Message 517954.  
Last modified: 16 Feb 2007, 0:20:11 UTC


Turns out Einstein@home was starting to get similar problems, so we pushed through some new BOINC server back end code. We'll observe closely to make sure this didn't break anything, and perhaps make more changes. We're not gaining anything positive as much as losing something negative.

- Matt


Hi Matt,

got a question about the latest server glitches,

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?











Join BOINC United now!
ID: 517996 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 518015 - Posted: 16 Feb 2007, 1:03:22 UTC - in response to Message 517996.  

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?


Not sure about what glitches you're talking about (I only keep tabs on SETI@home), and this wasn't a bug-fix as much as a code enhancement. The only projects I'm aware of having problems were us and Einstein, and it was mostly database size and load related - we just found a way to "streamline" the query process in certain cases.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 518015 · Report as offensive
Profile Crunch3r
Volunteer tester
Avatar

Send message
Joined: 15 Apr 99
Posts: 1546
Credit: 3,438,823
RAC: 0
Germany
Message 518017 - Posted: 16 Feb 2007, 1:08:34 UTC - in response to Message 518015.  

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?


Not sure about what glitches you're talking about (I only keep tabs on SETI@home), and this wasn't a bug-fix as much as a code enhancement. The only projects I'm aware of having problems were us and Einstein, and it was mostly database size and load related - we just found a way to "streamline" the query process in certain cases.

- Matt


That's what i'm refering to, the database issues.
Was there a change in any config files or db tables etc from v 5.04 to 5.05 ?

I'm really interested in what specifically was/is the cause of that issues.
Or what you've figured out that far causing issues.





Join BOINC United now!
ID: 518017 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 518152 - Posted: 16 Feb 2007, 11:27:19 UTC - in response to Message 518017.  
Last modified: 16 Feb 2007, 11:31:47 UTC

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?

Not sure about what glitches you're talking about (I only keep tabs on SETI@home), and this wasn't a bug-fix as much as a code enhancement. The only projects I'm aware of having problems were us and Einstein, and it was mostly database size and load related - we just found a way to "streamline" the query process in certain cases.

- Matt

That's what i'm refering to, the database issues.
Was there a change in any config files or db tables etc from v 5.04 to 5.05 ?

I'm really interested in what specifically was/is the cause of that issues.
Or what you've figured out that far causing issues.

From the projects/dev list:

David 13 Feb 2007
- Back end: change some of the daemons to use the "read uncommitted"
transaction isolation mode.
This means that their result sets are not locked,
which (at least on SETI@home) greatly boosts the performance
of the MySQL server.
See:
http://dev.mysql.com/doc/refman/5.1/en/innodb-transaction-isolation.html

db
db_base.C,h
sched/
db_dump.C
feeder.C
file_deleter.C
update_stats.C


The speedup is from permitting other db operations to continue whilst big (slow) stats collecting reads are being made. This frees up server memory and also frees up the db for continued updates. The penalty is that the stats may be inaccurate by the number of updates made whilst the read is running. But then, who cares when the stats are "wrong" (insignificantly inaccurate) after a few seconds in any case?

(Sometimes, a million can be a 'big' number even for fast computers!)

Happy crunchin',
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 518152 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13720
Credit: 208,696,464
RAC: 304
Australia
Message 518165 - Posted: 16 Feb 2007, 12:46:40 UTC - in response to Message 518152.  

But then, who cares when the stats are "wrong" (insignificantly inaccurate) after a few seconds in any case?

*Raises eyebrows*
You have read these forums before haven't you?


"I just returned 5,000 results, why don't i have all my Credits yet?!!!"
Grant
Darwin NT
ID: 518165 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 20147
Credit: 7,508,002
RAC: 20
United Kingdom
Message 518187 - Posted: 16 Feb 2007, 14:07:20 UTC - in response to Message 518165.  
Last modified: 16 Feb 2007, 14:08:26 UTC

But then, who cares when the stats are "wrong" (insignificantly inaccurate) after a few seconds in any case?

*Raises eyebrows*
You have read these forums before haven't you?...


*Thoughtful Smirk*

Sorry, missed out the winky smiley! :-o

(For the humour disadvantaged, those were rhetorical ironic comments.)


At least the stats can now be inaccurate more quickly!! :-p

Happy crunchin',
Martin
See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 518187 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 518354 - Posted: 16 Feb 2007, 22:10:10 UTC

NASA Television is carrying the THEMIS launch live -- it's about a half-hour to launch. They're saying that there is a 10% chance of weather prohibiting launch, and everything else looks good.
ID: 518354 · Report as offensive
Profile Labbie
Avatar

Send message
Joined: 19 Jun 06
Posts: 4083
Credit: 5,930,102
RAC: 0
United States
Message 518361 - Posted: 16 Feb 2007, 22:17:02 UTC - in response to Message 518354.  

NASA Television is carrying the THEMIS launch live -- it's about a half-hour to launch. They're saying that there is a 10% chance of weather prohibiting launch, and everything else looks good.


Thanks for the tip Ned.

Do you have any idea what the bubble-looking thing is on the bottom of the main engine?


Calm Chaos Forum...Join Calm Chaos Now
ID: 518361 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 518401 - Posted: 16 Feb 2007, 23:01:23 UTC - in response to Message 518361.  

NASA Television is carrying the THEMIS launch live -- it's about a half-hour to launch. They're saying that there is a 10% chance of weather prohibiting launch, and everything else looks good.


Thanks for the tip Ned.

Do you have any idea what the bubble-looking thing is on the bottom of the main engine?

I think I see what you're calling a bubble-looking thing, but I don't know what it is.

... and they just scrubbed for weather.
ID: 518401 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 518423 - Posted: 16 Feb 2007, 23:25:15 UTC

It looked REALLY good on HDNET until the launch scrubed.

Anyone know what time they will try tomorrow?


Boinc....Boinc....Boinc....Boinc....
ID: 518423 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 518427 - Posted: 16 Feb 2007, 23:29:02 UTC - in response to Message 518423.  
Last modified: 17 Feb 2007, 0:09:07 UTC

It looked REALLY good on HDNET until the launch scrubed.

Anyone know what time they will try tomorrow?


From NASA's THEMIS site

It's 6:01 to 6:19 EST on Saturday.
ID: 518427 · Report as offensive
Profile Misfit
Volunteer tester
Avatar

Send message
Joined: 21 Jun 01
Posts: 21804
Credit: 2,815,091
RAC: 0
United States
Message 518553 - Posted: 17 Feb 2007, 3:11:28 UTC - in response to Message 518427.  

For links to NASA TV/Radio visit the APOTD sticky in Science.
me@rescam.org
ID: 518553 · Report as offensive
Profile Jan Schotsmans
Avatar

Send message
Joined: 27 Oct 00
Posts: 98
Credit: 92,693
RAC: 0
Belgium
Message 519105 - Posted: 18 Feb 2007, 7:48:32 UTC
Last modified: 18 Feb 2007, 7:55:19 UTC

Matt: are you guys running InnoDB storage as files on filesystem or as raw on partitions?

You might know about the later flavor of doing things considering how knowledgeable you guys with databases and such, but most people don't it seems.

It speeds up I/O for InnoDB considerably removing the extra OS level filessystem logic in between.

At the most simplistic view, InnoDB as files on filesystem pritty much is a disk image (for the more computer illiterate, lets say its like a read/write able ISO file you store on your harddrive) that has its own filesystem internally, so letting it rip directly from its own internal logic to raw disk, you remove the OS level filesystem removing quite a bit of overhead.

If your already doing that, don't mind me :p

http://dev.mysql.com/doc/refman/5.0/en/innodb-configuration.html
http://dev.mysql.com/doc/refman/5.0/en/innodb-raw-devices.html
ID: 519105 · Report as offensive

Message boards : Technical News : Foam (Feb 15 2007)


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.