Foam (Feb 15 2007)


log in

Advanced search

Message boards : Technical News : Foam (Feb 15 2007)

Author Message
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 517954 - Posted: 15 Feb 2007, 23:16:50 UTC

We have a lot planned for next week.

First, we are going to finally upgrade isaac (the boinc.berkeley.edu web server, among other things) to increase disk space and put on a more modern linux OS. I just did some testing this afternoon - thanks to a DNS fake users were forwarded to a "we're down temporarily" web site. The bulk of this process will take place on Tuesday, spilling over into Wednesday if need be. During so, BOINC core client downloads will still be available. Monday is a holiday.

Second, unless THEMIS slips again, we're going to do the big network cut-over on Wednesday. More details will come once we have everything working.

Third, we got new drives for sidious (our new database replica server). We've been itching to get this machine on-line for months now. We'll simultaneously add these drives and do some surgery on ewen to add recently donated memory either Tuesday or Wednesday, depending on the timing of various things.

What else is new..? Well, per user suggestion I'm going to make the most recent threads here sticky. Seems like a perfectly good idea. We also just got some specially made foam/boxes for shipping of drives to/from Arecibo. Hopefully that will reduce drive failures in shipping.

We'll have a writeup on Bob's observations regarding recently database changes which hopefully fixed our slow query issues. Turns out Einstein@home was starting to get similar problems, so we pushed through some new BOINC server back end code. We'll observe closely to make sure this didn't break anything, and perhaps make more changes. We're not gaining anything positive as much as losing something negative.

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Crunch3r
Volunteer tester
Avatar
Send message
Joined: 15 Apr 99
Posts: 1540
Credit: 3,314,460
RAC: 0
Germany
Message 517996 - Posted: 16 Feb 2007, 0:19:56 UTC - in response to Message 517954.
Last modified: 16 Feb 2007, 0:20:11 UTC


Turns out Einstein@home was starting to get similar problems, so we pushed through some new BOINC server back end code. We'll observe closely to make sure this didn't break anything, and perhaps make more changes. We're not gaining anything positive as much as losing something negative.

- Matt


Hi Matt,

got a question about the latest server glitches,

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?










____________

Join BOINC United now!
Auto eVB | Autoversicherung

Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 1 Mar 99
Posts: 1389
Credit: 74,079
RAC: 0
United States
Message 518015 - Posted: 16 Feb 2007, 1:03:22 UTC - in response to Message 517996.

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?


Not sure about what glitches you're talking about (I only keep tabs on SETI@home), and this wasn't a bug-fix as much as a code enhancement. The only projects I'm aware of having problems were us and Einstein, and it was mostly database size and load related - we just found a way to "streamline" the query process in certain cases.

- Matt
____________
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude

Profile Crunch3r
Volunteer tester
Avatar
Send message
Joined: 15 Apr 99
Posts: 1540
Credit: 3,314,460
RAC: 0
Germany
Message 518017 - Posted: 16 Feb 2007, 1:08:34 UTC - in response to Message 518015.

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?


Not sure about what glitches you're talking about (I only keep tabs on SETI@home), and this wasn't a bug-fix as much as a code enhancement. The only projects I'm aware of having problems were us and Einstein, and it was mostly database size and load related - we just found a way to "streamline" the query process in certain cases.

- Matt


That's what i'm refering to, the database issues.
Was there a change in any config files or db tables etc from v 5.04 to 5.05 ?

I'm really interested in what specifically was/is the cause of that issues.
Or what you've figured out that far causing issues.




____________

Join BOINC United now!
Auto eVB | Autoversicherung

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8273
Credit: 4,072,496
RAC: 374
United Kingdom
Message 518152 - Posted: 16 Feb 2007, 11:27:19 UTC - in response to Message 518017.
Last modified: 16 Feb 2007, 11:31:47 UTC

I'm only assuming this one but i guess the "issues" where part of switching to V5.05 and later versions of the boinc server code ?

Or is this problem a "bug" in all boinc versions ?

Not sure about what glitches you're talking about (I only keep tabs on SETI@home), and this wasn't a bug-fix as much as a code enhancement. The only projects I'm aware of having problems were us and Einstein, and it was mostly database size and load related - we just found a way to "streamline" the query process in certain cases.

- Matt

That's what i'm refering to, the database issues.
Was there a change in any config files or db tables etc from v 5.04 to 5.05 ?

I'm really interested in what specifically was/is the cause of that issues.
Or what you've figured out that far causing issues.

From the projects/dev list:

David 13 Feb 2007
- Back end: change some of the daemons to use the "read uncommitted"
transaction isolation mode.
This means that their result sets are not locked,
which (at least on SETI@home) greatly boosts the performance
of the MySQL server.
See:
http://dev.mysql.com/doc/refman/5.1/en/innodb-transaction-isolation.html

db
db_base.C,h
sched/
db_dump.C
feeder.C
file_deleter.C
update_stats.C


The speedup is from permitting other db operations to continue whilst big (slow) stats collecting reads are being made. This frees up server memory and also frees up the db for continued updates. The penalty is that the stats may be inaccurate by the number of updates made whilst the read is running. But then, who cares when the stats are "wrong" (insignificantly inaccurate) after a few seconds in any case?

(Sometimes, a million can be a 'big' number even for fast computers!)

Happy crunchin',
Martin

____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

Grant (SSSF)
Send message
Joined: 19 Aug 99
Posts: 5698
Credit: 56,485,979
RAC: 49,044
Australia
Message 518165 - Posted: 16 Feb 2007, 12:46:40 UTC - in response to Message 518152.

But then, who cares when the stats are "wrong" (insignificantly inaccurate) after a few seconds in any case?

*Raises eyebrows*
You have read these forums before haven't you?


"I just returned 5,000 results, why don't i have all my Credits yet?!!!"
____________
Grant
Darwin NT.

Profile ML1
Volunteer tester
Send message
Joined: 25 Nov 01
Posts: 8273
Credit: 4,072,496
RAC: 374
United Kingdom
Message 518187 - Posted: 16 Feb 2007, 14:07:20 UTC - in response to Message 518165.
Last modified: 16 Feb 2007, 14:08:26 UTC

But then, who cares when the stats are "wrong" (insignificantly inaccurate) after a few seconds in any case?

*Raises eyebrows*
You have read these forums before haven't you?...


*Thoughtful Smirk*

Sorry, missed out the winky smiley! :-o

(For the humour disadvantaged, those were rhetorical ironic comments.)


At least the stats can now be inaccurate more quickly!! :-p

Happy crunchin',
Martin
____________
See new freedom: Mageia4
Linux Voice See & try out your OS Freedom!
The Future is what We make IT (GPLv3)

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 518354 - Posted: 16 Feb 2007, 22:10:10 UTC

NASA Television is carrying the THEMIS launch live -- it's about a half-hour to launch. They're saying that there is a 10% chance of weather prohibiting launch, and everything else looks good.
____________

Profile Labbie
Avatar
Send message
Joined: 19 Jun 06
Posts: 4083
Credit: 5,930,102
RAC: 0
United States
Message 518361 - Posted: 16 Feb 2007, 22:17:02 UTC - in response to Message 518354.

NASA Television is carrying the THEMIS launch live -- it's about a half-hour to launch. They're saying that there is a 10% chance of weather prohibiting launch, and everything else looks good.


Thanks for the tip Ned.

Do you have any idea what the bubble-looking thing is on the bottom of the main engine?

____________

Calm Chaos Forum...Join Calm Chaos Now

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 518401 - Posted: 16 Feb 2007, 23:01:23 UTC - in response to Message 518361.

NASA Television is carrying the THEMIS launch live -- it's about a half-hour to launch. They're saying that there is a 10% chance of weather prohibiting launch, and everything else looks good.


Thanks for the tip Ned.

Do you have any idea what the bubble-looking thing is on the bottom of the main engine?

I think I see what you're calling a bubble-looking thing, but I don't know what it is.

... and they just scrubbed for weather.
____________

Profile Geek@Play
Volunteer tester
Avatar
Send message
Joined: 31 Jul 01
Posts: 2463
Credit: 85,121,964
RAC: 12,158
United States
Message 518423 - Posted: 16 Feb 2007, 23:25:15 UTC

It looked REALLY good on HDNET until the launch scrubed.

Anyone know what time they will try tomorrow?


____________
Boinc....Boinc....Boinc....Boinc....

1mp0£173
Volunteer tester
Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 518427 - Posted: 16 Feb 2007, 23:29:02 UTC - in response to Message 518423.
Last modified: 17 Feb 2007, 0:09:07 UTC

It looked REALLY good on HDNET until the launch scrubed.

Anyone know what time they will try tomorrow?


From NASA's THEMIS site

It's 6:01 to 6:19 EST on Saturday.
____________

Profile Misfit
Volunteer tester
Avatar
Send message
Joined: 21 Jun 01
Posts: 21790
Credit: 2,510,901
RAC: 0
United States
Message 518553 - Posted: 17 Feb 2007, 3:11:28 UTC - in response to Message 518427.

For links to NASA TV/Radio visit the APOTD sticky in Science.
____________

Profile Jan Schotsmans
Avatar
Send message
Joined: 27 Oct 00
Posts: 98
Credit: 92,693
RAC: 0
Belgium
Message 519105 - Posted: 18 Feb 2007, 7:48:32 UTC
Last modified: 18 Feb 2007, 7:55:19 UTC

Matt: are you guys running InnoDB storage as files on filesystem or as raw on partitions?

You might know about the later flavor of doing things considering how knowledgeable you guys with databases and such, but most people don't it seems.

It speeds up I/O for InnoDB considerably removing the extra OS level filessystem logic in between.

At the most simplistic view, InnoDB as files on filesystem pritty much is a disk image (for the more computer illiterate, lets say its like a read/write able ISO file you store on your harddrive) that has its own filesystem internally, so letting it rip directly from its own internal logic to raw disk, you remove the OS level filesystem removing quite a bit of overhead.

If your already doing that, don't mind me :p

http://dev.mysql.com/doc/refman/5.0/en/innodb-configuration.html
http://dev.mysql.com/doc/refman/5.0/en/innodb-raw-devices.html

Message boards : Technical News : Foam (Feb 15 2007)

Copyright © 2014 University of California