Message boards :
Technical News :
Spindles (Jan 06 2011)
Message board moderation
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
The informix tweak planned yesterday was postponed and completed today. Why was it postponed? Because the weekly science backup (which happens in the background - doesn't require an outage like the mysql database) wasn't done yet. Normally it takes a few hours. But during major activity it looks like it'll take 10 days! Jeff stopped the ntpckr/rfi processes and that sped things up. This clearly points out oscar's inability to handle the crazy random i/o's we desire, though to be fair oscar is indeed operating better in its current state than the old science database. There's still MANY knobs to turn in informix-land before we need to add more disk spindles. For example, we still haven't given all the memory available in the system over to informix. The tweak we made today added an additional 20GB to the buffers. Note that it takes a bout a week to fill these buffers, so we won't notice any improvement, if any, until then. Meanwhile I've been back to working on my various ntpckr and data testing projects. It's hard to page these pieces of code back into my RAM once they've been flushed to disk - know what I mean? - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Claggy Send message Joined: 5 Jul 99 Posts: 4654 Credit: 47,537,079 RAC: 4 |
Thanks for the update Matt, Claggy |
Dirk Sadowski Send message Joined: 6 Apr 07 Posts: 7105 Credit: 147,663,825 RAC: 5 |
Matt, thanks for the news! |
Bad Spartan Send message Joined: 14 Mar 03 Posts: 6 Credit: 47,663,261 RAC: 117 |
Matt, I do know the new servers are HP Proliants. Which Smart Array controllers does it have and how many? How are the disk arrays and logical drive defined? I have worked with Proliants for many years and could be of help setting up yours!!!! |
musicplayer Send message Joined: 17 May 10 Posts: 2442 Credit: 926,046 RAC: 0 |
I was wondering about the disc space capacities on the new servers. Is it enough? I was able to read that the new specs went for a RAID of 12 1 TB SATA-discs. In earlier days magnetic tape was used to store large amounts of data which was meant for processing. Data went to / from the tapes to discs. Today 1 TB as well as 2 TB SATA-discs are available. If returned data is not stored on magnetic tape, you may end up having numerous discs which will have to be swapped in and out depending on have many discs or drives the RAID is able to handle simultaneously. Is this correct? |
Andy Lee Robinson Send message Joined: 8 Dec 05 Posts: 630 Credit: 59,973,836 RAC: 0 |
If returned data is not stored on magnetic tape, you may end up having numerous discs which will have to be swapped in and out depending on have many discs or drives the RAID is able to handle simultaneously. As I understand it, no disks are swapped in and out of the RAID set unless they fail. Drive syncing is expensive on system resources! It just looks like 1 big disk to the file system, that can be scaled to fit the requirements. Drives returning with data from Arecibo are treated exactly in the same way as tapes - as a data source, but cheaper and faster. They are mounted, read and split by the machines responsible for processing the raw data. This working data and project tracking end up on the RAID until purged, with only the results remaining, until those too are archived. |
soft^spirit Send message Joined: 18 May 99 Posts: 6497 Credit: 34,134,168 RAC: 0 |
The informix tweak planned yesterday was postponed and completed today. Why was it postponed? Because the weekly science backup (which happens in the background - doesn't require an outage like the mysql database) wasn't done yet. Normally it takes a few hours. But during major activity it looks like it'll take 10 days! Jeff stopped the ntpckr/rfi processes and that sped things up. I am curious Matt, perhaps you or someone could help me understand... While the Science database is too precious to trust to SSD drives, are the access needs mostly "read" or interactive read/write.. and if mostly "read" could the data be mirrored to SSD to speed up the access? Yet again, just a thought. Janice |
KWSN THE Holy Hand Grenade! Send message Joined: 20 Dec 05 Posts: 3187 Credit: 57,163,290 RAC: 0 |
As I understand the prices, SSD drives of sufficient size for the database are cost-prohibitive at this time... plus someone would have to donate the SSD drives, or their purchase price! . Hello, from Albany, CA!... |
rob smith Send message Joined: 7 Mar 03 Posts: 22491 Credit: 416,307,556 RAC: 380 |
A quick trawl says prices, about four times the money for one quarter the space. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
soft^spirit Send message Joined: 18 May 99 Posts: 6497 Credit: 34,134,168 RAC: 0 |
Of course I had that thought in mind. But the questions remain, would it work and would it be a good idea? I would cringe at the thought of the base science database being ONLY on such drives. So.. could they be mirrored? Janice |
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
Matt, I do know the new servers are HP Proliants. Which Smart Array controllers does it have and how many? How are the disk arrays and logical drive defined? I have worked with Proliants for many years and could be of help setting up yours!!!! They are Smart Array P212's - one in each server controlling 12 drives. Both systems are set up the same way: A 2-drive RAID1 mirror (for root), and an 8-drive RAID10 (for data), and 2 global hot spares. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Bad Spartan Send message Joined: 14 Mar 03 Posts: 6 Credit: 47,663,261 RAC: 117 |
I think your configuration is based on the standard 256MB onboard cache with 3.5" 1TB 7.2K rpm Hard Drives. As for DB operations are they mostly inserts, selects, or random I/O? |
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
I think your configuration is based on the standard 256MB onboard cache with 3.5" 1TB 7.2K rpm Hard Drives. As for DB operations are they mostly inserts, selects, or random I/O? random I/O - that's what the killer is. When it's sequential we get 200-300MB/sec reads/writes without breaking a sweat. When it's random, it's more like maxed at at 5-10MB/sec. - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Todd Hebert Send message Joined: 16 Jun 00 Posts: 648 Credit: 228,292,957 RAC: 0 |
Does it have the cache module on the board? From my experience the SmartArrary cards leave something to be desired especially when you load up the number of drives on the card and dealing with DB operations that are typically random in nature. A few of my clients have moved to LSI/Intel based cards that can use SSD's as cache with the addition of a feature key. http://www.intelraid.com/uploads_demo/Intel_RAID_SSD_Cache_with_FastPath.pdf Adaptec has a similar device too - http://www.adaptec.com/en-us/products/cloudcomputing/maxcache/ssd-cache-performance/ Here is a test example using the adaptec solution and MySQL http://www.adaptec.com/nr/rdonlyres/2767c960-2905-4695-9523-bc1e394943f4/0/applabsmysql.pdf Todd |
soft^spirit Send message Joined: 18 May 99 Posts: 6497 Credit: 34,134,168 RAC: 0 |
Does it have the cache module on the board? From my experience the SmartArrary cards leave something to be desired especially when you load up the number of drives on the card and dealing with DB operations that are typically random in nature. Matt: and yes we can shake the collection jar again if things like that will help. It might be close to tax time by the time we get it, but we can do it. Janice |
RottenMutt Send message Joined: 15 Mar 01 Posts: 1011 Credit: 230,314,058 RAC: 0 |
I think your configuration is based on the standard 256MB onboard cache with 3.5" 1TB 7.2K rpm Hard Drives. As for DB operations are they mostly inserts, selects, or random I/O? can't you enable write cache on raid controllers with a battery backup, which would dramatically improve write performance. but i suppose we don't have real raid controller. i wouldn't enable write cache with only a ups, a server crash can lead to data loss. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.