Message boards :
Technical News :
Taking longer than expected (Nov 13, 2014)
Message board moderation
Author | Message |
---|---|
Eric Korpela Send message Joined: 3 Apr 99 Posts: 1382 Credit: 54,506,847 RAC: 60 |
The Astropulse database fix is taking longer than expected. By one measure (the amount of data written into the new database spaces) we're 70% done. By another measure (the number of table rows showing up in the table stats) we are 25% done. This is after 5 days. I'm inclined to believe the 25% number, in which case we will run out of SETI@home work before Astropulse is back online. If you don't have a low share back-up project, now might be time to add one. Informix never ceases to astonish me with the way it does things. The table rebuild is neither maxing out CPUs or I/O, primarily because it doesn't seem to be running the table creation in parallel. It's working on one table fragment at a time. It also seems to be making a copy of the data in two places and leaving large portions the allocated data chunks unused. It's certainly not the way I would have written it, but then again I don't write database. The worst case is that these multiple copies of data will cause it to exceed the limits again and we'll have to start over. It looks like we're 68% of the way to exceeding those limits. Hope Informix is smart enough to avoid the problem. @SETIEric@qoto.org (Mastodon) |
rob smith Send message Joined: 7 Mar 03 Posts: 22538 Credit: 416,307,556 RAC: 380 |
If I recall rightly (and its a good few years since I last did anything in Informix) its ability to create, and then forget, orphan tables during a rebuild was "quite astounding" :-( I'd believe the 25% figure too, and remember that time is non-linear.... Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
Raistmer Send message Joined: 16 Jun 01 Posts: 6325 Credit: 106,370,077 RAC: 121 |
If complete rebuild inevitable maybe it's good time to think about different database engine, more optimal one (if such exits, of course...) |
Eric Korpela Send message Joined: 3 Apr 99 Posts: 1382 Credit: 54,506,847 RAC: 60 |
Good idea. I'll have to dig into what features are found in the various database options. @SETIEric@qoto.org (Mastodon) |
arkayn Send message Joined: 14 May 99 Posts: 4438 Credit: 55,006,323 RAC: 0 |
|
Cliff Harding Send message Joined: 18 Aug 99 Posts: 1432 Credit: 110,967,840 RAC: 67 |
Has anyone given any thought to contacting IBM concerning the Infomix limit problem? Maybe, just maybe they might have some insight into the problem, if someone were to ask nicely. Maybe we can get them on as a sponsor at same time. I don't buy computers, I build them!! |
Thomas Send message Joined: 9 Dec 11 Posts: 1499 Credit: 1,345,576 RAC: 0 |
Thanks for the heads-up Eric :) |
Cosmic_Ocean Send message Joined: 23 Dec 00 Posts: 3027 Credit: 13,516,867 RAC: 13 |
Has anyone given any thought to contacting IBM concerning the Infomix limit problem? Maybe, just maybe they might have some insight into the problem, if someone were to ask nicely. Maybe we can get them on as a sponsor at same time. That is definitely a good idea. Maybe there's nothing actually wrong with Informix, but S@H may be the first user to run into previously-unknown problems because of the size of the DB and the load and web of cross-references. The DB size and load has been well into "uncharted territory" for quite some time. I remember reading a news post a few years ago mentioning MySQL was suffering some strange behavior and there were no clear answers as to why, simply because "nobody's ever made a DB that big with MySQL before." So maybe contacting IBM and asking for some insight could not only resolve your own problems, but also result in preventing future problems for others. Linux laptop: record uptime: 1511d 20h 19m (ended due to the power brick giving-up) |
Darth Beaver Send message Joined: 20 Aug 99 Posts: 6728 Credit: 21,443,075 RAC: 3 |
Thanks Eric for the updates . Don't understand a word you said but nice to know your on top of it.....:) |
Dimly Lit Lightbulb 😀 Send message Joined: 30 Aug 08 Posts: 15399 Credit: 7,423,413 RAC: 1 |
Thanks Eric for the updates . I think it's like packing a suitcase when you go on holiday, first time you can't close it. Then you fold things differently and move stuff about till you can :) Good luck with it Eric :) Member of the People Encouraging Niceness In Society club. |
Cliff Harding Send message Joined: 18 Aug 99 Posts: 1432 Credit: 110,967,840 RAC: 67 |
Has anyone given any thought to contacting IBM concerning the Infomix limit problem? Maybe, just maybe they might have some insight into the problem, if someone were to ask nicely. Maybe we can get them on as a sponsor at same time. I'm more familiar with DB2 than Informix as a database manager. Both have been used in data warehousing for years, DB2 has been using terabyte databases in particular. Unfortunately, DB2 requires a more hands on DBA. Since SQL is basically the same across all platforms and management systems, and MySQL is essentially using an extension of SQL; I would love to take a peek at our database/table schemas. Another big problem that I see occurring is that Informix making data copies in multiple places while rebuilding the database, I've never heard or experienced that when rebuilding/restoring DB2 tables/databases. Since the rebuild/restore source is essentially the last full image copy (backup) merged with the succeeding incremental image copies, prior to the actual rebuild; then essentially it should be just a matter of copying one to another in an sequential manner. Once this is accomplished, there should be a log somewhere that has all of the current updates that were made to the table/database up to the time of the stoppage. Applying the log (LOGAPPLY) to the table/database will bring it current status. If necessary, the indices can be rebuilt using a REBUILD utility. I don't buy computers, I build them!! |
David S Send message Joined: 4 Oct 99 Posts: 18352 Credit: 27,761,924 RAC: 12 |
Thanks for the update, Eric. I wish I understood it on more than the most basic level. David Sitting on my butt while others boldly go, Waiting for a message from a small furry creature from Alpha Centauri. |
Angela Send message Joined: 16 Oct 07 Posts: 13131 Credit: 39,854,104 RAC: 31 |
New game for the Cafe... TLPTGAWUW!!! (The last person to get a work unit wins) |
Uli Send message Joined: 6 Feb 00 Posts: 10923 Credit: 5,996,015 RAC: 1 |
I like that idea! Pluto will always be a planet to me. Seti Ambassador Not to late to order an Anni Shirt |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
New game for the Cafe... TLPTGAWUW!!! I thought we were already playing "First one to infinity wins" :P SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
Dimly Lit Lightbulb 😀 Send message Joined: 30 Aug 08 Posts: 15399 Credit: 7,423,413 RAC: 1 |
New game for the Cafe... TLPTGAWUW!!! Done :) Member of the People Encouraging Niceness In Society club. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.