Unexplained database slowness

Message boards : News : Unexplained database slowness
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5 · Next

AuthorMessage
Profile Chet McEvers

Send message
Joined: 21 Jul 15
Posts: 1
Credit: 12,746,155
RAC: 0
United States
Message 1927812 - Posted: 2 Apr 2018, 18:39:53 UTC - in response to Message 1927518.  

You'd think giving us more than 8 hours worth of work would be a no brainer....
ID: 1927812 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24870
Credit: 3,081,182
RAC: 7
Ireland
Message 1927815 - Posted: 2 Apr 2018, 18:45:38 UTC - in response to Message 1927811.  

Dangerous road to travel. Been debated many times in the past. Green stars, donations, "super" crunchers, cpu crunchers, gpu crunchers.
Introduce limits & users & just watch all those donations disappear.
ID: 1927815 · Report as offensive
mrchips
Avatar

Send message
Joined: 12 Dec 04
Posts: 17
Credit: 26,590,842
RAC: 8
United States
Message 1927816 - Posted: 2 Apr 2018, 18:50:12 UTC - in response to Message 1927815.  

Good point, I've only donated once or twice, now I am retired, maybe I should donate again.
Slippery Slope...........
ID: 1927816 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22149
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1927818 - Posted: 2 Apr 2018, 18:55:38 UTC

The current limits of 100 for the cpu and 100 per gpu were introduced to protect the working database (BB2 in Richard's description) from overloading, which it is now suffering (or at least appears to be suffering from). Increasing these limits at this time would not help the situation, indeed might have undesirable effect of making things even worse. Thus there may be an argument for reducing these limits, and strictly enforcing them.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1927818 · Report as offensive
Sirius B Project Donor
Volunteer tester
Avatar

Send message
Joined: 26 Dec 00
Posts: 24870
Credit: 3,081,182
RAC: 7
Ireland
Message 1927819 - Posted: 2 Apr 2018, 18:57:05 UTC - in response to Message 1927818.  

Sorry Rob, I wasn't referring to WU lmiits.
ID: 1927819 · Report as offensive
PhonAcq

Send message
Joined: 14 Apr 01
Posts: 1656
Credit: 30,658,217
RAC: 1
United States
Message 1927845 - Posted: 2 Apr 2018, 22:40:46 UTC

Has anyone considered making the WU's longer in order to reduce the average number of transactions on the database? Figuratively, for example, making new WU's that contain 3x the data to reduce on the client than is normal today. A significant increase would be needed, I suspect.

And, is it possible to send such large WU's only to clients running GPU's? It seems absurd that GPU's should be treated the same as CPU's when sizing WU's or specific client compute tasks. Other projects do this, I think, e.g. PrimeGrid.
ID: 1927845 · Report as offensive
mmonnin
Volunteer tester

Send message
Joined: 8 Jun 17
Posts: 58
Credit: 10,176,849
RAC: 0
United States
Message 1927856 - Posted: 2 Apr 2018, 23:38:14 UTC - in response to Message 1927845.  

Has anyone considered making the WU's longer in order to reduce the average number of transactions on the database? Figuratively, for example, making new WU's that contain 3x the data to reduce on the client than is normal today. A significant increase would be needed, I suspect.

And, is it possible to send such large WU's only to clients running GPU's? It seems absurd that GPU's should be treated the same as CPU's when sizing WU's or specific client compute tasks. Other projects do this, I think, e.g. PrimeGrid.


This fixed MW in the past. 5 tasks were bundled together since they only took a few seconds on a 79xx AMD card. Now it has other DB issues.
ID: 1927856 · Report as offensive
Profile MOMMY: He is MAKING ME Read His Posts Thoughts and Prayers. GOoD Thoughts and GOoD Prayers. HATERWORLD Vs THOUGHTs and PRAYERs World. It Is a BATTLE ROYALE. Nobody LOVEs Me. Everybody HATEs Me. Why Don't I Go Eat Worms. Tasty Treats are Wormy Meat. Yes
Avatar

Send message
Joined: 16 Jun 02
Posts: 6895
Credit: 6,588,977
RAC: 0
United States
Message 1927870 - Posted: 3 Apr 2018, 1:13:10 UTC

MultiMILLIONAIRE and BILLIONaire LEFTIEs are SPENDING Their DOLLARs fO ManyManyMany 'Causes'.

Where Are dA LEFTY BIG BUCKs fO SETIatHOME?

Certaninly dA 'SCIENCE' is MEGA COMPELLING to ANY RichieRICH LEFTY.

There 'Is' 'SCIENCE' Here WORTH 'Promoting'; 'PUBLISHING'; and Donating HARD CASH to? Correct?

What Do dA SUPER RichieRICH Know dAt DEDICATED CRUNCHERs Do Not?

Only People Believing 'IT" CAN Happen Any Second or In 100s of Years Have dA COMMITMENT and FORTITUDE to 'KNOW" dA EFFORT is WORTH dA Coming REWARD?

While KNOWING Hardware Is NOT Up to Date fO Task, and 'RESULTs' NOT EVER Going to be 'ANALYZED' Further.

Where is dA Scientific Method to this MADNESS?

Is Da Moon Hollow?

Yap@Home

May we All have a METAMORPHOSIS. REASON. GOoD JUDGEMENT and LOVE and ORDER!!!!!
ID: 1927870 · Report as offensive
Profile betreger Project Donor
Avatar

Send message
Joined: 29 Jun 99
Posts: 11354
Credit: 29,581,041
RAC: 66
United States
Message 1927872 - Posted: 3 Apr 2018, 1:42:34 UTC - in response to Message 1927870.  

Oh, I see not your green star.
ID: 1927872 · Report as offensive
Profile j mercer
Avatar

Send message
Joined: 3 Jun 99
Posts: 2422
Credit: 12,323,733
RAC: 1
United States
Message 1927873 - Posted: 3 Apr 2018, 1:54:57 UTC - in response to Message 1927872.  

Oh, I see not your green star.

Book'um Dano! har... ;-)
...
ID: 1927873 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1927886 - Posted: 3 Apr 2018, 4:02:19 UTC - in response to Message 1927678.  
Last modified: 3 Apr 2018, 4:12:04 UTC

...The tasks which make the database 'messy' will be the ones which hang around, and their wingmates - the errors, the failed validations, the ghosts, the users who walk away without clearing their caches...


Speaking of ghosts: Is anyone working on removing the 71 ghost MB 7.0 WU's? That might clear up some space, as not only the space for those WU's could be re-allocated, but once they are gone, the entire set of MB 7.0 executables could be deleted... These ghosts have been bothering me ever since the number of "in the field" 7.0 WU's went to zero - and I'm on one of the ghosts! (as in: I processed the WU, and a wingman also came in, but the WU never got deleted!...)

And as far as "Walking away without clearing the cache" goes: sometimes you have no choice about it - sometimes a computer will fail on you without warning, and you have to either buy or put together a new one, and let those WU's that were on the failed computer get picked up when they exceed deadline...
.

Hello, from Albany, CA!...
ID: 1927886 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13715
Credit: 208,696,464
RAC: 304
Australia
Message 1927887 - Posted: 3 Apr 2018, 4:07:44 UTC - in response to Message 1927808.  

But on server-grade kit like this, they probably have access to multi-channel RAID controllers, and lots of drive bays, so my 'two database containers' suggestion doesn't mean two physical servers or even disk access contention.

Yep.
But the more storage pools there are, the greater the work of the server to manage them, and the more tweaking required to balance available RAM (for caching) between the pools (I've no idea what automatic resource allocation is available for such systems, nor what it's capabilities are if it exists).
The big advantage of multiple storage pools on a given server is that existing HDDs can be replaced with Flash storage so that at the very least indexes & logs can be placed on them, or if there is enough space in the server for the capacity necessary, one or more of the tables with the greatest I/O load could be moved to the Flash storage pool and save the HDDs for the not so busy tables.

Has Seti gone back to Hardware RAID controllers?
I recall they did have some, but when one died they didn't have any spares of the same model & version & so were unable to rebuild that particular array, hence Matt went for Software RAID. Cheaper and easier to deal with hardware failures, but certainly not the performance of dedicated hardware controllers.
Grant
Darwin NT
ID: 1927887 · Report as offensive
Profile marsinph
Volunteer tester

Send message
Joined: 7 Apr 01
Posts: 172
Credit: 23,823,824
RAC: 0
Belgium
Message 1927976 - Posted: 4 Apr 2018, 16:58:24 UTC

Hello all of you.
I think everyone see the problems y differents ways. All of uou, experimented tester knwns much ore than me.
My only idea is only to stop producing work. Arecibo is out. So no new tasks at this time.
Stopping producing work will for sure be not apreiated to all of us. But.
A least, it will give time to server to absorbe all WU in queue.
Servers will be more able to give more power to all other application.
So I really thik, after a few days, all very late works will be resorbed, the bottlneck reduced.
Then staff can work more easily on DB.
DB try to work, but receive more and more WU ! It is never ending.
Why not to FULLY stop the project for a few days and so give fresh air to server to breaths (also to staff)
I think everyone become tired about all the problems.
We, normal cruncher as me, probably, we will lost CR and AVG due to longer outage. And then ????
We will not die !
It is my suggestion : stopping producing work. And let the time , the time to solve by self, with help from staff.
Sorry, I stay with my idea : stopping producing work, Disable up/down loas server. I know, some WU will be out of time.
Or perhaps, no any work sending to us, upload server active to let finish WU out in field, And then closing up/down
after one week. Then let the staff working. It need one week ? two weeks ? 3 weeks ? no problem.
As long staff give information.
Best regards from Belgium
ID: 1927976 · Report as offensive
Profile Christian Diekmann

Send message
Joined: 30 Mar 00
Posts: 13
Credit: 60,615,800
RAC: 163
Germany
Message 1927998 - Posted: 4 Apr 2018, 19:29:05 UTC

Greetings to all and greetings to Belgium.

I cannot understand that with so many millions of donations in the last few years technically hardly anything has happened. In any case, not really positively noticeable. It was purchased and that (what you can find and read on the internet about the purchases at SETI) and sometimes the software improves from 7.8.x to 7.9.x but with SETI only a “handful of people work” ... - where is all the money there? Already consumed everything?

And if there are so many problems with hardware and software - what do they do all day? Look at the monitors and wait for the big WOW-two signal to come? The database problem and hardware difficulties have existed for months. It looks like Eric has to do it all by himself.

Maybe it would make sense to make a comprehensive improvement plan and then shut down SETI for 2-3 weeks. Clearing old stuff, clearing up statistics and "corpses", processing all outstanding data (WUs), then reworking the entire hardware and software and then going back online clean. ET will certainly not send a signal in the 3 weeks. But when it comes - and it will come - and all the hardware and software (as it is now) does not work properly, the whole years were really free for each and every one of us.

Much more likely it is for me that the signal is already there and is not recognized ...
But I'm not giving up yet and continue to search with you - if I get WUs.

So many good people write here who are familiar with databases and hardware but are not from SETI. Why are not these experiences, ideas and helps flowing into the project? Do the ideas evaporate here in the chat?

Science means to create knowledge and that is what many do here!
It is clear to me that in scientific projects, which are also subject to a certain level of security, not everyone can and should be allowed to work and have insight. But there is always talk of so many donations (money). Why are the experiences, ideas and knowledge donations not accepted?

Certainly, very creative and intelligent people work at SETI. But even these are limited with their knowledge and their performance. Maybe SETI will start a donation call - Mining of knowledge - for thoughts and ideas of professionals? I am sure that we can find a solution so quickly.

Well, for the most successful and implemented ideas there are no credits but instead of a donation star, graphics card symbol or disk icon, a bronze, silver or golden, pulsating brain as an achievement 😉

As reminder:
- (Info: 2011) Every year SETI needs about 2.5 million US dollars to operate.

- (Info: 2015) Milner and Hawking: "A Russian billionaire donates $ 100 million for the search for aliens."
The lack of money cannot really be.
English: https://motherboard.vice.com/en_us/article/pga5zv/who-is-the-russian-billionaire-who-just-pledged-100-million-to-find-alien-life
German: https://motherboard.vice.com/de/article/8q8ye4/russischer-milliardaer-geht-mit-stephen-hawking-auf-aliensuche-387
and http://breakthroughinitiatives.org/news/1
So, there should be enough money for hardware, software and specialists - at least in the last 3 years. Or did I misunderstand something?
But maybe all the faults are there too?
March 31, 2018: “Signal Received From Deep Space is Strongest Ever Recorded”
http://exonews.org/signal-received-from-deep-space-is-strongest-ever-recorded/
.
.
.
04.04.2018: Meanwhile, after a long period of maintenance, something has changed - it is running again. 😊😊😊
ID: 1927998 · Report as offensive
Profile Sharpshooter

Send message
Joined: 26 Mar 00
Posts: 47
Credit: 5,127,861
RAC: 2
United States
Message 1928052 - Posted: 5 Apr 2018, 0:26:47 UTC - in response to Message 1927998.  

I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me.
ID: 1928052 · Report as offensive
Profile KWSN THE Holy Hand Grenade!
Volunteer tester
Avatar

Send message
Joined: 20 Dec 05
Posts: 3187
Credit: 57,163,290
RAC: 0
United States
Message 1928064 - Posted: 5 Apr 2018, 3:23:26 UTC - in response to Message 1928052.  

I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me.


They recently (about 6 months ago, IIRC...) doubled the size of the WU, first on Beta, then on production.
.

Hello, from Albany, CA!...
ID: 1928064 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 30593
Credit: 53,134,872
RAC: 32
United States
Message 1928068 - Posted: 5 Apr 2018, 3:49:54 UTC - in response to Message 1928064.  

I do think that making work units longer would help if that is feasible to do. Over the years computers have gotten a lot faster and can do more WUs at one time. That means lots of people trying to connect and trying often. Many folks spend a lot of money on powerful rigs and pride themselves on doing a lot of work. More time crunching and less time trying to get enough work makes sense to me.


They recently (about 6 months ago, IIRC...) doubled the size of the WU, first on Beta, then on production.

I remember that too. The other issue we have is this project wants to support crunching on smart phones. They are snails compared to high end GPU's. W/U size has to stay reasonable for them as do deadlines considering their flop counts. Now if some of you programmer types want to change BOINC so it can issue dynamic deadlines based on reported flop count, the code is on github ... Making W/U sizes dynamic is much harder as that is the science code.
ID: 1928068 · Report as offensive
Profile Sharpshooter

Send message
Joined: 26 Mar 00
Posts: 47
Credit: 5,127,861
RAC: 2
United States
Message 1928083 - Posted: 5 Apr 2018, 12:40:07 UTC - in response to Message 1928068.  

I appreciate the info guys. Odd though, I haven't noticed my geriatric crunchers taking twice as long to get the work done. Must be the organic, non-GMO, 100% natural electricity I'm feeding them.
ID: 1928083 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 22149
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1928085 - Posted: 5 Apr 2018, 12:53:25 UTC

The "doubling" of the task size was triggered not by increasing the number of data points but by increasing the resolution. This change has had very little impact on the execution time of a task.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1928085 · Report as offensive
Profile Kissagogo27 Special Project $75 donor
Avatar

Send message
Joined: 6 Nov 99
Posts: 715
Credit: 8,032,827
RAC: 62
France
Message 1928667 - Posted: 7 Apr 2018, 18:30:20 UTC

ID: 1928667 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5 · Next

Message boards : News : Unexplained database slowness


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.