Unexplained database slowness

Message boards : News : Unexplained database slowness
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5

AuthorMessage
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 10088
Credit: 133,279,518
RAC: 83,883
Australia
Message 1930457 - Posted: 17 Apr 2018, 6:36:01 UTC - in response to Message 1930419.  

The experimental reorg that we did two weeks ago appears to have worked. Our speed is better, and our outage last week was down to about 3.5 hours.

Fingers crossed that things keep working out.

Prior to the Arecibo VLARs going out to NVidia GPUs, the deleters & purgers were falling behind again.
Will be interesting to see if they do any better then next time the Received-last-hour hits 130k-145k sustained, they were struggling with it just under 120k.
Grant
Darwin NT
ID: 1930457 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4890
Credit: 314,778,970
RAC: 686,305
United States
Message 1930560 - Posted: 17 Apr 2018, 15:51:38 UTC - in response to Message 1930457.  

I wonder if we will ever hit the 145K/hour return rate again as long as there are Arecibo tasks going out. Also, have you noticed the coincidence in the Haveland graphs where the splitter output has a negative spike to zero at the same time the purgers/deleters have a positive spike? I wonder is this is part of the database reorg reconfiguration. Looks like a script is running.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1930560 · Report as offensive
JohnDK Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 1029
Credit: 163,936,031
RAC: 173,094
Denmark
Message 1930568 - Posted: 17 Apr 2018, 21:06:09 UTC - in response to Message 1930427.  

Thanks for the progress report Eric. Fingers crossed. Everything seems to hitting on all cylinders since the reorg. Good Job!

+1
ID: 1930568 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4890
Credit: 314,778,970
RAC: 686,305
United States
Message 1930570 - Posted: 17 Apr 2018, 21:10:04 UTC

Isn't this a new process we haven't seen before on the servers?

splitter_throttle_sah bruno
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1930570 · Report as offensive
Kissagogo27 Special Project $75 donor

Send message
Joined: 6 Nov 99
Posts: 202
Credit: 5,166,393
RAC: 3,007
France
Message 1930579 - Posted: 17 Apr 2018, 22:18:54 UTC

i've just seen it , splitters speed limit ?
ID: 1930579 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4890
Credit: 314,778,970
RAC: 686,305
United States
Message 1930580 - Posted: 17 Apr 2018, 22:33:14 UTC - in response to Message 1930579.  

i've just seen it , splitters speed limit ?

No, I think it is the process that knocks the splitters offline when the RTS buffer hits its max limit trip-point of around 620-640K. I've caught the splitters offline several times when the RTS buffer gets full. When the splitters go offline, it frees up resources for the file deleters and purgers to get caught up. You can see the behavior in the Haveland graphs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1930580 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1097
Credit: 108,863,301
RAC: 33,360
United States
Message 1930599 - Posted: 18 Apr 2018, 1:13:02 UTC - in response to Message 1930560.  

I wonder if we will ever hit the 145K/hour return rate again as long as there are Arecibo tasks going out. . .

I imagine we will, once the glut of APs goes away.
I think that's the more relevant factor here, rather than Aricebo VLARs.
ID: 1930599 · Report as offensive
kittyman Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 50015
Credit: 923,173,527
RAC: 157,797
United States
Message 1930715 - Posted: 18 Apr 2018, 17:18:57 UTC

Great news on the DB reorg, Eric!
And the shortened outages are a refreshing change as well!
I hope things stay flowing if the result return rate goes back up at some point.
But in any case, very nice to see the servers working efficiently.
Happy is the person who shares their life with a cat. (Or two or three or........) =^.^=

Have made friends here.
Most were cats.
ID: 1930715 · Report as offensive
juan BFP Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 6940
Credit: 394,652,070
RAC: 140,706
Panama
Message 1930729 - Posted: 18 Apr 2018, 19:33:10 UTC - in response to Message 1930715.  

Great news on the DB reorg, Eric!
And the shortened outages are a refreshing change as well!
I hope things stay flowing if the result return rate goes back up at some point.
But in any case, very nice to see the servers working efficiently.

+ 1
ID: 1930729 · Report as offensive
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1097
Credit: 108,863,301
RAC: 33,360
United States
Message 1930788 - Posted: 19 Apr 2018, 2:43:38 UTC - in response to Message 1930729.  

Great news on the DB reorg, Eric!
And the shortened outages are a refreshing change as well!
I hope things stay flowing if the result return rate goes back up at some point.
But in any case, very nice to see the servers working efficiently.

+ 1

+1
ID: 1930788 · Report as offensive
Profile popandbob
Volunteer tester

Send message
Joined: 19 Mar 05
Posts: 542
Credit: 3,505,309
RAC: 566
Canada
Message 1931105 - Posted: 20 Apr 2018, 22:46:57 UTC - in response to Message 1930580.  

i've just seen it , splitters speed limit ?

No, I think it is the process that knocks the splitters offline when the RTS buffer hits its max limit trip-point of around 620-640K. I've caught the splitters offline several times when the RTS buffer gets full. When the splitters go offline, it frees up resources for the file deleters and purgers to get caught up. You can see the behavior in the Haveland graphs.


The Splitters have that function build in. I suspect the function of this new process is to reduce the number of splitters running when under heavy db load to allow for other processes to take priority.


Do you Good Search for Seti@Home? http://www.goodsearch.com/?charityid=888957
Or Good Shop? http://www.goodshop.com/?charityid=888957
ID: 1931105 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4890
Credit: 314,778,970
RAC: 686,305
United States
Message 1931194 - Posted: 21 Apr 2018, 4:42:55 UTC - in response to Message 1931105.  
Last modified: 21 Apr 2018, 4:43:36 UTC

Could be. Next we need to catch that process in the act of being enabled and then go look at the Haveland graphs for synchronicity.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1931194 · Report as offensive
Profile popandbob
Volunteer tester

Send message
Joined: 19 Mar 05
Posts: 542
Credit: 3,505,309
RAC: 566
Canada
Message 1931448 - Posted: 22 Apr 2018, 23:09:35 UTC - in response to Message 1931194.  

Could be. Next we need to catch that process in the act of being enabled and then go look at the Haveland graphs for synchronicity.

Process currently running - only 3 Green Bank splitters running - remaining splitters are not running.


Do you Good Search for Seti@Home? http://www.goodsearch.com/?charityid=888957
Or Good Shop? http://www.goodshop.com/?charityid=888957
ID: 1931448 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4890
Credit: 314,778,970
RAC: 686,305
United States
Message 1931460 - Posted: 23 Apr 2018, 0:11:10 UTC - in response to Message 1931448.  

Caught it once before but could not make any conclusions. It's running now and all MB splitters are running with NO AP splitters. There is a lot of "grass" spikes in the file deleters graphs. More than I am accustomed to seeing. I think we are just wildly guessing the purpose at this point and probably would need Eric to actually explain what the process is and when it gets enabled and disabled to understand it.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1931460 · Report as offensive
Profile James Butler
Volunteer tester

Send message
Joined: 29 Jun 17
Posts: 6
Credit: 1,873,019
RAC: 5,266
United Kingdom
Message 1931819 - Posted: 25 Apr 2018, 12:30:06 UTC

I am not getting any work today, please can someone look into this, Thankyou
ID: 1931819 · Report as offensive
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4890
Credit: 314,778,970
RAC: 686,305
United States
Message 1931858 - Posted: 25 Apr 2018, 17:20:38 UTC - in response to Message 1931819.  

Work is flowing fine to everyone else. You are the only one having issues. Post the first 30 lines of the Event Log after first startup of BOINC so we can see if any errors are being logged.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1931858 · Report as offensive
Profile James Butler
Volunteer tester

Send message
Joined: 29 Jun 17
Posts: 6
Credit: 1,873,019
RAC: 5,266
United Kingdom
Message 1931869 - Posted: 25 Apr 2018, 19:42:38 UTC - in response to Message 1931858.  
Last modified: 25 Apr 2018, 19:42:55 UTC

25/04/2018 15:46:18 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 15:46:18 | SETI@home Beta Test | Requesting new tasks for CPU
25/04/2018 15:46:20 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 15:46:20 | SETI@home Beta Test | Project has no tasks available
25/04/2018 15:52:38 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 15:52:38 | SETI@home Beta Test | Requesting new tasks for NVIDIA GPU
25/04/2018 15:52:40 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 15:52:40 | SETI@home Beta Test | Project has no tasks available
25/04/2018 17:01:32 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 17:01:32 | SETI@home Beta Test | Requesting new tasks for CPU
25/04/2018 17:01:34 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 17:01:34 | SETI@home Beta Test | Project has no tasks available
25/04/2018 17:19:04 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 17:19:04 | SETI@home Beta Test | Requesting new tasks for NVIDIA GPU
25/04/2018 17:19:05 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 17:19:05 | SETI@home Beta Test | Project has no tasks available
ID: 1931869 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 12055
Credit: 121,791,113
RAC: 64,045
United Kingdom
Message 1931870 - Posted: 25 Apr 2018, 19:48:43 UTC - in response to Message 1931869.  

Those are requests for SETI@home BETA work, where there is a front-page news item:

We will be out of work for a while

Check the discussion thread at that website for more recent updates.
ID: 1931870 · Report as offensive
rob smith Special Project $250 donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 16291
Credit: 325,423,583
RAC: 219,715
United Kingdom
Message 1931871 - Posted: 25 Apr 2018, 19:49:50 UTC

Ah - you are looking at the SETI Beta project, which has a very low availability of tasks. That project is used to test new applications, or to "prove" selected parts of the servers, and should not be replied on to be a continuous source of tasks.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1931871 · Report as offensive
Tutankhamon
Volunteer tester
Avatar

Send message
Joined: 1 Nov 08
Posts: 7217
Credit: 44,944,968
RAC: 9,179
Sweden
Message 1931878 - Posted: 25 Apr 2018, 21:00:21 UTC

After the Tuesday scheduled outage, Beta didn't turn on all the servers/processes, so Tasks ready to send=0
See: https://setiweb.ssl.berkeley.edu/beta/server_status.php

And the news item about "We will be out of work for a while", is old. It's from 20 Dec 2017, 1:30:42 UTC.
We've had plenty of work since then.
ID: 1931878 · Report as offensive
Previous · 1 · 2 · 3 · 4 · 5

Message boards : News : Unexplained database slowness


 
©2018 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.