Unexplained database slowness

Message boards : News : Unexplained database slowness
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · 4 · 5

AuthorMessage
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 9807
Credit: 126,658,040
RAC: 85,444
Australia
Message 1930457 - Posted: 17 Apr 2018, 6:36:01 UTC - in response to Message 1930419.  

The experimental reorg that we did two weeks ago appears to have worked. Our speed is better, and our outage last week was down to about 3.5 hours.

Fingers crossed that things keep working out.

Prior to the Arecibo VLARs going out to NVidia GPUs, the deleters & purgers were falling behind again.
Will be interesting to see if they do any better then next time the Received-last-hour hits 130k-145k sustained, they were struggling with it just under 120k.
Grant
Darwin NT
ID: 1930457 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4395
Credit: 262,502,071
RAC: 598,895
United States
Message 1930560 - Posted: 17 Apr 2018, 15:51:38 UTC - in response to Message 1930457.  

I wonder if we will ever hit the 145K/hour return rate again as long as there are Arecibo tasks going out. Also, have you noticed the coincidence in the Haveland graphs where the splitter output has a negative spike to zero at the same time the purgers/deleters have a positive spike? I wonder is this is part of the database reorg reconfiguration. Looks like a script is running.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1930560 · Report as offensive     Reply Quote
JohnDK Crowdfunding Project Donor*Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 28 May 00
Posts: 1023
Credit: 148,983,700
RAC: 185,189
Denmark
Message 1930568 - Posted: 17 Apr 2018, 21:06:09 UTC - in response to Message 1930427.  

Thanks for the progress report Eric. Fingers crossed. Everything seems to hitting on all cylinders since the reorg. Good Job!

+1
ID: 1930568 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4395
Credit: 262,502,071
RAC: 598,895
United States
Message 1930570 - Posted: 17 Apr 2018, 21:10:04 UTC

Isn't this a new process we haven't seen before on the servers?

splitter_throttle_sah bruno
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1930570 · Report as offensive     Reply Quote
Kissagogo27 Special Project $75 donor

Send message
Joined: 6 Nov 99
Posts: 193
Credit: 4,897,652
RAC: 5,261
France
Message 1930579 - Posted: 17 Apr 2018, 22:18:54 UTC

i've just seen it , splitters speed limit ?
ID: 1930579 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4395
Credit: 262,502,071
RAC: 598,895
United States
Message 1930580 - Posted: 17 Apr 2018, 22:33:14 UTC - in response to Message 1930579.  

i've just seen it , splitters speed limit ?

No, I think it is the process that knocks the splitters offline when the RTS buffer hits its max limit trip-point of around 620-640K. I've caught the splitters offline several times when the RTS buffer gets full. When the splitters go offline, it frees up resources for the file deleters and purgers to get caught up. You can see the behavior in the Haveland graphs.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1930580 · Report as offensive     Reply Quote
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1090
Credit: 106,465,064
RAC: 80,662
United States
Message 1930599 - Posted: 18 Apr 2018, 1:13:02 UTC - in response to Message 1930560.  

I wonder if we will ever hit the 145K/hour return rate again as long as there are Arecibo tasks going out. . .

I imagine we will, once the glut of APs goes away.
I think that's the more relevant factor here, rather than Aricebo VLARs.
ID: 1930599 · Report as offensive     Reply Quote
kittyman Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 49789
Credit: 910,381,630
RAC: 180,709
United States
Message 1930715 - Posted: 18 Apr 2018, 17:18:57 UTC

Great news on the DB reorg, Eric!
And the shortened outages are a refreshing change as well!
I hope things stay flowing if the result return rate goes back up at some point.
But in any case, very nice to see the servers working efficiently.
What meowing lurks in the hearts of man? The kittyman knows....MEOWhahahahahahha!

Have made friends here.
Most were cats.
ID: 1930715 · Report as offensive     Reply Quote
juan BFP Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 6856
Credit: 382,174,783
RAC: 180,947
Panama
Message 1930729 - Posted: 18 Apr 2018, 19:33:10 UTC - in response to Message 1930715.  

Great news on the DB reorg, Eric!
And the shortened outages are a refreshing change as well!
I hope things stay flowing if the result return rate goes back up at some point.
But in any case, very nice to see the servers working efficiently.

+ 1
ID: 1930729 · Report as offensive     Reply Quote
Profile Jimbocous Project Donor
Volunteer tester
Avatar

Send message
Joined: 1 Apr 13
Posts: 1090
Credit: 106,465,064
RAC: 80,662
United States
Message 1930788 - Posted: 19 Apr 2018, 2:43:38 UTC - in response to Message 1930729.  

Great news on the DB reorg, Eric!
And the shortened outages are a refreshing change as well!
I hope things stay flowing if the result return rate goes back up at some point.
But in any case, very nice to see the servers working efficiently.

+ 1

+1
ID: 1930788 · Report as offensive     Reply Quote
Profile popandbob
Volunteer tester

Send message
Joined: 19 Mar 05
Posts: 542
Credit: 3,422,582
RAC: 693
Canada
Message 1931105 - Posted: 20 Apr 2018, 22:46:57 UTC - in response to Message 1930580.  

i've just seen it , splitters speed limit ?

No, I think it is the process that knocks the splitters offline when the RTS buffer hits its max limit trip-point of around 620-640K. I've caught the splitters offline several times when the RTS buffer gets full. When the splitters go offline, it frees up resources for the file deleters and purgers to get caught up. You can see the behavior in the Haveland graphs.


The Splitters have that function build in. I suspect the function of this new process is to reduce the number of splitters running when under heavy db load to allow for other processes to take priority.


Do you Good Search for Seti@Home? http://www.goodsearch.com/?charityid=888957
Or Good Shop? http://www.goodshop.com/?charityid=888957
ID: 1931105 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4395
Credit: 262,502,071
RAC: 598,895
United States
Message 1931194 - Posted: 21 Apr 2018, 4:42:55 UTC - in response to Message 1931105.  
Last modified: 21 Apr 2018, 4:43:36 UTC

Could be. Next we need to catch that process in the act of being enabled and then go look at the Haveland graphs for synchronicity.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1931194 · Report as offensive     Reply Quote
Profile popandbob
Volunteer tester

Send message
Joined: 19 Mar 05
Posts: 542
Credit: 3,422,582
RAC: 693
Canada
Message 1931448 - Posted: 22 Apr 2018, 23:09:35 UTC - in response to Message 1931194.  

Could be. Next we need to catch that process in the act of being enabled and then go look at the Haveland graphs for synchronicity.

Process currently running - only 3 Green Bank splitters running - remaining splitters are not running.


Do you Good Search for Seti@Home? http://www.goodsearch.com/?charityid=888957
Or Good Shop? http://www.goodshop.com/?charityid=888957
ID: 1931448 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4395
Credit: 262,502,071
RAC: 598,895
United States
Message 1931460 - Posted: 23 Apr 2018, 0:11:10 UTC - in response to Message 1931448.  

Caught it once before but could not make any conclusions. It's running now and all MB splitters are running with NO AP splitters. There is a lot of "grass" spikes in the file deleters graphs. More than I am accustomed to seeing. I think we are just wildly guessing the purpose at this point and probably would need Eric to actually explain what the process is and when it gets enabled and disabled to understand it.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1931460 · Report as offensive     Reply Quote
James Butler
Volunteer tester

Send message
Joined: 29 Jun 17
Posts: 5
Credit: 1,682,034
RAC: 456
United Kingdom
Message 1931819 - Posted: 25 Apr 2018, 12:30:06 UTC

I am not getting any work today, please can someone look into this, Thankyou
ID: 1931819 · Report as offensive     Reply Quote
Profile Keith Myers Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 29 Apr 01
Posts: 4395
Credit: 262,502,071
RAC: 598,895
United States
Message 1931858 - Posted: 25 Apr 2018, 17:20:38 UTC - in response to Message 1931819.  

Work is flowing fine to everyone else. You are the only one having issues. Post the first 30 lines of the Event Log after first startup of BOINC so we can see if any errors are being logged.
Seti@Home classic workunits:20,676 CPU time:74,226 hours
ID: 1931858 · Report as offensive     Reply Quote
James Butler
Volunteer tester

Send message
Joined: 29 Jun 17
Posts: 5
Credit: 1,682,034
RAC: 456
United Kingdom
Message 1931869 - Posted: 25 Apr 2018, 19:42:38 UTC - in response to Message 1931858.  
Last modified: 25 Apr 2018, 19:42:55 UTC

25/04/2018 15:46:18 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 15:46:18 | SETI@home Beta Test | Requesting new tasks for CPU
25/04/2018 15:46:20 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 15:46:20 | SETI@home Beta Test | Project has no tasks available
25/04/2018 15:52:38 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 15:52:38 | SETI@home Beta Test | Requesting new tasks for NVIDIA GPU
25/04/2018 15:52:40 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 15:52:40 | SETI@home Beta Test | Project has no tasks available
25/04/2018 17:01:32 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 17:01:32 | SETI@home Beta Test | Requesting new tasks for CPU
25/04/2018 17:01:34 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 17:01:34 | SETI@home Beta Test | Project has no tasks available
25/04/2018 17:19:04 | SETI@home Beta Test | Sending scheduler request: To fetch work.
25/04/2018 17:19:04 | SETI@home Beta Test | Requesting new tasks for NVIDIA GPU
25/04/2018 17:19:05 | SETI@home Beta Test | Scheduler request completed: got 0 new tasks
25/04/2018 17:19:05 | SETI@home Beta Test | Project has no tasks available
ID: 1931869 · Report as offensive     Reply Quote
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 11965
Credit: 117,643,277
RAC: 53,882
United Kingdom
Message 1931870 - Posted: 25 Apr 2018, 19:48:43 UTC - in response to Message 1931869.  

Those are requests for SETI@home BETA work, where there is a front-page news item:

We will be out of work for a while

Check the discussion thread at that website for more recent updates.
ID: 1931870 · Report as offensive     Reply Quote
rob smith Special Project $250 donor
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 16071
Credit: 305,989,433
RAC: 311,146
United Kingdom
Message 1931871 - Posted: 25 Apr 2018, 19:49:50 UTC

Ah - you are looking at the SETI Beta project, which has a very low availability of tasks. That project is used to test new applications, or to "prove" selected parts of the servers, and should not be replied on to be a continuous source of tasks.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1931871 · Report as offensive     Reply Quote
Tutankhamon
Volunteer tester
Avatar

Send message
Joined: 1 Nov 08
Posts: 7103
Credit: 44,155,177
RAC: 2,737
Sweden
Message 1931878 - Posted: 25 Apr 2018, 21:00:21 UTC

After the Tuesday scheduled outage, Beta didn't turn on all the servers/processes, so Tasks ready to send=0
See: https://setiweb.ssl.berkeley.edu/beta/server_status.php

And the news item about "We will be out of work for a while", is old. It's from 20 Dec 2017, 1:30:42 UTC.
We've had plenty of work since then.
ID: 1931878 · Report as offensive     Reply Quote
Previous · 1 · 2 · 3 · 4 · 5

Message boards : News : Unexplained database slowness


 
©2018 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.