Transitioner backlog

Message boards : Number crunching : Transitioner backlog
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Qui-Gon
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 2940
Credit: 19,199,902
RAC: 11
United States
Message 134395 - Posted: 8 Jul 2005, 18:57:08 UTC

What is happening with the transitioner? What is causing the backlog (all four seem to be running) and what is the effect?
ID: 134395 · Report as offensive
Profile Qui-Gon
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 2940
Credit: 19,199,902
RAC: 11
United States
Message 134442 - Posted: 8 Jul 2005, 22:14:34 UTC

I wonder if I'm not asking this correctly? I have done a search looking for an answer, and though some have talked about the transitioner, usually someone else will come along and give a different opinion.

Six hours ago I uploaded about 20 results and over that six hour period my credit has increased by only about 40. Is this due to the transitioner being behind? I have seen opinions either way here (that the transitioner is the cause of the delay).
ID: 134442 · Report as offensive
Pascal, K G
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 2343
Credit: 150,491
RAC: 0
United States
Message 134448 - Posted: 8 Jul 2005, 22:23:42 UTC

Transitioner: Handles state transitions of workunits and results. Basically, the transitioners keep track of the many, many results in progress and makes sure they properly move down the pipeline. It is always asking the questions: Is this workunit ready to send out? Has this result been received yet? Is this a valid result? Can we delete it now? Since there are always a large set of results in progress, we need to run several transitioners in tandem to keep up with the workunit/result flow. The Waiting to transition queue tells you how many workunits/results are waiting to move down the pipeline. A large number means there is a problem somewhere in our backend server system.
Semper Eadem
So long Paul, it has been a hell of a ride.

Park your ego's, fire up the computers, Science YES, Credits No.
ID: 134448 · Report as offensive
Profile Qui-Gon
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 2940
Credit: 19,199,902
RAC: 11
United States
Message 134467 - Posted: 8 Jul 2005, 23:12:20 UTC - in response to Message 134448.  

Transitioner: Handles state transitions of workunits and results. Basically, the transitioners keep track of the many, many results in progress and makes sure they properly move down the pipeline. It is always asking the questions: Is this workunit ready to send out? Has this result been received yet? Is this a valid result? Can we delete it now? Since there are always a large set of results in progress, we need to run several transitioners in tandem to keep up with the workunit/result flow. The Waiting to transition queue tells you how many workunits/results are waiting to move down the pipeline. A large number means there is a problem somewhere in our backend server system.


Thank you, Blank,

I have read this description and is seems somewhat out of date, saying "The Waiting to transition queue tells you how many workunits/results are waiting to move down the pipeline.", but currently there is no number showing how many workunits are waiting, just a figure saying how long, in hours, the backlog is. I have seen nothing in the Technical news or on the front page news column that talks about the backlog and how it may be affecting the granting of credit.
ID: 134467 · Report as offensive
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 134496 - Posted: 9 Jul 2005, 0:16:05 UTC

> but currently there is no number showing how many workunits are waiting, just a
> figure saying how long, in hours, the backlog is.

The reason for that is that a query that counts anything that is stored in innodb database tables (one of many types of data storage supported by mysql) is an expensive query - it actually sequentially scans over the whole table to add everything up.

However.. all we have to do is ask a transitioner what it's currently working on and do a delta between now and the last modified time of the result to get a pretty good idea how far behind the transtioner is, at least in the time domain.

Anyway.. in general, if the transitioner is behind, everything is behind. Of course, more things than others depending on the situation. If the transitioner is behind AND the validator queue is high, then credit processing is doubly slow. As of now, I think only the transitioner is behind, so the credit processing is only that many hours behind.

- Matt
-- BOINC/SETI@home network/web/science/development person
-- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude
ID: 134496 · Report as offensive
Profile Qui-Gon
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 2940
Credit: 19,199,902
RAC: 11
United States
Message 134498 - Posted: 9 Jul 2005, 0:19:21 UTC - in response to Message 134496.  

Anyway.. in general, if the transitioner is behind, everything is behind. Of course, more things than others depending on the situation. If the transitioner is behind AND the validator queue is high, then credit processing is doubly slow. As of now, I think only the transitioner is behind, so the credit processing is only that many hours behind.
- Matt


That's very helpful. Thanks Matt.
ID: 134498 · Report as offensive
Profile MikeSW17
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 1603
Credit: 2,700,523
RAC: 0
United Kingdom
Message 134709 - Posted: 9 Jul 2005, 9:06:07 UTC

At least whatever it was is sorted. No Backlog, and a healthy Ready to Send queue for the first time in a week.
I Wonder what was holding everything up.

ID: 134709 · Report as offensive

Message boards : Number crunching : Transitioner backlog


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.