WU flow / validation / etc. status

Message boards : Number crunching : WU flow / validation / etc. status
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Matt Lebofsky
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 1 Mar 99
Posts: 1444
Credit: 957,058
RAC: 0
United States
Message 33726 - Posted: 7 Oct 2004, 18:31:34 UTC

Here an update on several fronts:

Despite all the favorable upgrades this week, the validator queue was still growing ever so slowly. Credits were being granted, but we couldn't keep up with demand.

However, sometime yesterday afternoon we finally hit a "sweet spot" where enough work was successfully getting through for long enough a time that the validator is now catching up, and the queue has been shrinking ever so slowly. Phew. Some programming changes to make the validator queries even more efficient may be put into effect soon, which should help credit granted happen even faster.

The upload/download disks are still sputtering, but they are working more or less. The upshot of this is that every so often they freeze up for a few minutes, and clients can't connect during that time.

As much as I'd like to, we can't migrate off these disks just yet because (1) we don't have disks space anywhere else, and (2) even if we did have disk space somewhere else, it takes a long time to move half a terabyte of data, during which the project would be down.

The bright side is that with the whole rest of the system working (especially the validator), the amount of data on these disks is shrinking very quickly. That is because work is being validated faster than it is arriving now, and after the data is assimilated into our database, we can delete the multiple results from the upload directory.

So.. as those disks contain less and less data, the demand will decrease, which will hopefully stop it from freaking out. As well, less data means more possibilities if and when we do have to migrate (and less downtime to copy the data over).

- Matt
BOINC/SETI@home
ID: 33726 · Report as offensive
Profile Benher
Volunteer developer
Volunteer tester

Send message
Joined: 25 Jul 99
Posts: 517
Credit: 465,152
RAC: 0
United States
Message 33727 - Posted: 7 Oct 2004, 18:35:06 UTC

You da man Matt,

Whole seti team is...wait..that doesn't make sense =P

Every day...and in every way..we are getting better and better. Now please take that white jacket with the very long sleeves off of me.
ID: 33727 · Report as offensive
Honie

Send message
Joined: 22 Jan 04
Posts: 141
Credit: 29,681,066
RAC: 0
Germany
Message 33731 - Posted: 7 Oct 2004, 18:36:58 UTC

Hey, thanks for the info.

But just one more question.

What will happen with the SNAP device ?

Right now it's like a Porsche just for driving for shopping.
I mean, did the SNAP folks analyse the errors, and is
there a time plan, when it will do the work it should do?
ID: 33731 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 33762 - Posted: 7 Oct 2004, 19:40:34 UTC

Matt,

Here is another suggestion. I don't think that there is a huge need to fill up the news with small details, might it be possible to have a set of pages, sort of like the old Technical News pages where observations like this coould be recorded?

And I mean as a "log" of activities. When I was in the Navy we used to keep all sorts of logs so that we could keep track of different aspects. If notes like this one were done every couple of days that would be a big help.

Automated pages like the status page could be exended to track things like file counts which then show the basic "health" of the system. Such as 5,000 results pending validation, 4,000 assimilated, etc. with automatic charts (like on the old Classic site) ...

I mean, there is information, but it is not systematically collected or displayed. Heck, these would be invaluable tools for you folks to keep tabs on the basic behavior of the system.

On one job I worked the boss could never figure out how I knew that something bad had happened with the server... but all I did was listen to the disk drives, when they sounded "funny" something bad had happened. But I also had a complete set of transaction logs from the night before to make sure tha tthe system had run ok overnight ...
<p>
For BOINC Documentation: Click Me!


ID: 33762 · Report as offensive
Profile Bruno Moretti IK2WQA
Avatar

Send message
Joined: 15 May 99
Posts: 284
Credit: 49,167
RAC: 0
Italy
Message 33797 - Posted: 7 Oct 2004, 20:31:51 UTC - in response to Message 33762.  
Last modified: 7 Oct 2004, 20:46:37 UTC

> Matt,
>
> Here is another suggestion. I don't think that there is a huge need to fill
> up the news with small details, might it be possible to have a set of pages,
> sort of like the old Technical News pages where observations like this coould
> be recorded?
>
> And I mean as a "log" of activities. When I was in the Navy we used to keep
> all sorts of logs so that we could keep track of different aspects. If notes
> like this one were done every couple of days that would be a big help.


Well, Paul! [/b][/url]
ID: 33797 · Report as offensive

Message boards : Number crunching : WU flow / validation / etc. status


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.