Project Status - 08/26/2005 4pm PST


log in

Advanced search

Message boards : Number crunching : Project Status - 08/26/2005 4pm PST

1 · 2 · 3 · 4 . . . 8 · Next
Author Message
Profile Rom Walton (BOINC)
Volunteer tester
Avatar
Send message
Joined: 28 Apr 00
Posts: 569
Credit: 106,633
RAC: 15
United States
Message 157794 - Posted: 26 Aug 2005, 22:52:16 UTC

Okay, here is an update.

Right now there are roughly 600,000 files left for the file deleter to deal with. Current estimates say the file deleters should be finished by tomorrow morning.

The reason that the validators are down is because both the validators and file deleters were getting in each others way in the upload directory. So the current line of thinking is let the file deleter queue drain quickly and then attempt to bring the project up and see if the validator queue can hold it’s own after a day or two of the project being up and people have reported in and have new work to crunch. If the validator cannot keep up then the project will be brought back down and the validator queue will be drained and the project will be brought back up.

While it hasn’t been stated, I believe that if S@H is still having problems after all of that then the upload directory is going to be moved to a different device and file system which should be better able to deal with a lot of small files better.


____________
----- Rom
BOINC Development Team, U.C. Berkeley
My Blog

Profile KWSN - MajorKong
Volunteer tester
Avatar
Send message
Joined: 5 Jan 00
Posts: 1406
Credit: 793,366
RAC: 0
United States
Message 157796 - Posted: 26 Aug 2005, 22:55:50 UTC

Thank you for the info, Rom!
____________
“The more corrupt the state, the more numerous the laws.”
― Tacitus, The Annals of Imperial Rome

“Truth is confirmed by inspection and delay; falsehood by haste and uncertainty.”
― Tacitus

Profile Shaktai
Volunteer tester
Avatar
Send message
Joined: 16 Jun 99
Posts: 211
Credit: 259,752
RAC: 0
United States
Message 157797 - Posted: 26 Aug 2005, 22:57:14 UTC

Thanks for the update ROM.

____________

Team MacNN - The best Macintosh team ever.

Blanckaert
Volunteer tester
Send message
Joined: 30 May 99
Posts: 15
Credit: 203,612
RAC: 0
United States
Message 157806 - Posted: 26 Aug 2005, 23:09:30 UTC - in response to Message 157794.

Okay, here is an update.



Thanks Rom... Cut and pasted to Synergy's News... ;)

____________
Mark

Profile The Gas Giant
Volunteer tester
Avatar
Send message
Joined: 22 Nov 01
Posts: 1896
Credit: 2,622,321
RAC: 26
Australia
Message 157816 - Posted: 26 Aug 2005, 23:22:54 UTC

The best little update todate.

Live long and crunch.

____________
Paul
(S@H1 8888)
And proud of it!

Profile [B^S] Paul@home
Volunteer tester
Send message
Joined: 20 Dec 99
Posts: 121
Credit: 1,685,545
RAC: 321
Ireland
Message 157819 - Posted: 26 Aug 2005, 23:24:18 UTC

Cheers Rom!

THanks for the update
____________
Wanna visit BOINC Synergy? Click my stats!

Join BOINC Synergy Team

Profile D.J. Schweitz
Volunteer tester
Avatar
Send message
Joined: 29 Oct 02
Posts: 157
Credit: 871,078
RAC: 0
United States
Message 157825 - Posted: 26 Aug 2005, 23:34:02 UTC

Thanks Rom, posted it to our team forum.
____________
Click below for our Team Website

Joe K
Send message
Joined: 3 Apr 99
Posts: 80
Credit: 17,001
RAC: 0
United States
Message 157827 - Posted: 26 Aug 2005, 23:41:01 UTC

Good info, Thanks
I would just stay offline and clean everything out before restarting. But you guys make the call. Thanks again.Keep up the good work.

Divide Overflow
Volunteer tester
Avatar
Send message
Joined: 3 Apr 99
Posts: 365
Credit: 131,684
RAC: 0
United States
Message 157829 - Posted: 26 Aug 2005, 23:42:54 UTC

Thanks for the update!
____________

dB_the_dba
Send message
Joined: 11 Feb 05
Posts: 27
Credit: 3,402,452
RAC: 155
United States
Message 157840 - Posted: 27 Aug 2005, 0:20:59 UTC

Thanks for the update --

However, I think there will be a heavy surge that will surprize everyone.
With the project offline for several days, a number of us have multuiple work-units ready for upload (I can account for about 15 WUs)
-- that will be a big huge surge.

From my past experience with systems -- i suspect it will take a few days (or a week) for the "steady-state" to come into come back into play.

Hmmm -- Have an idea about how to handle the Antique work unit bug that the project ran into, possible to have the client do a little handshaking with the Scheduler first (say send the WU id etc) .. If within valid date widow, then you can ack the request to send the WU. Sure, its a little more work on the front end (Hits the database some more) but would eliminate the antiques thus reducing the effects on the array. (can't imagine what the inodes look like lol )


Just an idea,

darryl (db_the_DBA )

____________

Profile [HWU] GHz & CO. - BOINC.Italy
Volunteer tester
Avatar
Send message
Joined: 1 Jul 02
Posts: 139
Credit: 1,279,729
RAC: 0
Italy
Message 157844 - Posted: 27 Aug 2005, 0:33:52 UTC - in response to Message 157794.

Thanks for the update Rom!

So the current line of thinking is let the file deleter queue drain quickly


How much time need the validator to finish the deletion of the files?
____________
GHz
BOINC.Italy

Martin Johnson
Send message
Joined: 9 Jun 01
Posts: 201
Credit: 224,995
RAC: 0
United Kingdom
Message 157856 - Posted: 27 Aug 2005, 1:11:39 UTC

I find it interesting that while the validators are off, and the deleters are chugging away, the "Ready to Send" queue is growing. Perhaps the deleters are finding some work units that need more crunching, and don't need to be deleted yet.

Profile Rom Walton (BOINC)
Volunteer tester
Avatar
Send message
Joined: 28 Apr 00
Posts: 569
Credit: 106,633
RAC: 15
United States
Message 157858 - Posted: 27 Aug 2005, 1:24:26 UTC

Actually it is the various results which have passed their deadline which are timed out and rescheduled.

____________
----- Rom
BOINC Development Team, U.C. Berkeley
My Blog

Profile John Cropper
Avatar
Send message
Joined: 3 May 00
Posts: 444
Credit: 416,933
RAC: 0
United States
Message 157882 - Posted: 27 Aug 2005, 2:32:45 UTC - in response to Message 157840.

I think there will be a heavy surge that will surprize everyone.
With the project offline for several days, a number of us have multuiple work-units ready for upload (I can account for about 15 WUs)
-- that will be a big huge surge.


I only wish I had fifteen...my current counts:

Sitting in Berkeley waiting for validation - 206
Sitting on my network waiting to upload as of 0230 UTC - 180 (this increases by about 70/day)

Even if the average client crunches 4 WU/day and the active client counts are in the 300K range, that's 1.2 million WUs at an average size of 15K apiece or 18 gigs of data to contend with, an interesting backlog to say the least!

____________

Stewie: So, is there any tread left on the tires? Or at this point would it be like throwing a hot dog down a hallway?

Fox Sunday (US) at 9PM ET/PT

EclipseHA
Send message
Joined: 28 Jul 99
Posts: 1018
Credit: 233,810
RAC: 128
United States
Message 157917 - Posted: 27 Aug 2005, 3:28:43 UTC - in response to Message 157794.

Okay, here is an update.

Right now there are roughly 600,000 files left for the file deleter to deal with. Current estimates say the file deleters should be finished by tomorrow morning.


Will anybody be around to fire up the validators and/or scheduler before Monday (once the deleters are done)?

If not, it seems that 1-2 days of draining the WFV queue might be lost.

____________

Profile Kajunfisher
Volunteer tester
Avatar
Send message
Joined: 29 Mar 05
Posts: 1407
Credit: 126,476
RAC: 0
United States
Message 157928 - Posted: 27 Aug 2005, 3:43:46 UTC - in response to Message 157917.

Okay, here is an update.

Right now there are roughly 600,000 files left for the file deleter to deal with. Current estimates say the file deleters should be finished by tomorrow morning.


Will anybody be around to fire up the validators and/or scheduler before Monday (once the deleters are done)?

If not, it seems that 1-2 days of draining the WFV queue might be lost.


More than likely Matt will. It seems they tend to take turns working weekends.

EclipseHA
Send message
Joined: 28 Jul 99
Posts: 1018
Credit: 233,810
RAC: 128
United States
Message 157936 - Posted: 27 Aug 2005, 3:58:48 UTC - in response to Message 157928.

Okay, here is an update.

Right now there are roughly 600,000 files left for the file deleter to deal with. Current estimates say the file deleters should be finished by tomorrow morning.


Will anybody be around to fire up the validators and/or scheduler before Monday (once the deleters are done)?

If not, it seems that 1-2 days of draining the WFV queue might be lost.


More than likely Matt will. It seems they tend to take turns working weekends.



Actually, it seems that no one regularly works weekends at UCB! That seems to be the mode over the last year! That's why I ask if the validators might be fired up before monday! (10am PDT - it gives the syops time to have a coffee after getting to work!)
____________

chrissard
Send message
Joined: 6 Apr 01
Posts: 8
Credit: 5,540,695
RAC: 2,469
United States
Message 157976 - Posted: 27 Aug 2005, 5:50:04 UTC - in response to Message 157840.

Thanks for the update --

However, I think there will be a heavy surge that will surprize everyone.
With the project offline for several days, a number of us have multuiple work-units ready for upload (I can account for about 15 WUs)
-- that will be a big huge surge.


I am sitting on about 50 WU and have nothing left. So I switched over to seti classic and am receiving all the work I can process.

____________
<img src="http://boinc.mundayweb.com/seti2/stats.php?userID=1899" border="0"> <img src="http://seti.mundayweb.com/stats.php?userID=567" border="0">
<img src="http://sero.cwsurf.de/dc.php/user/chrissard/project/ud.png">

Profile The Gas Giant
Volunteer tester
Avatar
Send message
Joined: 22 Nov 01
Posts: 1896
Credit: 2,622,321
RAC: 26
Australia
Message 158009 - Posted: 27 Aug 2005, 7:26:31 UTC - in response to Message 157858.

Actually it is the various results which have passed their deadline which are timed out and rescheduled.


Yep I have 6 or 7 that for one reason or another have not been able to either upload or report due to this extended outage...damn, double damn and bugger! But that only accounts for about 5hrs of crunching, so not so bad....

Live long and crunch.

____________
Paul
(S@H1 8888)
And proud of it!

Profile Scarecrow
Avatar
Send message
Joined: 15 Jul 00
Posts: 4382
Credit: 458,880
RAC: 0
United States
Message 158011 - Posted: 27 Aug 2005, 7:37:17 UTC - in response to Message 157936.

Actually, it seems that no one regularly works weekends at UCB! That seems to be the mode over the last year! That's why I ask if the validators might be fired up before monday! (10am PDT - it gives the syops time to have a coffee after getting to work!)

You mean people actually work on weekends?? (He says as he stares at his smurf-like blue fingers made that way by getting beeped out at 00:30 Saturday morning to replace a ribbon drive on a ((get this)) mission critical Printronix P600 line printer.) 02:30am CDT - gives the field grunt time for a beer before it's time for breakfast. :)

1 · 2 · 3 · 4 . . . 8 · Next

Message boards : Number crunching : Project Status - 08/26/2005 4pm PST

Copyright © 2014 University of California