Question About Report Function

Message boards : Number crunching : Question About Report Function
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 963789 - Posted: 16 Jan 2010, 14:15:40 UTC

If left alone, how often does BOINC "report" (i.e., UPDATE)? Is this on a schedule? Or what?
(Obviously, I mean in the case where the servers are working properly, and not down).

Normally, when the servers are working OK, uploads occur as WUs are finished, but the user credit totals are not changed until the Update function is performed. What determines when BOINC asks for an Update, and Reports the WUs that have been uploaded?

For example: 1/16/2010 2:07:15 AM SETI@home Reporting 80 completed tasks, requesting new tasks for CPU

But it doesn't always do this only when asking for new WUs, does it?

ID: 963789 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 963794 - Posted: 16 Jan 2010, 14:38:07 UTC - in response to Message 963789.  

Mine will ask for work without reporting...Take a look thru this thread as that was talked about quite a bit but a bit tecky...OMG please fix the scheduler

Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 963794 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 963796 - Posted: 16 Jan 2010, 14:38:57 UTC - in response to Message 963789.  

"Borrowed" from JM7's post:

BOINC returns work to a project at the first of:

1) 24 hours before deadline.
2) Connect Every X before deadline.
3) 24 hours after completion.
4) 60 seconds after the completion of upload if upload completes after any of 1 to 3.
5) On a work request to that project.
6) With any other reported work for that project.
7) On a trickle up to the project (CPDN only as far as I know).
8) On a trickle down request (no projects use this as far as I know).
9) On a server specified schedule (some project uses this, but not certain which).
10) On a BOINC Account Manager specified connection (only if using a BOINC Account Manager).
11) When the user pushes the update button.

ID: 963796 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 963804 - Posted: 16 Jan 2010, 15:16:48 UTC - in response to Message 963796.  

"Borrowed" from JM7's post:

BOINC returns work to a project at the first of:

1) 24 hours before deadline.
2) Connect Every X before deadline.
3) 24 hours after completion.
4) 60 seconds after the completion of upload if upload completes after any of 1 to 3.
5) On a work request to that project.
6) With any other reported work for that project.
7) On a trickle up to the project (CPDN only as far as I know).
8) On a trickle down request (no projects use this as far as I know).
9) On a server specified schedule (some project uses this, but not certain which).
10) On a BOINC Account Manager specified connection (only if using a BOINC Account Manager).
11) When the user pushes the update button.


Tx, ozzfan BUT:
I don't quite understand. Do you mean BOINC Updates in these circumstances? What is the "deadline" referred to in 1) and 2) and what is the "completion" referred to in 3)? AND: does this mean that BOINC will automatically update after an interval of no more than 24 hours since the last Update (that's what I was getting at)?

ID: 963804 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 963807 - Posted: 16 Jan 2010, 15:27:34 UTC

Courtesy of Crunch3r.....most of my rigs report every few moments........

And the Cuda rigs have been instructed, due to the instructions provided, to talk it up on a regular basis too......

Yeh, I do ramp the server traffic up a bit.........
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 963807 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 963809 - Posted: 16 Jan 2010, 15:36:23 UTC - in response to Message 963807.  
Last modified: 16 Jan 2010, 15:40:16 UTC

Courtesy of Crunch3r.....most of my rigs report every few moments........

And the Cuda rigs have been instructed, due to the instructions provided, to talk it up on a regular basis too......

Yeh, I do ramp the server traffic up a bit.........


Actually, what I am trying to find out is: if left alone, what is the maximum amount of time between Updates by BOINC (for SETI, where my machines process WUs in a few hours at most)? (Assuming that all is OK with the servers and there are no Berkeley network issues).
ID: 963809 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14653
Credit: 200,643,578
RAC: 874
United Kingdom
Message 963814 - Posted: 16 Jan 2010, 15:53:53 UTC - in response to Message 963804.  

"Borrowed" from JM7's post:

BOINC returns work to a project at the first of:

1) 24 hours before deadline.
2) Connect Every X before deadline.
3) 24 hours after completion.
4) 60 seconds after the completion of upload if upload completes after any of 1 to 3.
5) On a work request to that project.
6) With any other reported work for that project.
7) On a trickle up to the project (CPDN only as far as I know).
8) On a trickle down request (no projects use this as far as I know).
9) On a server specified schedule (some project uses this, but not certain which).
10) On a BOINC Account Manager specified connection (only if using a BOINC Account Manager).
11) When the user pushes the update button.


Tx, ozzfan BUT:
I don't quite understand. Do you mean BOINC Updates in these circumstances? What is the "deadline" referred to in 1) and 2) and what is the "completion" referred to in 3)? AND: does this mean that BOINC will automatically update after an interval of no more than 24 hours since the last Update (that's what I was getting at)?

1) and 2) The 'deadline' referred to is the deadline shown in BOINC Manager and on these web pages for each task - if you don't report the task by that time, it's re-issued to another computer, wasting everybody's time and bandwidth. I'm presuming (but have no certain knowledge) that when multiple tasks have been completed and are ready to report, the earliest of their respective deadlines will be the one considered.

3) 'Completion' is when a task reached 100% progress and stops running, handing over over to the next in the queue. Again, I'm not exactly certain whether the 24 hours is counted from when computation reaches 100%, or when the subsequent upload completes and the status changes to 'Ready to report' - usually, that's only a few seconds difference, in any event.

With regard to your underlying question - no, that doesn't mean that BOINC will report every 24 hours. When an update happens (automatic or manual), all tasks 'Ready to report' are reported in one go: there will be no completed tasks remaining on your system. JM7's conditions will only come into effect when the next task completes, and starts the countdown afresh. If your next SETI task doesn't complete for another two hours, the interval between updates will be 26 hours, not 24.
ID: 963814 · Report as offensive
Profile Keith T.
Volunteer tester
Avatar

Send message
Joined: 23 Aug 99
Posts: 962
Credit: 537,293
RAC: 9
United Kingdom
Message 963817 - Posted: 16 Jan 2010, 16:01:18 UTC - in response to Message 963809.  

...
Actually, what I am trying to find out is: if left alone, what is the maximum amount of time between Updates by BOINC (for SETI, where my machines process WUs in a few hours at most)? (Assuming that all is OK with the servers and there are no Berkeley network issues).


I think it is 24 hours, assuming that you have completed at least 1 task within those 24 hours.

AFIK, completed means finished crunching, and uploaded sucessfully.

Deadline is the date in BOINC Manager when the task is due by, this varies with each task according to Angle Range. It is usually between 14 days and ~2 months.
ID: 963817 · Report as offensive
Profile Michael Goetz
Avatar

Send message
Joined: 14 May 99
Posts: 56
Credit: 622,268
RAC: 0
United States
Message 963834 - Posted: 16 Jan 2010, 16:43:22 UTC - in response to Message 963809.  


Actually, what I am trying to find out is: if left alone, what is the maximum amount of time between Updates by BOINC (for SETI, where my machines process WUs in a few hours at most)? (Assuming that all is OK with the servers and there are no Berkeley network issues).


24 hours.

That's the maximum amount of time the BOINC client will wait before attempting to report a completed result back to the project.

As the earlier posts have said, there's a lot of conditions that will cause it to report earlier.

There is also a flag you can place into the <options> section of your cc_config.xml file that will cause BOINC to always report results as quickly as possible (usually after only a one minute delay.):

<report_results_immediately>1</report_results_immediately>

Note that for some projects (SETI in particular) this is bad for the project itself as it increases the load on their network and servers, especially if you have a large queue of tasks. Conversely, for other projects it's beneficial (GPUGRID, for example), because they need to get results back quickly in order to use that data to generate the following WUs. Unfortunately, you can't set this flag on a project by project basis.

Your RAC isn't affected by this flag as the same amount of work gets done either way.
Want to find one of the largest known primes? Try PrimeGrid. Or help cure disease at WCG.

ID: 963834 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 963859 - Posted: 16 Jan 2010, 17:59:58 UTC - in response to Message 963809.  
Last modified: 16 Jan 2010, 18:00:21 UTC

Actually, what I am trying to find out is: if left alone, what is the maximum amount of time between Updates by BOINC (for SETI, where my machines process WUs in a few hours at most)? (Assuming that all is OK with the servers and there are no Berkeley network issues).

The maximum amount of time, everything functioning, is 24 hours.

Think of the upload and download server(s) as a bucket. Files are thrown into the bucket and taken out of the bucket. It's entirely anonymous, and simple.

Whenever BOINC touches the scheduler, it has to connect to the database, find your entry in the table, look up the work units your reporting in that table, etc. There isn't a big difference between reporting one work unit, and reporting more than one -- it is a lot more efficient.

If reporting and requesting new work get combined, that's even better from a throughput standpoint.

Mark can talk about the scheduler bandwidth his hacked 5.x BOINC uses, but it's database activity that needs to be optimized.
ID: 963859 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1513
Credit: 370,893,186
RAC: 340
United States
Message 963896 - Posted: 16 Jan 2010, 19:43:51 UTC

OK - I think I got it now - 24 hours (max) between automatic updates by BOINC.

Thanks to all of you for your help,

Jon
ID: 963896 · Report as offensive
Norwich Gadfly
Avatar

Send message
Joined: 29 Dec 08
Posts: 100
Credit: 488,414
RAC: 0
United Kingdom
Message 964060 - Posted: 17 Jan 2010, 11:11:35 UTC - in response to Message 963896.  

Sorry to appear thick, but why does the client have to report a result after it has uploaded it ? Why can't the upload do the reporting ?
ID: 964060 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19080
Credit: 40,757,560
RAC: 67
United Kingdom
Message 964065 - Posted: 17 Jan 2010, 11:42:37 UTC

Think it is about time this info on reporting and requesting by Rom is brought up again.

BOINC Client: The evils of 'Returning Results Immediately' http://www.romwnet.org/dasblogce/PermaLink,guid,143d4295-8e5f-4361-a883-18ffad817094.aspx
ID: 964065 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14653
Credit: 200,643,578
RAC: 874
United Kingdom
Message 964070 - Posted: 17 Jan 2010, 11:52:53 UTC - in response to Message 964065.  

Think it is about time this info on reporting and requesting by Rom is brought up again.

BOINC Client: The evils of 'Returning Results Immediately' http://www.romwnet.org/dasblogce/PermaLink,guid,143d4295-8e5f-4361-a883-18ffad817094.aspx

I wonder whether we could get Rom to re-visit that blog (from October 2006), and update the numbers?

He says:

On average jocelyn is processing 314 queries per second.

Jocelyn is a "Sun V40z (4 x 2.2GHz Opteron, 28 GB RAM)"

The master database function is now run on mork, "Intel Server (4 x six-core 2.13GHz Xeon, 64 GB RAM)", handling 681 queries per second as I write.

Seems like we have slightly more slack in the system than we did back in the day.
ID: 964070 · Report as offensive
W-K 666 Project Donor
Volunteer tester

Send message
Joined: 18 May 99
Posts: 19080
Credit: 40,757,560
RAC: 67
United Kingdom
Message 964074 - Posted: 17 Jan 2010, 12:05:20 UTC - in response to Message 964070.  

Think it is about time this info on reporting and requesting by Rom is brought up again.

BOINC Client: The evils of 'Returning Results Immediately' http://www.romwnet.org/dasblogce/PermaLink,guid,143d4295-8e5f-4361-a883-18ffad817094.aspx

I wonder whether we could get Rom to re-visit that blog (from October 2006), and update the numbers?

He says:

On average jocelyn is processing 314 queries per second.

Jocelyn is a "Sun V40z (4 x 2.2GHz Opteron, 28 GB RAM)"

The master database function is now run on mork, "Intel Server (4 x six-core 2.13GHz Xeon, 64 GB RAM)", handling 681 queries per second as I write.

Seems like we have slightly more slack in the system than we did back in the day.

Don't know that there is that much slack, even with newer server, when Rom wrote that in Oct 2006, we were still on enhanced, not MB, and definitely no CUDA app. There are quite a few more tasks processed these days.
ID: 964074 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14653
Credit: 200,643,578
RAC: 874
United Kingdom
Message 964081 - Posted: 17 Jan 2010, 12:19:45 UTC - in response to Message 964074.  

Think it is about time this info on reporting and requesting by Rom is brought up again.

BOINC Client: The evils of 'Returning Results Immediately' http://www.romwnet.org/dasblogce/PermaLink,guid,143d4295-8e5f-4361-a883-18ffad817094.aspx

I wonder whether we could get Rom to re-visit that blog (from October 2006), and update the numbers?

He says:

On average jocelyn is processing 314 queries per second.

Jocelyn is a "Sun V40z (4 x 2.2GHz Opteron, 28 GB RAM)"

The master database function is now run on mork, "Intel Server (4 x six-core 2.13GHz Xeon, 64 GB RAM)", handling 681 queries per second as I write.

Seems like we have slightly more slack in the system than we did back in the day.

Don't know that there is that much slack, even with newer server, when Rom wrote that in Oct 2006, we were still on enhanced, not MB, and definitely no CUDA app. There are quite a few more tasks processed these days.

And we didn't have Astropulse either, which would - if only they could get that pesky science database sorted out - drive things in the opposite direction.

Fortunately, the vast majority of SETI users don't use optimised apps, and hence probably don't use deprecated practices like RRI either. Remember that analysis I did at Beta, just before AP was transferred to the main project?


(direct link)

Optimised applications (MB by that stage, but no AP or CUDA) are the faint blue mist to the right of the main population: a rare breed indeed, despite the disproportionate interest amongst us chattering classes.
ID: 964081 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 964083 - Posted: 17 Jan 2010, 12:27:42 UTC - in response to Message 964081.  
Last modified: 17 Jan 2010, 12:28:19 UTC

Think it is about time this info on reporting and requesting by Rom is brought up again.

BOINC Client: The evils of 'Returning Results Immediately' http://www.romwnet.org/dasblogce/PermaLink,guid,143d4295-8e5f-4361-a883-18ffad817094.aspx

I wonder whether we could get Rom to re-visit that blog (from October 2006), and update the numbers?

He says:

On average jocelyn is processing 314 queries per second.

Jocelyn is a "Sun V40z (4 x 2.2GHz Opteron, 28 GB RAM)"

The master database function is now run on mork, "Intel Server (4 x six-core 2.13GHz Xeon, 64 GB RAM)", handling 681 queries per second as I write.

Seems like we have slightly more slack in the system than we did back in the day.

Don't know that there is that much slack, even with newer server, when Rom wrote that in Oct 2006, we were still on enhanced, not MB, and definitely no CUDA app. There are quite a few more tasks processed these days.

And we didn't have Astropulse either, which would - if only they could get that pesky science database sorted out - drive things in the opposite direction.

Fortunately, the vast majority of SETI users don't use optimised apps, and hence probably don't use deprecated practices like RRI either. Remember that analysis I did at Beta, just before AP was transferred to the main project?


(direct link)

Optimised applications (MB by that stage, but no AP or CUDA) are the faint blue mist to the right of the main population: a rare breed indeed, despite the disproportionate interest amongst us chattering classes.

I would be in the 'faint blue mist' then. Guess I always have been.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 964083 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 964124 - Posted: 17 Jan 2010, 16:10:34 UTC - in response to Message 964060.  

Sorry to appear thick, but why does the client have to report a result after it has uploaded it ? Why can't the upload do the reporting ?


Because uploads are simple. Database accesses are expensive and slow, and thus are best batched together.
ID: 964124 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 964142 - Posted: 17 Jan 2010, 17:35:20 UTC - in response to Message 964124.  

Sorry to appear thick, but why does the client have to report a result after it has uploaded it ? Why can't the upload do the reporting ?


Because uploads are simple. Database accesses are expensive and slow, and thus are best batched together.

... and because the report tells the scheduler that the work is ready for validation.

If the report came before the upload, then the servers would have a list of "almost ready for validation" work -- and they'd have to keep checking the list to see if the uploads were completed.

Upload, then report is simpler, and that means more capacity.
ID: 964142 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 964145 - Posted: 17 Jan 2010, 17:47:53 UTC - in response to Message 964065.  

Think it is about time this info on reporting and requesting by Rom is brought up again.

BOINC Client: The evils of 'Returning Results Immediately' http://www.romwnet.org/dasblogce/PermaLink,guid,143d4295-8e5f-4361-a883-18ffad817094.aspx

While the details have changed (scheduler and database on better hardware, for example) the logic stays the same.

It isn't based on bandwidth, although I think we all benefit from a scheduler that isn't so busy.

Rom gave the number of query parts, but not the sums.

Reporting 20 results, one at a time, does 1285.4 query parts across 20 scheduler sessions.

Reporting 20 results, in a single session, does 389.36 query parts across 1 session.

Again, the savings on sessions is nice, but it's the amount repetitive database work that the batch queries avoid.

... and arguing that we have better hardware doesn't change the fact that for this scenario, reporting results immediately is three times more work for the same result.
ID: 964145 · Report as offensive
1 · 2 · 3 · Next

Message boards : Number crunching : Question About Report Function


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.