Uploading result vs. returning result

Message boards : Number crunching : Uploading result vs. returning result
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Profile Martin A. Boegelund
Volunteer tester
Avatar

Send message
Joined: 4 Jul 00
Posts: 292
Credit: 387,485
RAC: 1
Denmark
Message 184091 - Posted: 30 Oct 2005, 16:53:24 UTC

I think this is weird:

Whenever I finish a workunit, my SETI client tells me that it's "Starting upload..." and "Finished upload...", but the status page at Berkeley doesn't change for my result, as if it hasn't been uploaded. Also KBoincspy indicates that the finished result/WU is residing in my cache.

So if I select to "Update project" my client tell me that it is returning a result. And after that, my status page changes for that WU.

Anyone who can explain this non-uploading?

"Are you suggesting coconuts migrate?"

ID: 184091 · Report as offensive
nick
Volunteer tester
Avatar

Send message
Joined: 22 Jul 05
Posts: 284
Credit: 3,902,174
RAC: 0
United States
Message 184108 - Posted: 30 Oct 2005, 17:39:24 UTC

i get that to but just on my fast computer.


ID: 184108 · Report as offensive
Profile MJKelleher
Volunteer tester
Avatar

Send message
Joined: 1 Jul 99
Posts: 2048
Credit: 1,575,401
RAC: 0
United States
Message 184110 - Posted: 30 Oct 2005, 17:48:12 UTC - in response to Message 184091.  

Anyone who can explain this non-uploading?

Uploading and reporting are two separate actions. Your account doesn't reflect on upload, only on report.

How are Results reported back to the project? from the BOINC FAQ in the Wiki gives more detail, and links to other information.

MJ

ID: 184110 · Report as offensive
nick
Volunteer tester
Avatar

Send message
Joined: 22 Jul 05
Posts: 284
Credit: 3,902,174
RAC: 0
United States
Message 184112 - Posted: 30 Oct 2005, 17:52:42 UTC

but when you upload you get credit.


ID: 184112 · Report as offensive
Astro
Volunteer tester
Avatar

Send message
Joined: 16 Apr 02
Posts: 8026
Credit: 600,015
RAC: 0
Message 184146 - Posted: 30 Oct 2005, 19:16:36 UTC - in response to Message 184112.  

but when you upload you get credit.

When did this start? I thought I had to be reported and then the validator had to get to it to issue credit. you may see a credit increase, but I think that is for work already uploaded, reported and credited from your last connection period.
ID: 184146 · Report as offensive
Profile Tern
Volunteer tester
Avatar

Send message
Joined: 4 Dec 03
Posts: 1122
Credit: 13,376,822
RAC: 44
United States
Message 184170 - Posted: 30 Oct 2005, 20:03:09 UTC - in response to Message 184112.  

but when you upload you get credit.


No - you get credit when the validator runs against that WU. Which won't happen until you and two others have reported.
ID: 184170 · Report as offensive
Profile Misfit
Volunteer tester
Avatar

Send message
Joined: 21 Jun 01
Posts: 21804
Credit: 2,815,091
RAC: 0
United States
Message 184171 - Posted: 30 Oct 2005, 20:07:01 UTC - in response to Message 184112.  

but when you upload you get credit.

Your Boinc Manager may show an increase in credit when uploading but that's because it's picked up the new info when it connected to the server.
ID: 184171 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 184948 - Posted: 2 Nov 2005, 0:40:03 UTC - in response to Message 184170.  
Last modified: 2 Nov 2005, 0:46:45 UTC

but when you upload you get credit.


No - you get credit when the validator runs against that WU. Which won't happen until you and two others have reported.


I think what the original poster was getting at is that results
are being uploaded but not reported. There appears
to be a difference.

I think 5.x.x may be behaving differently than 4.45. I'm reading the wiki
to try to understand it...

Ah, yes, now I understand:

Note: There is a bug in the 4.4x Versions of the BOINC Client Software that causes Results to be reported immediately after the status changes to Ready to Report.

This explains why I'm seeing a change vs. 4.4x... LOL!

Brian
ID: 184948 · Report as offensive
Profile Crunch3r
Volunteer tester
Avatar

Send message
Joined: 15 Apr 99
Posts: 1546
Credit: 3,438,823
RAC: 0
Germany
Message 184953 - Posted: 2 Nov 2005, 0:46:22 UTC - in response to Message 184948.  
Last modified: 2 Nov 2005, 0:48:57 UTC

but when you upload you get credit.


No - you get credit when the validator runs against that WU. Which won't happen until you and two others have reported.


I think what the original poster was getting at is that results
are being uploaded but not reported. There appears
to be a difference. I came home today briefly just before 11:00AM EST
(16:00GMT) because I thought my computer had perhaps locked up because it had
not reported anything since I left for work. I'm only about 5 minutes from
work, so I figured I could take a "break" and come check on it. Well, my
computer was just crunching away and had results "Ready to Report". If you'll
look at my stats (hard to do because of so many results right now, but anyway),
you'll notice that I "reported" 3 work units at the same time, 15:54:57 UTC.
This is when I manually hit Update in the BOINC Manager... Trux 5.3.1...
After I hit Update, I checked the web page and it showed me as having
reported, even though they were being uploaded as they were being
completed. I think this is a 5.x.x behavior as I don't
remember it doing that on 4.45, but I guess it might've...

Brian



This behavior was introduced in boinc 4.45 and above. Previously it was cosidered a bug in the code (uploading and reporting at once).

In my point of view the new behavior of uploading a result and reporting it later is a bug and not a feature.

But that's only one of a lot of changes (bad ones) made to the client :-(



Join BOINC United now!
ID: 184953 · Report as offensive
Profile Crunch3r
Volunteer tester
Avatar

Send message
Joined: 15 Apr 99
Posts: 1546
Credit: 3,438,823
RAC: 0
Germany
Message 184956 - Posted: 2 Nov 2005, 0:48:17 UTC - in response to Message 184953.  
Last modified: 2 Nov 2005, 0:49:26 UTC

double post.
Sorry.

Join BOINC United now!
ID: 184956 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 184958 - Posted: 2 Nov 2005, 0:53:15 UTC - in response to Message 184953.  


This behavior was introduced in boinc 4.45 and above. Previously it was cosidered a bug in the code.

In my point of view the new behavior of uploading a result and reporting it later is a bug and not a feature.

But that's only one of a lot of changes (bad ones) made to the client :-(



I see you found my non-edited post... :)

I don't know if I would consider the 5.x.x behavior a "bug", but it did cause
me to make an unecessary trip home. I was having some stability problems
yesterday that I was monitoring remotely via the reporting mechanism today.
I thought I had locked up again, so I came home to address that (lower the
clock speed of my processor and memory again)...but lo' and behold, it was
running just fine. 3 hours was enough time for me to finish 3 units, and
I was expecting to see it and didn't. Looking at the messages, it didn't
start asking for more work until I updated. Perhaps it was the time correction needing to kick in??? Not real sure...but it is definitely
different. At least now I know... :)

Brian
ID: 184958 · Report as offensive
Ingleside
Volunteer developer

Send message
Joined: 4 Feb 03
Posts: 1546
Credit: 15,832,022
RAC: 13
Norway
Message 185022 - Posted: 2 Nov 2005, 3:46:30 UTC - in response to Message 184953.  

This behavior was introduced in boinc 4.45 and above. Previously it was cosidered a bug in the code (uploading and reporting at once).

In my point of view the new behavior of uploading a result and reporting it later is a bug and not a feature.

But that's only one of a lot of changes (bad ones) made to the client :-(


Uploading results 1st. and reporting them later isn't something new, most clients except apparently v4.4x has this behaviour. Example with v3, back when you still had two cache-settings you'll fill-up to high water-mark, and didn't ask for more work or report any results before you dropped below the low water-mark, meaning it could take days between each time reported any results.

Now when you only have one cache-setting, you'll normally ask for more work during your crunching of next result. For both project and user this slight delay for reporting a result has little impact, for even if you yourself is running with a 0.1-day cache-setting chances are you'll need to wait on someone with a 10-day cache before getting your credit, and project must likewise wait...

For scheduling-server and database on the other hand, not getting 2x the traffic just for reporting a result immediately but waiting a hour or so can mean a lot.

Going by High-Performance Task Distribution for Volunteer Computing, in a single-server-setup an extra connection to scheduling-server for reporting results means capasity just dropped 33%, while on multi-server-setup the database is the bottleneck, and capasity drops with 26%.


Not having a client that unnessesarily decreases capasity 33% is maybe a "bug" for you, but for a DC-project with limited funding it can be the difference between a cheap server and needing to buy a more expensive one, or the difference between one server and two servers, or if haven't the money anyway for more server-capasity can mean 33% less scientific work done...
ID: 185022 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 185034 - Posted: 2 Nov 2005, 4:53:24 UTC - in response to Message 185022.  

For scheduling-server and database on the other hand, not getting 2x the traffic just for reporting a result immediately but waiting a hour or so can mean a lot.


I'd have to take your word for it, but there is some doubt in my mind.
It seems more like a shuffling around of where your bottlenecks would be.
More likely, the intent here is to slow things down for the validators
and not the scheduler...imo.
ID: 185034 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 185222 - Posted: 3 Nov 2005, 0:08:58 UTC - in response to Message 185034.  

For scheduling-server and database on the other hand, not getting 2x the traffic just for reporting a result immediately but waiting a hour or so can mean a lot.


I'd have to take your word for it, but there is some doubt in my mind.
It seems more like a shuffling around of where your bottlenecks would be.
More likely, the intent here is to slow things down for the validators
and not the scheduler...imo.

The intent is to seperate processes so that they may be on different servers. The upload/download servers are basically HTTP "disk drives" that store files.

Uploading a result just stores a file. Uploads and downloads can work while the scheduler (or the scheduler's database) is down for maintenance.
ID: 185222 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 185292 - Posted: 3 Nov 2005, 4:06:15 UTC - in response to Message 185222.  

Uploads and downloads can work while the scheduler (or the scheduler's database) is down for maintenance.


Not according to what it says on the server status page:

"scheduler: Determines what work is going to be sent to/received from
requesting clients. Clients go to the scheduler first to request work, and the
scheduler tells the client what to get and where to get it. If this is off,
you cannot get any new work.
After a client sends a result back, it then contacts the scheduler which then marks it as received.

I "reported" 9 units at once tonight when I got home...because something
hosed up and I got this in the messages log (I hope this formats ok):

11/2/2005 6:15:53 PM|SETI@home|Started upload of 23my04ab.17965.28674.298580.52_1_0
11/2/2005 6:16:53 PM||Couldn't connect to hostname [setiboincdata.ssl.berkeley.edu]
11/2/2005 6:16:53 PM|SETI@home|Temporarily failed upload of 23my04ab.17965.28674.298580.52_1_0: system I/O
11/2/2005 6:16:53 PM|SETI@home|Backing off 1 minutes and 0 seconds on upload of file 23my04ab.17965.28674.298580.52_1_0
11/2/2005 6:17:53 PM|SETI@home|Started upload of 23my04ab.17965.28674.298580.52_1_0
11/2/2005 6:18:15 PM||Couldn't connect to hostname [setiboincdata.ssl.berkeley.edu]
11/2/2005 6:18:15 PM|SETI@home|Temporarily failed upload of 23my04ab.17965.28674.298580.52_1_0: system I/O
11/2/2005 6:18:15 PM|SETI@home|Backing off 1 minutes and 0 seconds on upload of file 23my04ab.17965.28674.298580.52_1_0
11/2/2005 6:19:15 PM|SETI@home|Started upload of 23my04ab.17965.28674.298580.52_1_0
11/2/2005 6:20:29 PM|SETI@home|Finished upload of 23my04ab.17965.28674.298580.52_1_0
11/2/2005 6:20:29 PM|SETI@home|Throughput 396 bytes/sec
11/2/2005 6:24:25 PM|SETI@home|Started upload of 31ja04ab.5055.13666.934640.41_2_0
11/2/2005 6:25:26 PM|SETI@home|Finished upload of 31ja04ab.5055.13666.934640.41_2_0
11/2/2005 6:25:26 PM|SETI@home|Throughput 360 bytes/sec
11/2/2005 7:07:43 PM||request_reschedule_cpus: process exited
11/2/2005 7:07:43 PM|SETI@home|Computation for result 31ja04ab.5055.13666.934640.31_0 finished
11/2/2005 7:07:43 PM|SETI@home|Starting result 31ja04ab.5055.13666.934640.38_1 using setiathome version 418
11/2/2005 7:07:45 PM|SETI@home|Started upload of 31ja04ab.5055.13666.934640.31_0_0
11/2/2005 7:09:07 PM|SETI@home|Finished upload of 31ja04ab.5055.13666.934640.31_0_0
11/2/2005 7:09:07 PM|SETI@home|Throughput 332 bytes/sec
11/2/2005 7:59:36 PM||request_reschedule_cpus: process exited
11/2/2005 7:59:36 PM|SETI@home|Computation for result 31ja04ab.5055.13666.934640.38_1 finished
11/2/2005 7:59:36 PM|SETI@home|Starting result 22my04aa.516.1456.928404.151_0 using setiathome version 418
11/2/2005 7:59:38 PM|SETI@home|Started upload of 31ja04ab.5055.13666.934640.38_1_0
11/2/2005 7:59:43 PM|SETI@home|Finished upload of 31ja04ab.5055.13666.934640.38_1_0
11/2/2005 7:59:43 PM|SETI@home|Throughput 8063 bytes/sec
11/2/2005 8:00:36 PM||request_reschedule_cpus: process exited
11/2/2005 8:00:36 PM|SETI@home|Computation for result 22my04aa.516.1456.928404.151_0 finished
11/2/2005 8:00:36 PM|SETI@home|Starting result 31ja04ab.5055.13666.934640.39_0 using setiathome version 418
11/2/2005 8:00:38 PM|SETI@home|Started upload of 22my04aa.516.1456.928404.151_0_0
11/2/2005 8:00:43 PM|SETI@home|Finished upload of 22my04aa.516.1456.928404.151_0_0
11/2/2005 8:00:43 PM|SETI@home|Throughput 22252 bytes/sec
11/2/2005 8:52:29 PM||request_reschedule_cpus: process exited
11/2/2005 8:52:29 PM|SETI@home|Computation for result 31ja04ab.5055.13666.934640.39_0 finished
11/2/2005 8:52:29 PM|SETI@home|Starting result 31ja04ab.5055.13666.934640.21_2 using setiathome version 418
11/2/2005 8:52:31 PM|SETI@home|Started upload of 31ja04ab.5055.13666.934640.39_0_0
11/2/2005 8:52:34 PM|SETI@home|Finished upload of 31ja04ab.5055.13666.934640.39_0_0
11/2/2005 8:52:34 PM|SETI@home|Throughput 18360 bytes/sec
11/2/2005 9:02:20 PM|SETI@home|Started upload of 31ja04ab.5055.13616.234660.241_1_0
11/2/2005 9:02:26 PM|SETI@home|Finished upload of 31ja04ab.5055.13616.234660.241_1_0
11/2/2005 9:02:26 PM|SETI@home|Throughput 8747 bytes/sec
11/2/2005 9:06:06 PM|SETI@home|Started upload of 31ja04ab.5055.13666.934640.44_0_0
11/2/2005 9:06:09 PM|SETI@home|Finished upload of 31ja04ab.5055.13666.934640.44_0_0
11/2/2005 9:06:09 PM|SETI@home|Throughput 16700 bytes/sec
11/2/2005 9:44:24 PM||request_reschedule_cpus: process exited
11/2/2005 9:44:24 PM|SETI@home|Computation for result 31ja04ab.5055.13666.934640.21_2 finished
11/2/2005 9:44:24 PM|SETI@home|Starting result 31ja04ab.5055.13616.234660.244_2 using setiathome version 418
11/2/2005 9:44:26 PM|SETI@home|Started upload of 31ja04ab.5055.13666.934640.21_2_0
11/2/2005 9:44:29 PM|SETI@home|Finished upload of 31ja04ab.5055.13666.934640.21_2_0
11/2/2005 9:44:29 PM|SETI@home|Throughput 18393 bytes/sec
11/2/2005 10:34:20 PM||request_reschedule_cpus: project op
11/2/2005 10:34:23 PM|SETI@home|Sending scheduler request to http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi
11/2/2005 10:34:23 PM|SETI@home|Reason: Requested by user
11/2/2005 10:34:23 PM|SETI@home|Reporting 9 results
11/2/2005 10:34:28 PM|SETI@home|Scheduler request to http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi succeeded
11/2/2005 10:34:33 PM|SETI@home|Deferring communication with project for 10 minutes and 1 seconds
11/2/2005 10:36:31 PM||request_reschedule_cpus: process exited


Now, while I don't agree this is a "bug" in the new version, I will say
that I think it has some issues. I don't know how long it would've taken
for the app to start "reporting" the results that it is uploading had I not
intervened. As you can see by the logging, there is uploading but nothing
even attempted to be downloaded, perhaps it is because I've got more than 5
days worth? Not really... In 5 days I can chew through about a minimum
of about 130 units. Due to the mixing in of results that take less than a
minute, I'd say 160 is probably not out of the realm of possibility for me
in 5 days time (160/5 = 32/day = 45 minutes (2700 seconds) / WU...
If you look at some of my results, you'll see a large batch of 2500-2600
and then some that are now running 3000-3100...

Anyway, I'm sure the _concept_ is nice, and probably is correct, but the
implementation may need review...IMO. YMMV...

Brian
ID: 185292 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 185310 - Posted: 3 Nov 2005, 5:56:07 UTC - in response to Message 185292.  


Now, while I don't agree this is a "bug" in the new version, I will say
that I think it has some issues. I don't know how long it would've taken
for the app to start "reporting" the results that it is uploading had I not
intervened. As you can see by the logging, there is uploading but nothing
even attempted to be downloaded, perhaps it is because I've got more than 5
days worth? Not really... In 5 days I can chew through about a minimum
of about 130 units. Due to the mixing in of results that take less than a
minute, I'd say 160 is probably not out of the realm of possibility for me
in 5 days time (160/5 = 32/day = 45 minutes (2700 seconds) / WU...
If you look at some of my results, you'll see a large batch of 2500-2600
and then some that are now running 3000-3100...

Anyway, I'm sure the _concept_ is nice, and probably is correct, but the
implementation may need review...IMO. YMMV...

Brian

If you run one project, and you set the "connect every 'x' setting" to 5 days, and BOINC will connect to the scheduler once every 5 days.

When it connects to the scheduler, it will report all work that has been uploaded since it last contacted the scheduler, and request another 5 days worth of work.

If you get a bunch of noisy work units (the ones that go really fast) and BOINC runs out of work, it'll connect early to report results and get more work.

As a general rule, BOINC will download a bunch of files when the scheduler assigns more work -- if you say "connect every five days" then it'll tend to download roughly five days worth of work, upload the work units as they are finished, then report 'em at the five day mark (or when it runs out of work, whichever comes earliest).

There are additional rules that kick in when deadlines get close, but they probably don't apply to your circumstances.
ID: 185310 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 185326 - Posted: 3 Nov 2005, 8:21:42 UTC

Brian,

The scheduler does not "top off" the amount of work in the work buffer. so, you will see a gradual decrease until the BOINC Daemon determines that the level is too low and at that point it will request more work. I forget what the exact rules are as I just let it flow ...
ID: 185326 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 185327 - Posted: 3 Nov 2005, 8:24:58 UTC - in response to Message 185310.  


As a general rule, BOINC will download a bunch of files when the scheduler assigns more work -- if you say "connect every five days" then it'll tend to download roughly five days worth of work, upload the work units as they are finished, then report 'em at the five day mark (or when it runs out of work, whichever comes earliest).

There are additional rules that kick in when deadlines get close, but they probably don't apply to your circumstances.


OK... I'll tinker around with it... It was more of an irritant because
I was expecting the same behavior as before and had planned on using the
reporting of credits as a remote way of seeing if my computer had locked up
again...and it got in the way of my plans... :( The 5-day thing really works
now...as opposed to only giving a 2 or 3-day buffer, so I probably need to
reduce the cache anyway... Watch there be a sudden long-duration outage now!

Brian

ID: 185327 · Report as offensive
Brian Silvers

Send message
Joined: 11 Jun 99
Posts: 1681
Credit: 492,052
RAC: 0
United States
Message 185330 - Posted: 3 Nov 2005, 8:30:07 UTC - in response to Message 185326.  

Brian,

The scheduler does not "top off" the amount of work in the work buffer. so, you will see a gradual decrease until the BOINC Daemon determines that the level is too low and at that point it will request more work. I forget what the exact rules are as I just let it flow ...


This, at least to me, seems to contradict what you wrote:

11/3/2005 3:08:08 AM|SETI@home|Sending scheduler request to http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi
11/3/2005 3:08:08 AM|SETI@home|Reason: To fetch work
11/3/2005 3:08:08 AM|SETI@home|Requesting 1 seconds of new work, and reporting 1 results
11/3/2005 3:08:13 AM|SETI@home|Scheduler request to http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi succeeded
11/3/2005 3:08:14 AM|SETI@home|Started download of 18my04ab.20278.4210.648582.119
11/3/2005 3:08:16 AM|SETI@home|Finished download of 18my04ab.20278.4210.648582.119
11/3/2005 3:08:16 AM|SETI@home|Throughput 224268 bytes/sec
11/3/2005 3:08:17 AM||request_reschedule_cpus: files downloaded


Like I said, I'll tinker with it... I was thinking it would get a little
better once it re-stabilized the correction factor from my blowup the other
day, but who knows...?
ID: 185330 · Report as offensive
Profile MikeSW17
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 1603
Credit: 2,700,523
RAC: 0
United Kingdom
Message 185332 - Posted: 3 Nov 2005, 8:46:15 UTC - in response to Message 185326.  

Brian,

The scheduler does not "top off" the amount of work in the work buffer. so, you will see a gradual decrease until the BOINC Daemon determines that the level is too low and at that point it will request more work. I forget what the exact rules are as I just let it flow ...


I don't see this behavior - to any significant extent. My queues is constantly being topped-off with between 1 and 5 WUs. (BOINC 5.2.4)

I hope this _is_ the design behaviour, because its just what I want.
Also, I'd be pretty miffed if the system ran my queue down to near empty just at the time the project decided to have a days outage.
The whole point of my havimg a large buffer is to automatically span long project outages.

ID: 185332 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Uploading result vs. returning result


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.