Topping off the cache - Idea

Message boards : Number crunching : Topping off the cache - Idea
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile JigPu
Avatar

Send message
Joined: 16 Feb 00
Posts: 99
Credit: 2,513,738
RAC: 0
Message 117840 - Posted: 3 Jun 2005, 1:05:58 UTC
Last modified: 3 Jun 2005, 1:06:43 UTC

I'm not a 56K user, but I've heard quite a few complain about the current situation where one cannot top off their cache while online. Obviously this can be a problem at times, since you may disconnect before BOINC decides to get more work. Therefore, I suggest the addition of a "connect now" (or simiar) button in the BOINC client. I see no reason that such a feature does NOT exist in any caching system, and I don't see any problems implementing it with the current scheduling system either.

I have an idea on how it would be implemented (so the devs can have an idea of how to put it into BOINC), though obviously others should comment on it (wether it's needed, how the implementation could be tweaked, etc) before such a feature is haphazardly thrown into the system.

Quoted from my post in the LHC forums
~~~~~~~~~~~~~~~~~~
"From what I've read about the latest scheduler, there are two scheduling modes and three work-fetch modes. Of the two schedule modes, panic mode can be set off by either a) short deadlines or b) tight deadlines.
   If the scheduler is panicking due due a short deadline, the documentation itself says "Having the CPU scheduler in panic mode for a short deadline will not preclude the downloading of work. If the work unit is due today, but the work otherwise is not in time trouble, there is no reason not to download some more work." -- Force work-fetch to "download required" mode on clicking the button.
   If the scheduler is panicking due due a tight deadline, new work may obviously miss the deadline. -- Warn user that forcing a work-request may result in work missing it's deadline. If the user is sure this is OK, force work-fetch into "download required" mode.
   If the scheduler is not panicing at all, there's no harm in downloading more work. Instead of waiting a few more days, just connect right now for more work. -- Force work-fetch into "download required" mode.

Again according to the documentation, "The work fetch will always be done in order of highest long term debt." Forcing a work-fetch then would still honor resource share because it would initially attempt to download new work from a project which needs to be run more. A line between resource share and downloading from everybody does not have to be walked with the implementation of a cache refill button, since it would use the the very same debt calculations any other work-fetch would."
~~~~~~~~~~~~~~~~~~

Puffy
ID: 117840 · Report as offensive
Heffed
Volunteer tester

Send message
Joined: 19 Mar 02
Posts: 1856
Credit: 40,736
RAC: 0
United States
Message 117912 - Posted: 3 Jun 2005, 3:09:13 UTC

Sounds good to me. :D
ID: 117912 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 117929 - Posted: 3 Jun 2005, 3:39:34 UTC - in response to Message 117840.  

I'm not a 56K user, but I've heard quite a few complain about the current situation where one cannot top off their cache while online. Obviously this can be a problem at times, since you may disconnect before BOINC decides to get more work. Therefore, I suggest the addition of a "connect now" (or simiar) button in the BOINC client. I see no reason that such a feature does NOT exist in any caching system, and I don't see any problems implementing it with the current scheduling system either.

I have an idea on how it would be implemented (so the devs can have an idea of how to put it into BOINC), though obviously others should comment on it (wether it's needed, how the implementation could be tweaked, etc) before such a feature is haphazardly thrown into the system.

Quoted from my post in the LHC forums
~~~~~~~~~~~~~~~~~~
"From what I've read about the latest scheduler, there are two scheduling modes and three work-fetch modes. Of the two schedule modes, panic mode can be set off by either a) short deadlines or b) tight deadlines.
   If the scheduler is panicking due due a short deadline, the documentation itself says "Having the CPU scheduler in panic mode for a short deadline will not preclude the downloading of work. If the work unit is due today, but the work otherwise is not in time trouble, there is no reason not to download some more work." -- Force work-fetch to "download required" mode on clicking the button.
   If the scheduler is panicking due due a tight deadline, new work may obviously miss the deadline. -- Warn user that forcing a work-request may result in work missing it's deadline. If the user is sure this is OK, force work-fetch into "download required" mode.
   If the scheduler is not panicing at all, there's no harm in downloading more work. Instead of waiting a few more days, just connect right now for more work. -- Force work-fetch into "download required" mode.

Again according to the documentation, "The work fetch will always be done in order of highest long term debt." Forcing a work-fetch then would still honor resource share because it would initially attempt to download new work from a project which needs to be run more. A line between resource share and downloading from everybody does not have to be walked with the implementation of a cache refill button, since it would use the the very same debt calculations any other work-fetch would."
~~~~~~~~~~~~~~~~~~

Puffy


The problem is that if work could be downloaded from high LT debt projects, it would have already been downloaded from high LT debt projects. The download scheduler attempts to keep enough work in the cache to ride out being disconnected for the size of the queue. For the cases where it is not normally a good idea to reach into thenegative LT debt projects for work (as this will break resource share) for always on connections, it may be an excellent idea for modem users so that they have something to crunch on even if it breaks the resource shares further.


BOINC WIKI
ID: 117929 · Report as offensive
Profile JigPu
Avatar

Send message
Joined: 16 Feb 00
Posts: 99
Credit: 2,513,738
RAC: 0
Message 117988 - Posted: 3 Jun 2005, 7:05:59 UTC - in response to Message 117929.  
Last modified: 3 Jun 2005, 7:07:21 UTC

The download scheduler attempts to keep enough work in the cache to ride out being disconnected for the size of the queue.

I'm not 100% sure how this works (I haven't found a good explination, and the source code is too hard to navigate for a C++ newbie of my caliber), though it seems as though at least one or two modem users are managing to not have enough data in their caches.

From what I've read, it sounds as if the problem is BOINC downloading X days of work (where X is the "connect to network about every..." setting), and then downloading new work once the cache is depleted. A modem user may very well be offline when the cache depletes though, leaving them idle until they can reconnect. If they have only 1 WU remaining in the cache, they would either need to remain connected until the WU finishes (so the cache can refill), or disconnect and allow the idle time.

This could be fixed obviously through the use of a button as I suggested, though yet more tweaks to BOINC itself could also do it. One possible solution would be to keep track of how often the user has a net connection (would this be the already existing "% of time host is connected"?), and then kick in the download scheduler once CONNECT_TO_NET_EVERY_X_DAYS * NET_CONNECTED_% has passed (and keep attempting to connect every 30 minutes or so). For example, if a host is set to connect to the net every 3 days, and is connected 50% of the time, begin attempting to download new work (to fill the whole 3 day cache once more) once 1.5 days of work has been processed. This may cause connections quite a bit earlier than set in the preferences, and will likely throw in a lot more errors into the Messages while it can't find the projects due to disconnection. However, it should allow connection attempts early enough that the host will likely reconnect before depleting the cache, which is a must for them modem users.

Just throwing out ideas :)
Puffy
ID: 117988 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 118116 - Posted: 3 Jun 2005, 15:59:51 UTC - in response to Message 117988.  

The download scheduler attempts to keep enough work in the cache to ride out being disconnected for the size of the queue.

I'm not 100% sure how this works (I haven't found a good explination, and the source code is too hard to navigate for a C++ newbie of my caliber), though it seems as though at least one or two modem users are managing to not have enough data in their caches.

From what I've read, it sounds as if the problem is BOINC downloading X days of work (where X is the "connect to network about every..." setting), and then downloading new work once the cache is depleted. A modem user may very well be offline when the cache depletes though, leaving them idle until they can reconnect. If they have only 1 WU remaining in the cache, they would either need to remain connected until the WU finishes (so the cache can refill), or disconnect and allow the idle time.

This could be fixed obviously through the use of a button as I suggested, though yet more tweaks to BOINC itself could also do it. One possible solution would be to keep track of how often the user has a net connection (would this be the already existing "% of time host is connected"?), and then kick in the download scheduler once CONNECT_TO_NET_EVERY_X_DAYS * NET_CONNECTED_% has passed (and keep attempting to connect every 30 minutes or so). For example, if a host is set to connect to the net every 3 days, and is connected 50% of the time, begin attempting to download new work (to fill the whole 3 day cache once more) once 1.5 days of work has been processed. This may cause connections quite a bit earlier than set in the preferences, and will likely throw in a lot more errors into the Messages while it can't find the projects due to disconnection. However, it should allow connection attempts early enough that the host will likely reconnect before depleting the cache, which is a must for them modem users.

Just throwing out ideas :)
Puffy

If there is less work available in the queue than the queue size, BOINC will be looking for work from any project with positive LT debt to fill the hole. If the project already has what it considers enough work, BOINC will ask for work one WU at a time until the cache is filled to the minimum queue size (this is a total for all projects, not for each project individually). If after filling the queue to the minimum there are not any scheduler problems and there are still projects that are OK to download work from at the moment, they will be contacted for work. Once the queue is filled to the minimum, work fetch will stop if there is a detected CPU scheduler problem.

The key point that seems to be missed is that the queue is a total for all of the projects.


BOINC WIKI
ID: 118116 · Report as offensive

Message boards : Number crunching : Topping off the cache - Idea


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.