I have three suggestions that I feel would greatly enhance the user experience of BOINC.

First, I'm with Boinc Beta, SAH, and Predictor. With Boinc Beta down at the moment, and SaH refusing to hand out work units, almost every request to the server from any project results in Predictor requesting more work units, which represents a significant drain on bandwidth. I propose that there should be an advanced option set within BOINC that allows more complete control over projects and workunits than is currently allowed, such as: specifying precisely how many workunits to cache at any given time and to allow manual control over the low-water mark; individualized project network suspension, for when you want workunits from SaH, for example, to download before workunits from Predictor; an option to force suspension of one project in favour of a primary choice; maybe even a server status option, for when you want to know the health of the server, but don't necessarily want to request more work, etc.

Second, I propose that archive decompressors be built into project executables, so work units can be compressed on the server side, sent, and then decompressed. With the .zip format, a 25% compression ratio can be achieved with the Seti workunits, and I believe an even greater ratio would be achieved with Predictor units, since they are mostly straightforward text. This would be incredibly useful for those of us on dialup, not only because of the decreased file sizes, but would help with projects like Predictor (work units are actually about 8 files, and the handshake period between files sometimes takes longer than the actual file download).

Third, and maybe most important of all, is an issue that has been raised before, and goes hand in hand with some of the other suggestions I've made above. It would be very handy to suspend computations on a particular project, not only for the reasons that have been given before me, but also because some people like me are on dialup, and when a project like Predictor goes slightly nutty (which I'll cover in a moment) and starts downloading slews of half meg units, it takes over my bandwidth for up to a half hour on end, sometimes two or three times a day. Being able to suspend a project in this case means that focus can be turned toward getting more work for SaH downloaded, so that while it is running, those Predictor units can continue to download, if that is the choice of the user.

Basically, all of this boils down to allowing the End User to have a much greater control over what is going on on his or her computer. As it stands, BOINC tries to force the issue on some things, and while that might be fine for a casual user that just wants to let the project run in the background, BOINC and Seti@Home has a tendancy to attract the power users out there, and these are the people who want to be in charge of what goes on in their presence.

Just a couple comments to add to a few of the things you mention.

As far as your first item, remember that your low-water mark is system wide and is determined by you, not seti or predictor. If you need work, you need it from somewhere. If seti is next in line and it's down, then it just moves to whatever comes after that (predictor, in your case). Boinc won't ask seti for work if it doesn't NEED work from somewhere, per the instructions you have given it in your preferences (minimum amount of work to keep on disk) - therefore just because seti is down doesn't mean it really doesn't need work, therefore it moves on to predictor to get that work that your settings are telling it that it needs from somewhere. You can control the number of work units it gets globally by reducing (or increasing) the size of your cache.

And while I agree that there should be a way to "suspend" participation in a project on a temporary basis (something short of detaching), I do not agree that there should be a way to force suspension once a work unit is downloaded in favor of a preferred project. If you're going to throw out predictor units as soon as seti units become available (by repeatedly forcing them to the end where they will just pass their deadline date), then you just shouldn't participate in the predictor project in the first place. As far as boinc is concerned, every project is (and should be) equally important. Whatever got into your queue first gets processed first. If you don't agree with boinc that they're equally important, then again, you shouldn't have put something in your queue in the first place that you don't consider worthy of finishing.

Predictor is not designed to be - and should not be used as - simply a fall-back from seti when there's nothing better to do. Such an approach would only hurt their science, and it would be better for everyone involved if people who ONLY want seti simply let their system sit idle when seti is not available.

And lastly, remember that boinc is not seti. Boinc is itself. Seti runs within boinc, just as other dc projects do now and will in the future. Boinc itself absolutely should not be designed so that the "focus can be turned toward getting more work for SaH downloaded". Boinc is neutral, and should stay that way. You are the one who tells boinc when to get work and where the options are to get that work from when that time arrives. If it's downloading "too much" work from predictor, then you are the one who has it set to download so much work, likely in an attempt to grab up a bunch of seti workunits when they come available. Turn the numbers down to something reasonable and it will not overload you with work from predictor. If you set your max to 0.5 days, it will not download more than about 2 workunits at a time - from either predictor or seti.

> And while I agree that there should be a way to "suspend" participation in a
> project on a temporary basis (something short of detaching), I do not agree
> that there should be a way to force suspension once a work unit is downloaded
> in favor of a preferred project. If you're going to throw out predictor units
> as soon as seti units become available (by repeatedly forcing them to the end
> where they will just pass their deadline date), then you just shouldn't
> participate in the predictor project in the first place. As far as boinc is
> concerned, every project is (and should be) equally important. Whatever got
> into your queue first gets processed first. If you don't agree with boinc
> that they're equally important, then again, you shouldn't have put something
> in your queue in the first place that you don't consider worthy of finishing.

Well said.

I did ask for a tempary disable project function in another thread.

"Predictor is not designed to be - and should not be used as - simply a fall-back from seti when there's nothing better to do. Such an approach would only hurt their science, and it would be better for everyone involved if people who ONLY want seti simply let their system sit idle when seti is not available."

I dont think that was Benjanmin's intention. I personaly would pause the boinc beta if i could as it sometimes causes boinc to download loads of excess work units. I have my work unit prefs set to 0.2 to 0.4 days what should be about 2-4 workunts. And at times when i get home from work i could have somewhere in the region of 10-12 sitting on my machine. Thats nearly 3 days of work. So i have to keep network access disabled just to make sure that it doesnt go on a download run.

Also the pause option i dont think should be a instant affect. For example: I currently have 4 predictor work units in cache. If i paused that project it would wait till the cache is empty and have reported the work units before disabling communication/downloading of work from that project.

Jamie

> Also the pause option i dont think should be a instant affect. For example: I
> currently have 4 predictor work units in cache. If i paused that project it
> would wait till the cache is empty and have reported the work units before
> disabling communication/downloading of work from that project.
>

one thing I would like to see is a separate option for suspend network activity so it will keep working but not attempt u/l or d/l until re-specified.

> As far as your first item, remember that your low-water mark is system wide
> and is determined by you, not seti or predictor. If you need work, you need
> it from somewhere. If seti is next in line and it's down, then it just moves
> to whatever comes after that (predictor, in your case). Boinc won't ask seti
> for work if it doesn't NEED work from somewhere, per the instructions you have
> given it in your preferences (minimum amount of work to keep on disk) -
> therefore just because seti is down doesn't mean it really doesn't need work,
> therefore it moves on to predictor to get that work that your settings are
> telling it that it needs from somewhere. You can control the number of work
> units it gets globally by reducing (or increasing) the size of your cache.

Unfortunately, whether by design or by bug, this is NOT true. I have Predictor set at 0.1 to 0.2 days, yet I still find it is downloading 20+ work units at a time, piled up due to aforementioned downloading sprees spread across half a dozen or more low-water mark gets. And it is not because Seti is giving no work units. It *is* sending units now, yet Predictor is behaving in the exact same way as before. I am beginning to suspect that the Predictor system itself might have a fault in it, and not the Boinc client, but we'll see. No offense, but please don't lecture me about the system. I've been around since the beginning and have a decent idea about how it all fits together. The system just isn't working exactly as it should, at least, not for me at the moment.

> And while I agree that there should be a way to "suspend" participation in a
> project on a temporary basis (something short of detaching), I do not agree
> that there should be a way to force suspension once a work unit is downloaded
> in favor of a preferred project. If you're going to throw out predictor units
> as soon as seti units become available (by repeatedly forcing them to the end
> where they will just pass their deadline date), then you just shouldn't
> participate in the predictor project in the first place. As far as boinc is
> concerned, every project is (and should be) equally important. Whatever got
> into your queue first gets processed first. If you don't agree with boinc
> that they're equally important, then again, you shouldn't have put something
> in your queue in the first place that you don't consider worthy of finishing.

And no, I am not going to throw out Predictor units at all, I simply would like to specify that Seti units get processed before Predictor units, as my current project ratio already stipulates. The problem is, Predictor completely ignores my stated ratio of Seti 80% and Predictor 20%, and will continue dominating my system until I am forced to knock it completely off my computer. It went through a large number of units, with several Seti units waiting yet never being given a chance for work. I'm sorry, but I've always been partial to Seti, and always will be. I have a long history with the program, and will continue to give it preferrential treatment, as is my right as computer owner and *donator* of processing cycles. I'm more than happy to do Predictor units, but if I am to participate, it bloody well better follow the rules I set up, and thus far, it isn't doing so. My suggestions would allow a user to force the hand of a buggy or belligerent project.

> Unfortunately, whether by design or by bug, this is NOT true. I have
> Predictor set at 0.1 to 0.2 days, yet I still find it is downloading 20+ work
> units at a time, piled up due to aforementioned downloading sprees spread
> across half a dozen or more low-water mark gets.

Have you tried to set seti to keep more than 0.1 to 0.2 days? If you have, this is a global setting that applies to every project and it will have reset predictor also. When you change a general preference file on any project, boinc propagates that new setting to every project you participate in. The low-water mark is part of these general settings, so changing the low-water mark (or any part of the cache size) in seti, by default, also changes it in predictor. One of your log entries for each rpc will show you where your general preferences came from (i.e. which one was changed most recently).


> I am beginning to suspect that the Predictor
> system itself might have a fault in it, and not the Boinc client, but we'll
> see.

When it makes a request, note how many seconds of work it is asking for and see if this is realistic to the difference between the minimum and maximum amount of data you want to keep on the disk. If the amount of data being asked for is totally off base then your cache settings are not being applied somewhere along the line. If it's asking for the correct number of seconds of work but getting a tremendous amount more than what it asked for, then your benchmarks are likely indicating that your system can process the work much faster than it really can.


> And no, I am not going to throw out Predictor units at all, I simply would
> like to specify that Seti units get processed before Predictor units, as my
> current project ratio already stipulates.

But my original point was that you can't stipulate when the work units get processed by the setting you give for your ratio. Your ratio only determines which project boinc will attempt to contact for filling the cache with the most frequency. If it's 80/20 in favor of seti, boinc will give seti first shot to refill your cache 80% of the time. If seti fails for any reason, it automatically gives the chance to the other project. However, once the work units go into your cache, they process in the order they went in there.

If you could specify for it to process one type of work unit out of your cache before another type, then it would, whether it's your intent or not, simply defer the "less desirable" work unit all the way until it expired (assuming your preferred project continued to provide work) - thus, "throwing it out".

> The problem is, Predictor
> completely ignores my stated ratio of Seti 80% and Predictor 20%, and will
> continue dominating my system until I am forced to knock it completely off my
> computer. It went through a large number of units, with several Seti units
> waiting yet never being given a chance for work.

Again, once their turn comes up based upon the order they went into the queue in the first place, they will be processed. If 50 predictor units were queued and then 5 seti units got added to the queue, those seti units will be nos. 51 through 55 in line for processing. Your 80/20 setting does not in any way call for 80% of the process time on your computer to go to seti - it only calls for 80% of the chances to fill your queue to go to seti.

> I'm sorry, but I've always
> been partial to Seti, and always will be. I have a long history with the
> program, and will continue to give it preferrential treatment, as is my right
> as computer owner and *donator* of processing cycles. I'm more than happy to
> do Predictor units, but if I am to participate, it bloody well better follow
> the rules I set up, and thus far, it isn't doing so. My suggestions would
> allow a user to force the hand of a buggy or belligerent project.

Nothing wrong with being partial to seti - I am too. It wasn't my intent to imply that *you* would mistreat another program, but simply to point out that some of what you suggested could allow users to mistreat other programs. Since boinc is a multi-project platform, it can't allow preferential treatment of seti or it would have no other projects willing to participate using the boinc platform.

If your desire is to fully support seti and only fall back to predictor when seti is not available, that can be done, but it involves using settings a lot of seti people won't like. Seti people, more so than any other project I've participated in, seem to want to horde work units. If you are willing to get your work units in a give and take manner, try something similar to this:

Set your minimum work to 0.1 days and your maximum work to 0.2 days. This will force work units to be downloaded one at a time with each new work unit being downloaded a couple hours before the current one finishes. Then set seti for 1000 and predictor for 0.1. With that, only 1 in 10000 contacts will default to predictor for first shot. All other attempts will go to seti. When seti is up, you will get one work unit, but when it's not it will fall back to predictor. However, it will also only get one predictor unit when it does fall back to that project, because your cache settings will only allow for one.

This is the easiest way to ensure the most goes to seti AND your computer is always doing something for someone. But, if you set your cache to allow for more seti units at one time, remember that it automatically propagates and thus means you will also get more predictor units when seti doesn't respond.

Remember:

Each project SEEMS to have a "general" preferences and a "project name" preferences.

There is ONLY ONE "general" preferences.

If you set general on preditor's page to .1 and .2, and later set the general on seti's page to 2 days to 5 days ---- 2 days to 5 days is the general preference for ALL projects.

> Have you tried to set seti to keep more than 0.1 to 0.2 days? If you have,
> this is a global setting that applies to every project and it will have reset
> predictor also.

Negative. I have general settings set at 0.1 to 0.3 days. Again, I know well how this setting works, and while seti is obeying properly (usually only downloading two workunits at a time, each time), predictor is going overboard in the extreme. It will usually download 4 or 5 units per Get attempt, but each time Seti or Boinc Beta requests workunits, Predictor seemingly overrides and will declare a low-water mark and get four or five *more* units, usually maxing out at 25 or 30 units. I say 'override' because Seti or Boinc beta rarely even returns an RPC succeeded or failed notice, simply giving way to Predictor's attempt to get more units.

> When it makes a request, note how many seconds of work it is asking for and
> see if this is realistic to the difference between the minimum and maximum
> amount of data you want to keep on the disk. If the amount of data being
> asked for is totally off base then your cache settings are not being applied
> somewhere along the line. If it's asking for the correct number of seconds of
> work but getting a tremendous amount more than what it asked for, then your
> benchmarks are likely indicating that your system can process the work much
> faster than it really can.

Usually Predictor will ask for...about 150,000 to 175,000 seconds worth of work, sometimes higher, but rarely lower. This is definitely off base compared to my settings. Beyond even that, I typically run the benchmarks with several programs running as well just to avoid over-estimating the system ability, so I doubt this is the cause of the problem.

> But my original point was that you can't stipulate when the work units get
> processed by the setting you give for your ratio. Your ratio only determines
> which project boinc will attempt to contact for filling the cache with the
> most frequency. If it's 80/20 in favor of seti, boinc will give seti first
> shot to refill your cache 80% of the time. If seti fails for any reason, it
> automatically gives the chance to the other project. However, once the work
> units go into your cache, they process in the order they went in there.

This is apparently the thing I was confusing. I had the idea that the ratio was a setting of how much actual processor time was dedicated to running the different projects...that it would run seti units 80% of the time, and Predictor units 20%. Thank you for clarifying this point. I must admit, I believe this is a bit of a flaw in the system, though the programmers obviously feel this is the best approach, so I'll deal with it :)



> Nothing wrong with being partial to seti - I am too. It wasn't my intent to
> imply that *you* would mistreat another program, but simply to point out that
> some of what you suggested could allow users to mistreat other programs.
> Since boinc is a multi-project platform, it can't allow preferential treatment
> of seti or it would have no other projects willing to participate using the
> boinc platform.

Hmm, I understand what you are saying here, and it makes me think that a server side routine could be built that would look at how work units are returned. If they seem to be dealt with client-side normally, no action would need to be taken. However, if units were routinely being sent in well after time, or never returned at all (with the exception of a user doing a project reset), the scheduler could begin throttling units sent to a particular user, or denying altogether requests for units, for say a week, then a month. This is a bit of a down and dirty way to deal with the problem, but it might be one possible solution to keeping users more honest... thoughts?

> If your desire is to fully support seti and only fall back to predictor when
> seti is not available, that can be done, but it involves using settings a lot
> of seti people won't like. Seti people, more so than any other project I've
> participated in, seem to want to horde work units. If you are willing to get
> your work units in a give and take manner, try something similar to this:

You are absolutely correct about the hording thing. That is why caching programs like SetiMonitor, etc, were so popular for SaH Classic. Also, this is an exceedingly good idea, one I'd never have thought of (mostly since my mind favours nice round numbers and percentages, like 80 and 20, hehe). I'll give it a try.

"However, once the work units go into your cache, they process in the order they went in there."
_______________

No they don't. They process according to the WU deadline.

<a> [/url]

> "However, once the work units go into your cache, they process in the order
> they went in there."
> _______________
>
> No they don't. They process according to the WU deadline.

Heffed, all of his work units are coming from either seti or predictor. They all have a 2 week deadline, so they will all expire in the exact same order they were acquired - therefore the first in will be the first out.

He's not participating in any project with an expiration of anything other than 2 weeks.




<a> [/url]

> each time Seti or Boinc Beta requests workunits, Predictor seemingly overrides
> and will declare a low-water mark and get four or five *more* units, usually
> maxing out at 25 or 30 units. I say 'override' because Seti or Boinc beta
> rarely even returns an RPC succeeded or failed notice, simply giving way to
> Predictor's attempt to get more units.
> ...
> Usually Predictor will ask for...about 150,000 to 175,000 seconds worth of
> work, sometimes higher, but rarely lower. This is definitely off base
> compared to my settings. Beyond even that, I typically run the benchmarks
> with several programs running as well just to avoid over-estimating the system
> ability, so I doubt this is the cause of the problem.

Yes, there is definately a problem there. It's asking for about 3.5 to 4 days worth of work. But remember that boinc makes this determination, not predictor. Boinc has full control over your cache, and when it decides it's low it starts shopping around for work units. With your minimum and maximum settings it shouldn't be asking for so much. The project only supplies the amount of work that boinc asks for.

The only thing that should cause a project that would normally be up to fill the cache to be skipped without even an attempt (or waiting for a reply) would be if that project is already under deferral. If you get one of those really long deferral times from seti, get a work unit from elsewhere, then hit the low-water mark again while your still under the original deferral from seti - it won't ask seti again until you have both a low-water mark and you're out of the deferral time. That, though, shouldn't happen very often so it's not likely anyone would see that with enough frequency to really even notice it happening.


> Hmm, I understand what you are saying here, and it makes me think that a
> server side routine could be built that would look at how work units are
> returned. If they seem to be dealt with client-side normally, no action would
> need to be taken. However, if units were routinely being sent in well after
> time, or never returned at all (with the exception of a user doing a project
> reset), the scheduler could begin throttling units sent to a particular user,
> or denying altogether requests for units, for say a week, then a month. This
> is a bit of a down and dirty way to deal with the problem, but it might be one
> possible solution to keeping users more honest... thoughts?

This was discussed in beta a bit. Actually, what was discussed was a little more blunt, and was just monitoring for abandoned work units and throwing out people who abandon them. I was against this then and still am now, as there are sometimes valid reasons a work unit gets abandoned. I would say there would need to be very definate trends of tossing work units before such an action should be taken, otherwise it's inevitable that people who legitimately did nothing wrong would get "punished". Now, that said I still fully believe you should have the option of suspending the project from sending you additional work on a short-term basis without having to detach from it.

With a little more work, many options could have been included that would give the user more control from the very beginning, without giving them an easy way to mistreat a project they have already gotten work from. Many people watch their systems quite closely, so even an optional (on or off as requested by the user) system message to the effect that boinc will begin downloading x amount of work from project x in x number of minutes (depending upon how much notice you want), and give the user the option to abort that download altogether. Of course, no one watches it 24 hours a day, but you could still get more control that way. It could allow the user to define the default action for no response to the message also - if I don't tell you yes or no within the specified time, then automatically make the download, or automatically refuse the download.

It could also have incorporated a dual cache level system, allowing you to specify a normal low-water mark and a critical low-water mark. For example, if you specify seti as your primary project it could try to fill the cache when it drops to your normal low-water mark. But if seti doesn't fill it AND it drops to the critical level, then try to fill the cache from an alternate project. People who want only seti when possible, but still want to do something if seti is down could have a large seti cache, with the critical cache set very low so it would only kick-in if seti was almost totally out of work and it was about to shut down. That would give what most people want - a lot of seti on their system at any given moment, but only 1 or 2 work units from somewhere else if seti fails. And that way it's not built into boinc to automatically prefer seti - the user has to make these settings, and the user of any project could set their project to always try to fill from its cache and fall back on seti if their project cache goes below the critical level.


<a> [/url]

> > "However, once the work units go into your cache, they process in the
> order
> > they went in there."
> > _______________
> >
> > No they don't. They process according to the WU deadline.
>
> Heffed, all of his work units are coming from either seti or predictor. They
> all have a 2 week deadline, so they will all expire in the exact same order
> they were acquired - therefore the first in will be the first out.

Nope, predictor has kicked out some chramm WUs again and they have a deadline of 3 month and one week...
I have 3 Chramm WUs in my cache right now...

News-update from Predictor homepage:

7/04/04
In a while we will submit some more Charmm workunits on the grid again.

> > But my original point was that you can't stipulate when the work units
> get
> > processed by the setting you give for your ratio. Your ratio only
> determines
> > which project boinc will attempt to contact for filling the cache with
> the
> > most frequency. If it's 80/20 in favor of seti, boinc will give seti
> first
> > shot to refill your cache 80% of the time. If seti fails for any reason,
> it
> > automatically gives the chance to the other project. However, once the
> work
> > units go into your cache, they process in the order they went in there.
>
>
> This is apparently the thing I was confusing. I had the idea that the ratio
> was a setting of how much actual processor time was dedicated to running the
> different projects...that it would run seti units 80% of the time, and
> Predictor units 20%. Thank you for clarifying this point. I must admit, I
> believe this is a bit of a flaw in the system, though the programmers
> obviously feel this is the best approach, so I'll deal with it :)
>
This is not quite right. BOINC keeps track of how much CPU time has been spent on a project, and calculates a resource usage debt. It then attempts to retrieve work from the project with the greatest resource usage debt. If it cannot get work from that project, it starts a wait for that project and immediately requests work from the process with the next highest resource debt. If no project has work, then each project will be querried at the end of the wait, and if there is not work, then it will wait again.

It is expected that after joining a new project, that project will dominate the work done until its resource debt is reduced. The CPU time that BOINC keeps is an exponentially reducing average.