Project sharing idea

Message boards : Number crunching : Project sharing idea
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Papa Zito
Avatar

Send message
Joined: 7 Feb 03
Posts: 257
Credit: 624,881
RAC: 0
United States
Message 29655 - Posted: 24 Sep 2004, 15:07:58 UTC

I would like to see an alternative to the percentage sharing concept that BOINC currently uses. I'm not advocating getting rid of the current system, but perhaps adding an alternative.

The idea is to have a single "primary" project and then multiple "secondary" or even "tertiary" projects. You would attach to these projects in the same way you attach to projects now, but how they are handled would be different.

1. Your computer(s) download x number of WUs from the primary project.
2. Your computer(s) happily crunch on these WUs.
3. Your computer(s) reconnect to the primary project for reporting.

Now, here's where things are really different.

4. Primary project scheduler isn't responding/is down/is short of WUs/whatever. Your computer(s) then connect to a SECONDARY project and download some WUs. However, it only downloads a limited amount (say, for the sake of argument, Primary WUs/2)

5. Your computer(s) happily crunch on the secondary project's WUs.
6. Your computer(s) upload the secondary WUs. Then it attempts to contact the PRIMARY project again. If it succeeds, then primary WUs are downloaded, otherwise secondary WUs are downloaded.

This can be extended, of course... if your first secondary project is down, it might try another secondary project. If that one is down, it could try another one, and so on. It could also handle "short work" situations, in which the primary project has some work but not enough to fill the requested cache. The computer(s) could then contact a secondary project and download enough secondary WUs to fill the cache.

The reason I'd like to see this is because I'm primarly a "fan" of SETI and I'd like to support SETI as much as possible. However, I also don't want my computers to be idle when they could be doing someone some good.




------------------------------------


The game High/Low is played by tossing two nuclear warheads into the air. The one whose bomb explodes higher wins. This game is usually played by people of low intelligence, hence the name High/Low.
ID: 29655 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 29666 - Posted: 24 Sep 2004, 16:04:38 UTC - in response to Message 29655.  

> The reason I'd like to see this is because I'm primarly a "fan" of SETI and
> I'd like to support SETI as much as possible. However, I also don't want my
> computers to be idle when they could be doing someone some good.

The current system accomidates that with the share. Say you connect to two projects, a primary and a secondary. With one set to 90, and the other to 10 you would be allocating 90% of the time to one project vs the other.

With a low connect number, say one day, though you would not have much of a back-log, you would be doing work primarily for one project over the other.

We are still in an artificial environment in that we do not have free and unrestricted access to projects so the pool is a little smaller than any of us would like. Some time ago I did SETI@Home and Predictor@Home and I felt that I was doing a little too much Predictor. But that was mostly a side effect in that time because of the problems that SETI@Home was wrestling with.

Now, we do still have two open projects with work available to both with SETI@Home and cp.net so we can still do multi-project. When Predictor gets the new servers in and running I would hope that they would open back up again. LHC@Home has said that they want to open up also. WIth this, we would have 4 projects with a nice mix of "fast" work units and some with long running. One of the reasons I have been trying to get people onto cp.net, you can't hardly run out of work with the models taking a month or so to complete.

Again, we are still in early days and I, for one, am very pleased with the current progress. The system works. Not as well as I would like, and not without problems. But it works. On my 5 windows computers I have 3 projects turning and burning (naval aviation, sorry) and the Macintosh has the two. But with two cp.net models, whew! no worries.

<p>
For BOINC Documentaion: Click Me!


ID: 29666 · Report as offensive
Profile Papa Zito
Avatar

Send message
Joined: 7 Feb 03
Posts: 257
Credit: 624,881
RAC: 0
United States
Message 29727 - Posted: 24 Sep 2004, 18:30:20 UTC - in response to Message 29666.  

> > The reason I'd like to see this is because I'm primarly a "fan" of SETI
> and
> > I'd like to support SETI as much as possible. However, I also don't want
> my
> > computers to be idle when they could be doing someone some good.
>
> The current system accomidates that with the share. Say you connect to two
> projects, a primary and a secondary. With one set to 90, and the other to 10
> you would be allocating 90% of the time to one project vs the other.
>
> With a low connect number, say one day, though you would not have much of a
> back-log, you would be doing work primarily for one project over the other.
>

Not exactly. With the current system, I'm constantly downloading and crunching WUs from the 2nd project. With the system I proposed, I'd only be downloading and crunching WUs from the 2nd project if the 1st project was unavailable for some reason. This shifts the balance more towards the preferred project.




------------------------------------


The game High/Low is played by tossing two nuclear warheads into the air. The one whose bomb explodes higher wins. This game is usually played by people of low intelligence, hence the name High/Low.
ID: 29727 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 29783 - Posted: 24 Sep 2004, 21:12:36 UTC - in response to Message 29727.  

> Not exactly. With the current system, I'm constantly downloading and
> crunching WUs from the 2nd project. With the system I proposed, I'd only be
> downloading and crunching WUs from the 2nd project if the 1st project was
> unavailable for some reason. This shifts the balance more towards the
> preferred project.

Well, these are not exactly perfect times. RIght now, SETI@Home is having teething problems. So, yes, you will be doing more of something else. When we do get to a more stable platform and things are working as planned, the system will, over time, self-correct.

We are seeing the "Perfect Storm" kind of situation. And because SETI (I am assuming SETI is your preference) is having back-end problems that reflects forward to making it seem un-good ...
<p>
For BOINC Documentaion: Click Me!


ID: 29783 · Report as offensive
Profile Papa Zito
Avatar

Send message
Joined: 7 Feb 03
Posts: 257
Credit: 624,881
RAC: 0
United States
Message 30016 - Posted: 25 Sep 2004, 17:23:19 UTC - in response to Message 29783.  

> Well, these are not exactly perfect times. RIght now, SETI@Home is having
> teething problems. So, yes, you will be doing more of something else. When
> we do get to a more stable platform and things are working as planned, the
> system will, over time, self-correct.
>
> We are seeing the "Perfect Storm" kind of situation. And because SETI (I am
> assuming SETI is your preference) is having back-end problems that reflects
> forward to making it seem un-good ...

I'm not sure you're getting it.

Even if all BOINC projects were running at 100% uptime, the results would still be different than what I'm proposing.

Any percentage split (current system) will result in my computer doing work on 2 projects, switching between the two based on my preferences.

I'm saying I'd like a "fallback" project, that ONLY runs if something is wrong with the first project.

Let's try some numbers:

We'll call the Primary project Project A, and the Secondary project Project B.

Current System (80/20 split):

Day 1: 4 WUs of Project A, 1 WU Project B
Day 2: 4 WUs of Project A, 1 WU Project B
((Project A goes down))
Day 3: 4 WUs of Project A (from cache), 1 WU Project B
Total: 12 Wus of Project A, 3 WUs Project B

Proposed System (Primary A, Secondary B):

Day 1: 5 WUs of Project A
Day 2: 5 WUs of Project A
((Project A goes down))
Day 3: 5 WUs of Project A (from cache; 3 WUs from Project B downloaded)
Total: 15 WUs of Project A

Is this making more sense?




------------------------------------


The game High/Low is played by tossing two nuclear warheads into the air. The one whose bomb explodes higher wins. This game is usually played by people of low intelligence, hence the name High/Low.
ID: 30016 · Report as offensive
Thunder

Send message
Joined: 3 May 03
Posts: 65
Credit: 993,581
RAC: 0
United States
Message 30020 - Posted: 25 Sep 2004, 17:40:44 UTC

Sort of Papa, but I think this is a little outside the way they intend BOINC to function, so I'm not sure if we'd ever see it implemented as such.

For what it's worth, I've found that setting your "backup" project at 5% (95/5 or 90/5/5 for more than 2 projects) is sufficient to make sure it does a little bit of work and keeps some queued without letting them expire before completing them, while only cutting minimally into the production of your primary project.

I know it's not your perfect solution, but it's darn close and I think sufficient for me. :)
ID: 30020 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 30037 - Posted: 25 Sep 2004, 19:33:39 UTC

Papa,

I follow your logic, and agree that it seems like a better plan. But, I think that in the long run the numbers Thunder proposed would be just as good of a way to get the results you want.

I am not an expert in queuing theory so I cannot easily refute the logic, but, my "gut" call is that with a careful selection of resource shares the end result is the same.

Don't lose the idea, now is not the time to get wrapped up in it ... we have too many issues that are a little more important. But you can shoot a line to Dr. Anderson and propose it as an addition in M4 ... Since we are in M1 or M2 (I can't find out if we, in fact, are considered to have entered M2 with M1 tasks still open), it will be some time before you might see it ...
<p>
For BOINC Documentaion: Click Me!


ID: 30037 · Report as offensive
Anthony Brixey
Avatar

Send message
Joined: 24 Jun 00
Posts: 102
Credit: 1,757,916
RAC: 0
United Kingdom
Message 30270 - Posted: 26 Sep 2004, 12:37:09 UTC

I feel that being able to set a project as a backup project would be a good idea not only for people who really only want to run one project. It would also be good for people who have slower machines that setting a project at 5% could put the WU close to or past its return date (not everyone has a fast machine or leaves it on 24/7).

With regard to future versions is there a page somewhere that gives what is to be added into what version and a list of what might be added to some unnamed version in the future. It would be nice if it was in plain English, as I don’t speak Techno very well.

Anthony
ID: 30270 · Report as offensive
KnightFire

Send message
Joined: 3 Apr 99
Posts: 9
Credit: 2,485
RAC: 0
United Kingdom
Message 30319 - Posted: 26 Sep 2004, 17:02:50 UTC - in response to Message 30270.  

> I feel that being able to set a project as a backup project would be a good
> idea not only for people who really only want to run one project. It would
> also be good for people who have slower machines that setting a project at 5%
> could put the WU close to or past its return date (not everyone has a fast
> machine or leaves it on 24/7).
>
> With regard to future versions is there a page somewhere that gives what is to
> be added into what version and a list of what might be added to some unnamed
> version in the future. It would be nice if it was in plain English, as I don’t
> speak Techno very well.
>
> Anthony
>
>

Having a 'secondary' project causes problems for the secondary project though, it could be done but needs thought. For example, you would make sure you didnt download any WUs for the secondary project until you were about to crunch them, otherwise they would probably go past their deadline. You would also need to finish any WUs that you did start otherwise again, you are causing more problems than good for the secondary project. Some projects (like CPDN) wouldnt make a good secondary project due to the long running WUs, it takes around 600 hours or so to finish one so having it only run occasionally will not get the work done.

DC projects always need to be thought through from the projects point of view, rather than the user, and the project doesnt want users that keep downloading WUs and not finishing them in time.

ID: 30319 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 30402 - Posted: 26 Sep 2004, 23:19:15 UTC - in response to Message 30319.  

> DC projects always need to be thought through from the projects point of view,
> rather than the user, and the project doesnt want users that keep downloading
> WUs and not finishing them in time.

They had better be thought through from that perspective or they will not succeed. SETI@Home Classic did and got 5,000,000 users to try it ... other projects did not, and got nothing to speak of ...

I am not convinced that this is a neccessary addition that would not work acceptably with the current mechanisms, but you are correct that the queueing is an issue. With a short life queue (1 to 2 days) this might be viable, or that the secondary project has to meet some much more restrictive rules than are currently in the system.

The real issue right now is that we do not have a seamless environment. Heck, we don't even have a good availablility of projects. We have some of that for some of us (like me) but most people do not have access to Predictor@Home, LHC@Home as "fill-in" when SETI@Home is off the air. I still do think that cp.net is a very good safety-net to keep us occupied, and still suggest until the other projects open back up for members that they go there ....
<p>
For BOINC Documentaion: Click Me!


ID: 30402 · Report as offensive
Anthony Brixey
Avatar

Send message
Joined: 24 Jun 00
Posts: 102
Credit: 1,757,916
RAC: 0
United Kingdom
Message 30521 - Posted: 27 Sep 2004, 9:40:50 UTC
Last modified: 27 Sep 2004, 9:45:31 UTC

Below is a thread I put in the Wish List a couple of weeks ago. Any project that is suitable as a backup project (not CPDN as Knightfire says) has a tick box in the project preferences page, there could also be a number box so you could have a number of backup projects and set them in your order of preference, just in case there is no work from your first choice of backup project.

"Something that I think would be an improvement would be to be able to set a project as a backup project so if someone only wants to run one project they could without having any downtime, as all projects can have extended periods of no work being issued. At the moment the only way to have a backup project is to have a second project and set its resource share to a minimum which is taking CPU time away from the primary project and means that the WU’s from the secondary project could become out of date. If when a project is set as a backup project no WU’s are stored on the computer, then when the computer would be idle because of no work from the primary project an automatic setting of about one day’s work would then be downloaded and worked on. Then when there is work from the primary project the backup projects work continues being processed (equivalent to 100% resource share) until all of the WU’s have been completed. Of course not all projects would be suitable as backup projects but any project with WU’s that can be completed within a couple of days would."

ID: 30521 · Report as offensive
Profile Papa Zito
Avatar

Send message
Joined: 7 Feb 03
Posts: 257
Credit: 624,881
RAC: 0
United States
Message 30561 - Posted: 27 Sep 2004, 14:13:15 UTC - in response to Message 30319.  

> Having a 'secondary' project causes problems for the secondary project though,
> it could be done but needs thought. For example, you would make sure you didnt
> download any WUs for the secondary project until you were about to crunch
> them, otherwise they would probably go past their deadline. You would also
> need to finish any WUs that you did start otherwise again, you are causing
> more problems than good for the secondary project.

Yes, you are correct. Also, I already said all this, and gave examples of how it could be done.

> Some projects (like CPDN)
> wouldnt make a good secondary project due to the long running WUs, it takes
> around 600 hours or so to finish one so having it only run occasionally will
> not get the work done.

Agreed. I personally think the CPDN WUs are ridiculous anyway.




------------------------------------


The game High/Low is played by tossing two nuclear warheads into the air. The one whose bomb explodes higher wins. This game is usually played by people of low intelligence, hence the name High/Low.
ID: 30561 · Report as offensive

Message boards : Number crunching : Project sharing idea


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.