There will NEVER be enough work to keep everyones computer crunching 24/7.

This is NOT Seti 1, where they just kept sending out units over and over again to keep you happy. In addition, the majority of Seti crunchers haven't even come over to BOINC yet. Guess what will happen when they do.

BOINC was developed to be a MULTI Project Platform and to keep your comps crunching you're going to have to develop a MULTI Project attitude.

[/url] [/url]we find you ;-)

BOINC was developed to be a MULTI Project Platform and to keep your comps crunching you're are going to have to develop a MULTI Project attitude.
===========

Well thats all good advice Free Tips, funny I'm attached to Predictor and Seti & can't get work from either right now.

They should have put in the Headlines "Give you Computers a break Today, Join Boinc & you'll never have to run any WU's because we won't send you any"

JOIN TEAM

Well, lets assume for a moment that the sky a big with a lot of stars and planets capable of sustaining life. Lets also assume that one cannot look often enough to find that one signal that is going to change the world. This means there will always be work to do.

Lets assume the radio telescope remains accessible fur the SETI purpose. The problem is therefor not the amount of work, but to splitt the data coming from the radio telescope into workunits fast enough, right ?

Without further information on details how the splitting is done, I have to refer to the pieces that I found on this board. It appears that the splitters in Berkeley produce like 4 units per minute, which seems to be the reason why there is sometimes no work available. Hey, we are having the biggest supercomputer in the world here and can't solve a capacity problem ?

Does anyone understand where I'm going ?

Why not think about a way how we can use the massive computing power to take care of the splitting as well ? I can imagine an additional project. Priority project would be SETI, but if there is no workunits, then the SPLIT project would kick in. I'm sure that should be possible to do with the existing BOINC.

Lets say the splitter drops below a certain level of WU's. Instead of sending out workunits, it then sends out split units and gives them highest priority. I understand that the transfer of a large chunk of raw data would take a moment, but on the return path only the position where the split has to be made has to be transferred. These split positions can be validated the same way as the workunits.

It all depends on how large the raw file is and on how much bandwidth is available. Since the average upload and download speed is tracked anyway, these priority splitter units could for example only be handed out to the participants with high averages to reduce upload time.

What do you think ?

In theory that idea isn't too bad. But you would have to send a large chunk of data out. I wouldn't be surprised if the time to send the data to be split and then send it back again to be sent out yet again actually took a lot longer than what Berkeley can do on their splitters. Plus there are reliability factors to consider. Transmission errors, user computer failures and crashes, reinstallations, deletions, etc. There is also the potential for tampering that is a lot harder to catch. And there is the tagging/logging of workunits by Berkeley dealing with the resending of data and such. Right now they have enough capability to keep up with the demand as far as splitters go, but there are bottlenecks causing problems. I think that distributing the splitting would practically be another project in and of itself. I've thought of the possibility of using other machines for other tasks performed at Berkeley as well, but the workunit processing is really the biggest time restraint of any and seemingly has the least amount of risk without just being silly with the redundancies.

One thing I was curious about though is what each of the components are and their purpose. I know data is recorded onto tapes of some sort at Aricibo (however it's spelled) and mailed to Berkeley since there isn't a broadband internet connection there. From there it is a bit hazy though. The transitioner I don't know anything about really, but I suppose it might be the method of putting the tape to computer, and then the splitter would divide up what the transitioner gave it. And then assuming there aren't any other steps, it goes to the Scheduler and then to us. After that it gets processed and sent to the Validator. I've read the stuff on the SETI Classic page, but it doesn't provide much information about what happens when the tapes get to Berkeley. Perhaps Rom or one of the other Developers could provide some clarification.

Chris


> Well, lets assume for a moment that the sky a big with a lot of stars and
> planets capable of sustaining life. Lets also assume that one cannot look
> often enough to find that one signal that is going to change the world. This
> means there will always be work to do.
>
> Lets assume the radio telescope remains accessible fur the SETI purpose. The
> problem is therefor not the amount of work, but to splitt the data coming from
> the radio telescope into workunits fast enough, right ?
>
> Without further information on details how the splitting is done, I have to
> refer to the pieces that I found on this board. It appears that the splitters
> in Berkeley produce like 4 units per minute, which seems to be the reason why
> there is sometimes no work available. Hey, we are having the biggest
> supercomputer in the world here and can't solve a capacity problem ?
>
> Does anyone understand where I'm going ?
>
> Why not think about a way how we can use the massive computing power to take
> care of the splitting as well ? I can imagine an additional project. Priority
> project would be SETI, but if there is no workunits, then the SPLIT project
> would kick in. I'm sure that should be possible to do with the existing BOINC.
>
>
> Lets say the splitter drops below a certain level of WU's. Instead of sending
> out workunits, it then sends out split units and gives them highest priority.
> I understand that the transfer of a large chunk of raw data would take a
> moment, but on the return path only the position where the split has to be
> made has to be transferred. These split positions can be validated the same
> way as the workunits.
>
> It all depends on how large the raw file is and on how much bandwidth is
> available. Since the average upload and download speed is tracked anyway,
> these priority splitter units could for example only be handed out to the
> participants with high averages to reduce upload time.
>
> What do you think ?
>
>

> One thing I was curious about though is what each of the components are and
> their purpose. I know data is recorded onto tapes of some sort at Aricibo
> (however it's spelled) and mailed to Berkeley since there isn't a broadband
> internet connection there. From there it is a bit hazy though. The
> transitioner I don't know anything about really, but I suppose it might be the
> method of putting the tape to computer, and then the splitter would divide up
> what the transitioner gave it. And then assuming there aren't any other steps,
> it goes to the Scheduler and then to us. After that it gets processed and sent
> to the Validator. I've read the stuff on the SETI Classic page, but it doesn't
> provide much information about what happens when the tapes get to Berkeley.
> Perhaps Rom or one of the other Developers could provide some clarification.
>
I believe, but I am not quite certain that the splitter takes the raw tapes, and creates WUs out of them. The scheduler takes the WUs and distributes them to clients, and also takes reports and adds them to the database. The data server takes results (uploaded before the report is sent), and adds them to a database. The transitioner has some responsibility during the verification phase, I believe that it either creates new copies of the WU to be sent again, or adds Credit where it is due, or does the verification, or maybe all of the above.

OK, ythis has gotten me intrigued. Would somebody from the project tell us if this is how it all fits together? Or if it's now, how do all the machines work into the picture. Just in broad terms like John used, we're not looking to earn a degree on the topic. But I am curious now how it all goes together.


> I believe, but I am not quite certain that the splitter takes the raw tapes,
> and creates WUs out of them. The scheduler takes the WUs and distributes them
> to clients, and also takes reports and adds them to the database. The data
> server takes results (uploaded before the report is sent), and adds them to a
> database. The transitioner has some responsibility during the verification
> phase, I believe that it either creates new copies of the WU to be sent again,
> or adds Credit where it is due, or does the verification, or maybe all of the
> above.
>

--
Mark

You can find a great deal of information about BOINC from here:
http://boinc.berkeley.edu/

A quick breakdown of the backend logic is here:
http://boinc.berkeley.edu/backend_logic.php

Several tapes worth of data is stored on a drive array where the splitters proceed to cut the raw data into chunks. Each workunit is added to the boinc database and the master science database. After a workunit is added to the boinc database it is scheduled to be looked at by the trasitioner.

The transitioner looks at a workunit and all the results generated for the workunit. If their are existing results it looks through them all to see if any errors or timeouts have ocurred, if so it'll generate another result until ntarget results are reached. If the workunit has enough successful results to meet the min quorum it schedules the workunit for the validator. If it has sucessfully been validated then it is scheduled for the assimilator. If it has been successfully assimilated into the master science database, then it is scheduled for deletion.

When the core client submits results it is talking to the scheduler, each result submitted causes that workunit to be scheduled for the transitioner to look at.


----- Rom
BOINC Development Team, U.C. Berkeley

Thanks, Rom. Contrary to the flack you guys are getting in some of these threads, some of us really appreciate the effort you guys are putting forth.

Most of the whiners probably don't even understand networking more than 2 desktops together, let alone a multi server platform you guys are dealing with.


--
Mark

> You can find a great deal of information about BOINC from here:
> http://boinc.berkeley.edu/
>
> A quick breakdown of the backend logic is here:
> http://boinc.berkeley.edu/backend_logic.php
>
> Several tapes worth of data is stored on a drive array where the splitters
> proceed to cut the raw data into chunks. Each workunit is added to the boinc
> database and the master science database. After a workunit is added to the
> boinc database it is scheduled to be looked at by the trasitioner.
>
> The transitioner looks at a workunit and all the results generated for the
> workunit. If their are existing results it looks through them all to see if
> any errors or timeouts have ocurred, if so it'll generate another result until
> ntarget results are reached. If the workunit has enough successful results to
> meet the min quorum it schedules the workunit for the validator. If it has
> sucessfully been validated then it is scheduled for the assimilator. If it
> has been successfully assimilated into the master science database, then it is
> scheduled for deletion.
>
> When the core client submits results it is talking to the scheduler, each
> result submitted causes that workunit to be scheduled for the transitioner to
> look at.
>
>
> ----- Rom
> BOINC Development Team, U.C. Berkeley
>
>


OWOWOWOWOw now I have a headache, thank you Rom, err may I call you Rom, for that excellent explanation hang in ther we are behind you 99.99% lolol I am grumpy the other .01% lololololol

Bah!!! I am in it for the money.

Message from the Head ;o) of the
Horse Head Nebula Branch of Seti@Home,

Check is in the mail

I must say that all the whiners, many of who probably run windows, and all who have experience of it do amuse me. Bill Gates has the cheek to ask for sometimes-large sums of money for his software that is constantly in need of patches and service packs etc. Think of all the developers coding and money invested to turn out these products!!

I must say that I too suffer from some frustration when there are no Seti2 wus to crunch, however, I do have back up projects for those occasions so I switch over and wait for the new work to turn up.

Thank you Rom for your explanation of how the system works, and congratulations on getting this far with the resources that you do have available. I have enjoyed the ride through Beta into Seti at Home, and will continue to enjoy it whether or not the work is 24/7. If it isn't I will fill up that time with another project.

Dunc