Message boards :
Technical News :
Tenaya (Feb 24 2009)
Message board moderation
Previous · 1 · 2 · 3 · 4 · Next
Author | Message |
---|---|
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
Aloha! Better for who? For those not in the top 25 trying to get work (which the server doesn't prioritize based upon position anyway, so those in the top 25 are just as likely having problems getting more work as you do), or those in the top 25 who have to load their workunits via DVD and pay money to send the results back through the mail (because we know SETI@Home doesn't have the money to pay for pre-paid envelopes for these participants). I would kindly suggest that if you don't like reading those messages from your messages tab to simply not view your messages tab. Its not an error at all, its simply letting you know that it didn't get any work and it will try again in the future. That's the way BOINC was designed to work. |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
Isn't the real problem here that BOINC is not scalable? They do not permit (as far as I understand it) the creation of secondary boinc nodes to then re-distribute wu's received from commandcentral. (I could be wrong because I don't keep up with the secondary applications out there, but I know seti classic could do that). So the log jam is ultimately the bandwidth to the mothership (Berkeley) and no one should be surprised. More users--> less available bandwidth, for a fixed wu size. Of course, one could increase the wu size (length of required processing time), but that just shifts the log jam out a ways. |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
Einstein has at least one mirror site for downloads, so it CAN be done. |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
I forgot about mirroring in my question. However, does mirroring actually reduce bandwidth at the orginating hub? All the mirrors have to be filled and synch'd, which consumes bandwidth. And since these wu's are use-and-forget, I wonder if mirroring helps? (I mean if seti were to scale up to, say, 3M active hosts from our present 300K and if splitting and sourcing the data wouldn't be an issue, would mirroring enable or disable the scale-up.) |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
I forgot about mirroring in my question. However, does mirroring actually reduce bandwidth at the orginating hub? All the mirrors have to be filled and synch'd, which consumes bandwidth. And since these wu's are use-and-forget, I wonder if mirroring helps? (I mean if seti were to scale up to, say, 3M active hosts from our present 300K and if splitting and sourcing the data wouldn't be an issue, would mirroring enable or disable the scale-up.) Bandwidth would be reduced by a minimum of 50% at the source. i.e. The WU is downloaded to the mirror and any copies are transmitted from there. The assumption being that the mirror has greater bandwidth than the source site. Two WUs are downloaded from the mirror to the required hosts for an initial reduction of 50% bandwidth. Any reissues are additional bandwidth savings at the source site. This is basically guido.man's idea below. |
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
I forgot about mirroring in my question. However, does mirroring actually reduce bandwidth at the orginating hub? All the mirrors have to be filled and synch'd, which consumes bandwidth. And since these wu's are use-and-forget, I wonder if mirroring helps? (I mean if seti were to scale up to, say, 3M active hosts from our present 300K and if splitting and sourcing the data wouldn't be an issue, would mirroring enable or disable the scale-up.) Wouldn't the mirrors have to eventually send/receive data back to the main host (not necessary for static files, but workunits would be a different beast)? If you mirror the workunits to other servers, they still have to get data from SETI@Home and send it back to turn it in. This will simply move the problem around, but I don't think it will actually solve the problem. |
John McLeod VII Send message Joined: 15 Jul 99 Posts: 24806 Credit: 790,712 RAC: 0 |
I forgot about mirroring in my question. However, does mirroring actually reduce bandwidth at the orginating hub? All the mirrors have to be filled and synch'd, which consumes bandwidth. And since these wu's are use-and-forget, I wonder if mirroring helps? (I mean if seti were to scale up to, say, 3M active hosts from our present 300K and if splitting and sourcing the data wouldn't be an issue, would mirroring enable or disable the scale-up.) It does shave some of the bandwidth off. If the WU were sent out and teh cannonical result sent back, the bandwidth would be reduced by about half for those WUs. Mirroring the executables to be sent out can shave an enormous amount of bandwidth as the mirror sites only need to download one copy in order to send out a hundred thousand. BOINC WIKI |
[AF>France>Bourgogne]Patouchon Send message Joined: 25 Aug 01 Posts: 7 Credit: 3,461,672 RAC: 4 |
danke schoen, i will have a lok next time ;) seti1 was pretty good, seti2 will be better ? |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
I forgot about mirroring in my question. However, does mirroring actually reduce bandwidth at the orginating hub? All the mirrors have to be filled and synch'd, which consumes bandwidth. And since these wu's are use-and-forget, I wonder if mirroring helps? (I mean if seti were to scale up to, say, 3M active hosts from our present 300K and if splitting and sourcing the data wouldn't be an issue, would mirroring enable or disable the scale-up.) If the ONLY thing done on the mirror is to send out WUs and Science apps, then the transmissions between the mirror and the Lab would be minimal. Results would be sent directly from the client to the Lab, not back to the mirror. Only the splitters and the deleters would need to talk to the mirror. If there was more than one mirror, the first mirror could transmit to the second and others. |
PhonAcq Send message Joined: 14 Apr 01 Posts: 1656 Credit: 30,658,217 RAC: 1 |
Then mirroring provides an element of enhanced reliability but doesn't reduce the bandwidth problem, as long as every wu is created at Berkeley. With the mirror, Berkeley uses bandwidth to send the wu's to the mirror and Clients can download from the mirror. But Berkeley has used the same amount of bandwidth. Plus if the mirror is a mirror, then the same wu's are at Berkeley. So 2x the storage is needed. And then one needs to tell both sites when each wu is no longer needed. and so on. I see overhead and reliability but no reduction in bandwidth. Perhaps I'm wrong. |
Pooh Bear 27 Send message Joined: 14 Jul 03 Posts: 3224 Credit: 4,603,826 RAC: 0 |
During new application time mirroring would help because the science application would come from multiple sources. That's the only benefit I could see. My movie https://vimeo.com/manage/videos/502242 |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
I forgot about mirroring in my question. However, does mirroring actually reduce bandwidth at the orginating hub? All the mirrors have to be filled and synch'd, which consumes bandwidth. And since these wu's are use-and-forget, I wonder if mirroring helps? (I mean if seti were to scale up to, say, 3M active hosts from our present 300K and if splitting and sourcing the data wouldn't be an issue, would mirroring enable or disable the scale-up.) Currently: Telescope -> splitter -> download server -> client Proposed: Telescope -> splitter -> download server -> mirror(s) -> client The only possible advantage is that you could throttle the mirror process for optimal transfer -- but the same number of work units go from the download server to the mirrors. ... and that's only if downloads come exclusively from the mirror. You could get nearly the same gain by adding the ability to throttle the standard BOINC client, without the extra complexity. |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
Then mirroring provides an element of enhanced reliability but doesn't reduce the bandwidth problem, as long as every wu is created at Berkeley. With the mirror, Berkeley uses bandwidth to send the wu's to the mirror and Clients can download from the mirror. But Berkeley has used the same amount of bandwidth. Plus if the mirror is a mirror, then the same wu's are at Berkeley. So 2x the storage is needed. And then one needs to tell both sites when each wu is no longer needed. and so on. I see overhead and reliability but no reduction in bandwidth. Perhaps I'm wrong. You are exactly right. It is easier to move a problem around than it is to solve the problem. |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
No: Telescope -> splitter -> mirror(s) -> client Although so far as bandwidth is concerned it's a wash, but local storage is reduced.
??? Why throttle the mirror if it runs at 1GB?
That's what's proposed
Agree that there is less complexity, but why throttle anything if the mirror is located 'down the hill' from the lab...i.e. at the 1 GB end of the link? |
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
So essentially what you're proposing is not really a mirror, but a separate connection in the lab that runs at the full 1Gb (Gigabit) speed where the workunits are directly loaded. A mirror is exactly that, a mirror. It would 'mirror' or reflect what's located on another server, and any change to the original would automatically be updated on the mirror. It would need to get its data directly from the original to be considered a mirror. |
Josef W. Segur Send message Joined: 30 Oct 99 Posts: 4504 Credit: 1,414,761 RAC: 0 |
If it's a mirror, there has to be something which it mirrors; the term implies at least two copies. Simply locating the download server downhill with 1 GB access to the world is what would improve the situation. Each WU or application would only go from the SSL lab to that server once, multiple downloads by hosts would not be seen on the 100 MB link. The issue is maintenance and remote management, not any technical difficulty in having that server at a distance. On the Server Status page, the counts of 'Results' indicate a completed or expected host download of a WU, the number of WU files is half or less. For S@H Enhanced, there are less than 2.1 downloads of each WU. For AP the ratio had gotten down to about 2.7 prior to last weekend's difficulty. Each 50.2 GB 'tape' produces around 130 GB of WUs after being split for both Enhanced and Astropulse... Joe |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
If it's a mirror, there has to be something which it mirrors; the term implies at least two copies. You're right. I guess we should call it a remote download server instead of a mirror. |
1mp0£173 Send message Joined: 3 Apr 99 Posts: 8423 Credit: 356,897 RAC: 0 |
There is a huge difference between throttling the mirror, and throttling the process that moves work to the mirror. You have to see this to believe it, and most here haven't realized what they're seeing: a connection running at 95% of capacity works very well. A connection running at 105% of capacity works incredibly poorly. When you push past 95% the actual throughput drops to a fraction of the wire capacity.
This is really important: It is easier to move a problem around than it is to solve it. That gigabit connection does not matter if you can't load up the server faster than we can download from it. If the best connection from the lab to the download server is 100 megabits, then we can pull ten hours of downloads off in an hour. So, we reduce the client-to-download server limit, and we trade that for a splitter-to-download server limit. If copying work to the "mirror" uses up all the bandwidth, then uploads and reporting will suffer. There is no free lunch. |
RandyC Send message Joined: 20 Oct 99 Posts: 714 Credit: 1,704,345 RAC: 0 |
This is really important: You have yet to propose a solution. |
Grant (SSSF) Send message Joined: 19 Aug 99 Posts: 13715 Credit: 208,696,464 RAC: 304 |
You have yet to propose a solution. In previous discussion of the subject, he's made a couple of proposals. Grant Darwin NT |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.