Message boards :
Technical News :
Maxed (Dec 16 2010)
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · 6 · Next
Author | Message |
---|---|
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
FYI, limits have been raised... - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
Gary Charpentier Send message Joined: 25 Dec 00 Posts: 30919 Credit: 53,134,872 RAC: 32 |
FYI, limits have been raised... Thanks for the update. |
Andy Lee Robinson Send message Joined: 8 Dec 05 Posts: 630 Credit: 59,973,836 RAC: 0 |
It's entirely up to the lab staff how they handle it, but either compression or raw CDATA for MB would shave ~30% off that part of the project's download requirements. Whether modifying the splitter code to generate CDATA format, or post-processing the existing format with compression, I'll leave to them. I'll only mention that post-processing would also compress the (non-trivial) XML header, whereas CDATA would not. Could either compress as is, or do a cdata version and compress that too, the xml header would be compressed in both cases, even if the payload grows a bit. A quick test - I converted an existing wu, decompressed the payload with ascii85 and made a cdata version: wu.xml = 375353 wu.xml.gz = 273716 wu-cdata.xml = 295612 wu-cdata.xml.gz = 279310 Interesting in this case that gzipping the current format WUs made a smaller file. So, all they need to do is make and store the current WUs as gz and add Content-Encoding: gzip to the http header on transfer, and all should be fine. |
Blurf Send message Joined: 2 Sep 06 Posts: 8962 Credit: 12,678,685 RAC: 0 |
FYI, limits have been raised... To.....what levels? |
RottenMutt Send message Joined: 15 Mar 01 Posts: 1011 Credit: 230,314,058 RAC: 0 |
FYI, limits have been raised... that's nice, but there is very little work to be split. please feed the splitters. |
Richard Haselgrove Send message Joined: 4 Jul 99 Posts: 14672 Credit: 200,643,578 RAC: 874 |
FYI, limits have been raised... Please enjoy your Saturday night away from the lab. |
Dirk Sadowski Send message Joined: 6 Apr 07 Posts: 7105 Credit: 147,663,825 RAC: 5 |
FYI, limits have been raised... Thanks for the news! But it's a pity, that the available WUs were not much. I agree. There is no reason whatsoever that the staff should bother about this small insignificant problem of no work to send out, on a weekend. Anyone who disagree with that should immediately sign up to baby sit the servers at the lab on weekends, for no pay whatsoever. Now we have again two different kind of humans. Members with enough WUs for at least 6 or 7 days. And members which have maybe only WUs for 3 days - which worry that they can't bridge the next 3 day outage. I'm the last ones. I guess my PCs will idle during the next 3 day outage, because they will not DL enough WUs from Monday to Tuesday morning (Berkeley time). But nothing for to upset. It's a pity that SETI@home can't use my PCs then. I would look into the lab at weekends, but what I could do? Clean the lab? This wouldn't help. OTOH, Berkeley is too far away from here where I live. The weather news from my home (Germany/Baden-Württemberg), on the main street no snow, the small street in front of my house with ~ 10 cm snow/ice and melts, the half windshield of my car is closed 50 % with snow (not more snow on the car) and also melts. From today to mid of the week we will have here + ~ 4 - 6 °C outside at the day. Maybe not white X-Mas. I wish all a nice 4th Advent.. |
Cheopis Send message Joined: 17 Sep 00 Posts: 156 Credit: 18,451,329 RAC: 0 |
With the servers running out of work units so fast these days, have there been any thoughts on how to process more work units between each required physical visit to the server room to load tapes or disks of whatever media the raw data is stored on these days? Is this process already optimized, or would it be possible to perhaps store the data on multiple external hard drives or on it's own separate RAID array? If it is possible to read and verify the data from the media faster than the machines can initially process said data to create workunits, than it should be possible to build up a store of unprocessed data when humans are available to feed the machine. The "extra" data could then be queued for access whenever one of the splitter CPU's has no available data to make work units from. Great for weekends, or after outages. Not sure if this would be something that the project would actually want to run full time, as I understand the bottleneck for the actual science is in the post-home-user stages. I'm imagining a couple really big USB external hard drives for "extra" data storage. These are getting pretty cheap. |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
Thanks for raising the limits, Matt. However, it appears that Worf is having trouble keeping up with the many kibble bowls to fill. Might Oscar help him out with the radar blanking work? "Time is simply the mechanism that keeps everything from happening all at once." |
JohnDK Send message Joined: 28 May 00 Posts: 1222 Credit: 451,243,443 RAC: 1,127 |
FYI, limits have been raised... Wasn't there too few tapes to split to begin with when they left on Friday? |
Robert Gammon Send message Joined: 29 Aug 01 Posts: 21 Credit: 1,573,250 RAC: 0 |
I am not sure how this could be done, but some of us ache for a fairness doctrine as applied to downloaded WUs. While the system was up for the last couple of weeks, I managed to snag 2 Astropulse and 8-9 Seti Enhanced WUs. Now the my last 6 Seti enhanced WUs are Pending Validation as the second computer is one of the machines that downloads 4-10 WUs per day. Granted most of them get returned inside the window of time, but still, I turn them around in 2 days, and wait 3-6 weeks for validation to happen. Seti is out of WUs now so my Boinc is working on other projects JUST to stay busy |
soft^spirit Send message Joined: 18 May 99 Posts: 6497 Credit: 34,134,168 RAC: 0 |
I am not sure how this could be done, but some of us ache for a fairness doctrine as applied to downloaded WUs. I think the "fairness" doctrine is known as "grab and growl". Grab what is there, and growl about it all you want, it will not change a thing. Janice |
Bill Walker Send message Joined: 4 Sep 99 Posts: 3868 Credit: 2,697,267 RAC: 0 |
I think the "fairness" doctrine is known as "grab and growl". Grab what is there, and growl about it all you want, it will not change a thing. Grrrrrrrrrrrrrrr. |
RottenMutt Send message Joined: 15 Mar 01 Posts: 1011 Credit: 230,314,058 RAC: 0 |
how come the people with a low RAC are always ridiculing the people, usually high RAC users, who are almost out of work when they are reporting a problem. i'm for a more fair work distribution method in which RAC is the basis. machines/users with a higher RAC should be rewarded more work, directly proportional to RAC, during the rationing periods. then we all could listen to the "computationally challenged" users complain they don't have enough work... |
Albireo380 Send message Joined: 21 Mar 08 Posts: 119 Credit: 1,570,025 RAC: 0 |
how come the people with a low RAC are always ridiculing the people, usually high RAC users, who are almost out of work when they are reporting a problem. I have a low RAC, Rottenmutt, and I haven't ridiculed anyone. I guess some people with low RAC are unhappy that high RAC crunchers seem to get lots of work units. Personally, as long as I can contribute a little, my life wont end if I run out of WUs for a while. But some folk get very "antsy" about it - just people I suppose - such is life. Never mind though, SETI is back up and running - that has got to be good. Regards & keep crunching : ) Tom |
Robert Gammon Send message Joined: 29 Aug 01 Posts: 21 Credit: 1,573,250 RAC: 0 |
Fairness lies in the eye of the beholder. I see the rationale for having as much seti work as your machine can handle, as the seti folks have had extended downtime over the last several months. High RAC, low RAC, that all depends on just how many computers you can employ and the relative hotness of each of the machines. Some of us only have one computer, and that one is several years old. If the High RAC users get all the WUs then we can be shut out. A slightly better approach would reward users who consistently turn in units that pass validation, turn them in on time, and grade higher for short turn around time. People who fail to turn in units on time get throttled on how many work units they have at one time. People who have compute errors on their work get limited on how many WUs can be outstanding at one time. People who turn in work units well inside the time limit (lets assume that they return them inside 5 days, but sysadmin makes the decision) get rewarded with more WUs. |
Geek@Play Send message Joined: 31 Jul 01 Posts: 2467 Credit: 86,146,931 RAC: 0 |
I don't understand all this bickering about who should and should not get work and suggesting new plans to implement a new "fairness doctorine". The only completely fair distribution method is the current one. The client makes a request and then either it is successful for not. If not try again. Any other plan is actually unfair to someone. If you feel that you are being restricted in some way in getting through to Berkeley servers I would suggest you try a different ISP. Boinc....Boinc....Boinc....Boinc.... |
Westsail and *Pyxey* Send message Joined: 26 Jul 99 Posts: 338 Credit: 20,544,999 RAC: 0 |
I don't understand all this bickering about who should and should not get work and suggesting new plans to implement a new "fairness doctorine". +100000 "The most exciting phrase to hear in science, the one that heralds new discoveries, is not Eureka! (I found it!) but rather, 'hmm... that's funny...'" -- Isaac Asimov |
tbret Send message Joined: 28 May 99 Posts: 3380 Credit: 296,162,071 RAC: 40 |
This isn't the politics board, but I didn't want to let either idea pass without adding a "correction," make of it what you will. I believe the first quote from Marx was intended as sarcasm. Not even Marx used that as a slogan. The "From each..." line is supposed to be the end result of establishing a Socialist utopia where Socialism has created an over-abundence of everything for everyone; not a motto or method for reaching utopia. That's just simply a statement of fact and not an opinion about that fact, one way or the other. Not even Marx (who was both nuts and a hypocrite - look it up) was so deluded as to think that people would work slavishly to live like their neighbor who doesn't work at all but gets a lot by needing a lot. That's a misuse of that quote and it's done all the time. Frankly, I can see where work unit distribution would be best for the project if those who produced more, got more. I *think* that was the point of the sarcasm; it makes little sense to feed everyone equally when all are not equally committed. I don't know how anyone would enforce such a thing, and if it were done I might well find myself with less work (I don't have any big number-crunchers). I can also see the frustration of those who have spent thousands of dollars and hundreds or thousands of hours optimizing their systems, being capable of thru-putting massive amounts of work units then find themselves unable to get the work. That's a lot more work and commitment than just downloading a screensaver. Shouldn't they get first dibs on the work? I just wanted to make clear that my opinion isn't self-serving; if there were only enough work to keep the big guns in work units I might find myself without work units. Still, to have all that optimized computing sitting around idle so I can let my P4 1.8MHz plod along for days doing the work someone else wanted and could have completed in a few minutes makes very little sense. Bret |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
That is the most sensible comment I have seen in quite a while on the subject. "Time is simply the mechanism that keeps everything from happening all at once." |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.