SETI@HOME -- A Victim of it's Own Success?!?

Message boards : Number crunching : SETI@HOME -- A Victim of it's Own Success?!?
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
w1hue Project Donor
Volunteer tester

Send message
Joined: 4 Aug 00
Posts: 69
Credit: 5,492,898
RAC: 7
United States
Message 1340931 - Posted: 26 Feb 2013, 4:22:34 UTC

It's now taking over 1 day for my machines to successfully download a WU! I was considering adding the SETI project to a new machine, but why bother?
ID: 1340931 · Report as offensive
Speedy
Volunteer tester
Avatar

Send message
Joined: 26 Jun 04
Posts: 1639
Credit: 12,921,799
RAC: 89
New Zealand
Message 1340937 - Posted: 26 Feb 2013, 5:09:07 UTC

You ask why bother? I would do it because you want to help with the science or you would like to find out whether or not there are other people in the world apart from humans.

Part of the reason for your computer taking a day to successfully download one work unit may have been because the system was recovering from lab wide car appears over the weekend and this servers needed to catch up with the demand of the of the projects large user base.
ID: 1340937 · Report as offensive
Claggy
Volunteer tester

Send message
Joined: 5 Jul 99
Posts: 4654
Credit: 47,537,079
RAC: 4
United Kingdom
Message 1340987 - Posted: 26 Feb 2013, 10:58:13 UTC - in response to Message 1340931.  

It's now taking over 1 day for my machines to successfully download a WU! I was considering adding the SETI project to a new machine, but why bother?

Did your machine run of work while that Wu was downloading?

If it didn't, then it's not a problem,

If it did, then increase you cache size a little so you have a few more cached tasks in hand.

Claggy
ID: 1340987 · Report as offensive
Profile Paul D Harris
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1122
Credit: 33,600,005
RAC: 0
United States
Message 1340988 - Posted: 26 Feb 2013, 11:02:15 UTC

Same problem here why bother? It has been over 10 years now a new profile or design is needed or formula or whatever.
ID: 1340988 · Report as offensive
Profile Blurf
Volunteer tester

Send message
Joined: 2 Sep 06
Posts: 8962
Credit: 12,678,685
RAC: 0
United States
Message 1341075 - Posted: 26 Feb 2013, 21:16:16 UTC - in response to Message 1340988.  

Same problem here why bother? It has been over 10 years now a new profile or design is needed or formula or whatever.


Neat idea Paul but that'll cost money....no money, no major changes.


ID: 1341075 · Report as offensive
w1hue Project Donor
Volunteer tester

Send message
Joined: 4 Aug 00
Posts: 69
Credit: 5,492,898
RAC: 7
United States
Message 1341155 - Posted: 27 Feb 2013, 4:27:18 UTC - in response to Message 1340937.  

You ask why bother? I would do it because you want to help with the science or you would like to find out whether or not there are other people in the world apart from humans.

As for the science, the only real science, as far as I am concerned, is with the "AstroPulse" sub-project -- and for the most part, I only accept AP WUs.

As far as finding out if "intelligent beings are out there", I am already 99.999% sure that there are. But as far as ever receiving any signal from "them", I am just about as sure that we will NOT. Why? Mainly because the probability that other "people" with technology similar to ours exist within a reasonable radius (say within 100 light-years or so) at this particular sliver of time in the overall age of the universe is mighty, mighty slim.

That is why I dedicate most of my computer time to projects that have more of a chance of providing some "real" science.

So have at it, all you geeks out there expecting to receive a SETI signal within your life times! :-)
ID: 1341155 · Report as offensive
robertmiles
Volunteer tester

Send message
Joined: 16 Jan 12
Posts: 213
Credit: 4,117,756
RAC: 6
United States
Message 1341179 - Posted: 27 Feb 2013, 6:24:20 UTC

I've found that Astropulse workunits take days to download the input files to my computers, then only a few hours to run.

Therefore, this seems like time to ask: If your download server had enough GPUs, would it put less load on the server to just run all the Astropulse workunits than to send them anywhere?
ID: 1341179 · Report as offensive
robertmiles
Volunteer tester

Send message
Joined: 16 Jan 12
Posts: 213
Credit: 4,117,756
RAC: 6
United States
Message 1341185 - Posted: 27 Feb 2013, 6:35:19 UTC

Another idea: Is the largest input file of Astropulse workunits compressed in any way? If not, would compressing reduce its size enough to be worthwhile, after you add a decompression program to the list of files needed for each workunit?
ID: 1341185 · Report as offensive
Ianab
Volunteer tester

Send message
Joined: 11 Jun 08
Posts: 732
Credit: 20,635,586
RAC: 5
New Zealand
Message 1341220 - Posted: 27 Feb 2013, 9:48:29 UTC - in response to Message 1341185.  

Issue with the project using GPUs to process the work units is they would need thousands of them. A whole room full, needing maybe a megawatt of power, and the same to keep them cool. The project would need to build an actual supercomputer.

The project might take all day to get the work to you, but it's still handling 30-40,000 AP units every day. Imagine the rack of computers that would be needed to do this all stacked up in one place.

As for the compression, my understanding is the data is more like "white noise". A normal compression (zip) doesn't have much effect on it. A "lossy" compression might loose the faint data we are actually looking for, or introduce false data.

So, good questions, but the answers are negative.

That leaves the current situation, where the project is in the fortunate situation of having more volunteers than it can effectively supply work too 100% of the time. Arguably a good situation for the project, but a bit frustrating for the volunteers. Hence the "Victim of it's own success" comment. But, not being able to supply all the work we want is not a failure for the project. It's not even in their "mission statement". If they merely supply all the work that their hardware and net link can handle, then they are doing well.

Leaves us with 2 things we can do.

1 - choose a backup project. Set it for a lower priority, then if you have trouble get SETI work, your machine fetches work from whatever other projects you have chosen.
2 - Raise some more funds so they can buy better servers and routers to allow for more work units to be processed.

Just remember the Project NEVER promised that there would be a steady or unlimited supply of work units. That's just what is happening now.

Frustrating to the guys that are in competition trying to achieve high RACs, but that's not actually a priority of the project.

Ian
ID: 1341220 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1341222 - Posted: 27 Feb 2013, 10:01:53 UTC - in response to Message 1341220.  

Issue with the project using GPUs to process the work units is they would need thousands of them. A whole room full, needing maybe a megawatt of power, and the same to keep them cool. The project would need to build an actual supercomputer.

The project might take all day to get the work to you, but it's still handling 30-40,000 AP units every day. Imagine the rack of computers that would be needed to do this all stacked up in one place.

As for the compression, my understanding is the data is more like "white noise". A normal compression (zip) doesn't have much effect on it. A "lossy" compression might loose the faint data we are actually looking for, or introduce false data.

So, good questions, but the answers are negative.

That leaves the current situation, where the project is in the fortunate situation of having more volunteers than it can effectively supply work too 100% of the time. Arguably a good situation for the project, but a bit frustrating for the volunteers. Hence the "Victim of it's own success" comment. But, not being able to supply all the work we want is not a failure for the project. It's not even in their "mission statement". If they merely supply all the work that their hardware and net link can handle, then they are doing well.

Leaves us with 2 things we can do.

1 - choose a backup project. Set it for a lower priority, then if you have trouble get SETI work, your machine fetches work from whatever other projects you have chosen.
2 - Raise some more funds so they can buy better servers and routers to allow for more work units to be processed.

Just remember the Project NEVER promised that there would be a steady or unlimited supply of work units. That's just what is happening now.

Frustrating to the guys that are in competition trying to achieve high RACs, but that's not actually a priority of the project.

Ian

Well said - I think that sums it up nicely.

The one other thing we can usefully do is to keep our equipment (as far as possible) in good working order, and within limits of temperature etc. so that they return accurate results. That would ensure that the limited resources of bandwidth would be used as much as possible for science, rather than for correcting errors by crunching extra copies.
ID: 1341222 · Report as offensive
Wedge009
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 451
Credit: 431,396,357
RAC: 553
Australia
Message 1341240 - Posted: 27 Feb 2013, 12:19:18 UTC - in response to Message 1341185.  

Another idea: Is the largest input file of Astropulse workunits compressed in any way? If not, would compressing reduce its size enough to be worthwhile, after you add a decompression program to the list of files needed for each workunit?

I've been wanting to ask about this as well. I know it's been raised before and I recall back then that the answer was something along the lines of 'there would be too much stress on the servers to compress every work-unit'. Given that the servers are a lot more powerful now than they were back then, is this still an infeasible suggestion? I'll understand if it is not, but I am just wondering if it is still not even remotely possible. Given AP's 8 MiB WU size vs MB's ~300 KiB, it's clear to anyone that AP WUs consume a lot of what little available bandwidth there is.
Soli Deo Gloria
ID: 1341240 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1341249 - Posted: 27 Feb 2013, 12:46:53 UTC - in response to Message 1341240.  

Another idea: Is the largest input file of Astropulse workunits compressed in any way? If not, would compressing reduce its size enough to be worthwhile, after you add a decompression program to the list of files needed for each workunit?

I've been wanting to ask about this as well. I know it's been raised before and I recall back then that the answer was something along the lines of 'there would be too much stress on the servers to compress every work-unit'. Given that the servers are a lot more powerful now than they were back then, is this still an infeasible suggestion? I'll understand if it is not, but I am just wondering if it is still not even remotely possible. Given AP's 8 MiB WU size vs MB's ~300 KiB, it's clear to anyone that AP WUs consume a lot of what little available bandwidth there is.

The usual answer I give - and it's been raised multiple times - is "try one and see".

Even using LZMA compression, I don't think anyone has got above ~3% compression - except if you happen to try a B3_P1 task with the stuck bit. The ~50% compression you get then is a sure indicator that the job will overflow with 100% blanking in a very few seconds.

In other words, it isn't just the server load, but a cost-benefit balance between server load and minimal gain.
ID: 1341249 · Report as offensive
Wedge009
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 451
Credit: 431,396,357
RAC: 553
Australia
Message 1341253 - Posted: 27 Feb 2013, 12:55:21 UTC

No worries. I suppose I should have done that first - for some reason, I had the impression that the 8 MiB file would compress well. But maybe because I'm thinking of the XML part of the WU, which is text-based, whereas the bulk of the WU is binary data, right?
Soli Deo Gloria
ID: 1341253 · Report as offensive
juan BFP Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 16 Mar 07
Posts: 9786
Credit: 572,710,851
RAC: 3,799
Panama
Message 1341271 - Posted: 27 Feb 2013, 14:09:29 UTC - in response to Message 1341253.  
Last modified: 27 Feb 2013, 14:10:16 UTC

No worries. I suppose I should have done that first - for some reason, I had the impression that the 8 MiB file would compress well. But maybe because I'm thinking of the XML part of the WU, which is text-based, whereas the bulk of the WU is binary data, right?

Not be so sure, the compression of a highly random AP 8MB file will not produce a real gain in their size, and need a aditional work from the allready busy SETI servers to do that, remember what happens when AP splitters start.

Why they realy need is a way to use the 1 Gbps link (today they paid for 1 Gbps but could use only 100Mbps - something that makes no sense) or Split the Project in 2 diferent 100 Mbps (or more) links, one for AP and one for MB.
ID: 1341271 · Report as offensive
David S
Volunteer tester
Avatar

Send message
Joined: 4 Oct 99
Posts: 18352
Credit: 27,761,924
RAC: 12
United States
Message 1341275 - Posted: 27 Feb 2013, 14:24:22 UTC - in response to Message 1340937.  

... the system was recovering from lab wide car appears over the weekend ...

Does this mean something in NZ-ese that it doesn't in American? :-)

David
Sitting on my butt while others boldly go,
Waiting for a message from a small furry creature from Alpha Centauri.

ID: 1341275 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1341280 - Posted: 27 Feb 2013, 15:09:39 UTC - in response to Message 1341253.  
Last modified: 27 Feb 2013, 15:10:11 UTC

No worries. I suppose I should have done that first - for some reason, I had the impression that the 8 MiB file would compress well. But maybe because I'm thinking of the XML part of the WU, which is text-based, whereas the bulk of the WU is binary data, right?

Yes, that's right. And in the case of AP, the compressible XML part is a very, very small fraction of the file.

Actually, that reminded me of a discussion we had a while back about MB ('setiathome_enhanced') data files. As befits a long-term longitudinal study like ours, the basic format specification for a MB WU hasn't changed since the original Classic project was launched. And, I was surprised to remind myself, the project is already two-thirds of the age of the internet itself (or, to be strict, of the World Wide Web, if we date that - as most people do - from the release of the Mosaic browser in 1993).

Back in those early days, not every ISP, or even every long-distance or intercontinental link, was fully transparent to 8-bit binary data. In order to allow as many people as possible to join in, even over those difficult links, the data format SETI chose was a 6-bit binary coding, so that every character can be represented by a printable byte in the 128-character ASCII set (the first 32 non-printable control characters being too risky to use on the comms lines of the era).

Fourteen years later, I think we can trust 8-bit data ;). In theory, the MB application is fully compatible with the 8-bit binary data format used by AP files: some months ago, Eric put a test of this theory on his ever-lengthening Beta 'to-do' list. If that works out, a simple change to the splitter configuration will allow the generation of smaller (denser) data files, without any extra compression/decompression stages - that way, we get almost the same benefit as compression, practically for free. But we do have to find time to test it first...
ID: 1341280 · Report as offensive
Profile Mr. Kevvy Crowdfunding Project Donor*Special Project $250 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 3776
Credit: 1,114,826,392
RAC: 3,319
Canada
Message 1341322 - Posted: 27 Feb 2013, 18:36:02 UTC - in response to Message 1341280.  

Fourteen years later, I think we can trust 8-bit data ;). In theory, the MB application is fully compatible with the 8-bit binary data format used by AP files: some months ago, Eric put a test of this theory on his ever-lengthening Beta 'to-do' list. If that works out, a simple change to the splitter configuration will allow the generation of smaller (denser) data files, without any extra compression/decompression stages - that way, we get almost the same benefit as compression, practically for free. But we do have to find time to test it first...


Nice... had no idea they were using 6-bit characters. If there is a time to make the change, this would be it: the AP units are apparently having a "shorty storm" that's plugging up the bandwidth and aborting downloads for everyone, and it doesn't seem to be going away.
ID: 1341322 · Report as offensive
Profile Paul D Harris
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 1122
Credit: 33,600,005
RAC: 0
United States
Message 1341531 - Posted: 28 Feb 2013, 6:05:01 UTC - in response to Message 1341155.  

You ask why bother? I would do it because you want to help with the science or you would like to find out whether or not there are other people in the world apart from humans.

As for the science, the only real science, as far as I am concerned, is with the "AstroPulse" sub-project -- and for the most part, I only accept AP WUs.

As far as finding out if "intelligent beings are out there", I am already 99.999% sure that there are. But as far as ever receiving any signal from "them", I am just about as sure that we will NOT. Why? Mainly because the probability that other "people" with technology similar to ours exist within a reasonable radius (say within 100 light-years or so) at this particular sliver of time in the overall age of the universe is mighty, mighty slim.

That is why I dedicate most of my computer time to projects that have more of a chance of providing some "real" science.

So have at it, all you geeks out there expecting to receive a SETI signal within your life times! :-)


If they do exist they may not use radio waves or micro waves technology. They may use something entirely different such as light waves or mental telepathy or some other organic technology that SETI cannot detect.
ID: 1341531 · Report as offensive
Profile Mark Wyzenbeek
Avatar

Send message
Joined: 28 Jun 99
Posts: 134
Credit: 6,203,079
RAC: 0
United States
Message 1341567 - Posted: 28 Feb 2013, 7:45:28 UTC - in response to Message 1340931.  

It's now taking over 1 day for my machines to successfully download a WU! I was considering adding the SETI project to a new machine, but why bother?

Because Seti is at the bleeding edge of distributed computing.
The Universe is not only stranger than you imagine, it's stranger than you can imagine.

SETI@home classic workunits 1,405 CPU time 57,318 hours
ID: 1341567 · Report as offensive
robertmiles
Volunteer tester

Send message
Joined: 16 Jan 12
Posts: 213
Credit: 4,117,756
RAC: 6
United States
Message 1341572 - Posted: 28 Feb 2013, 7:48:35 UTC

Another idea for AstroPulse workunits: I assume that the load on the server varies with time of day or night. Are you able to set the server so that it will try to schedule the retries for incomplete downloads of the larger files to be preferably during the less busy times?
ID: 1341572 · Report as offensive
1 · 2 · 3 · Next

Message boards : Number crunching : SETI@HOME -- A Victim of it's Own Success?!?


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.