Do we have more data than volunteers?

Message boards : Number crunching : Do we have more data than volunteers?
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1870035 - Posted: 29 May 2017, 0:23:36 UTC

I just finished an article in "EndGadget" which was sent out via a notification on my Bonic client.

It asserted that we didn't have enough volunteers to crunch the data that the Seti@home project telescopes are making available.

Only 150,000 now (down from 1.5 Million).

Are we that backed up?

I apologize if this is the wrong message area to post it in. Please move it to a more reasonable choice.

Thank you.

Tom Miller
A proud member of the OFA (Old Farts Association).
ID: 1870035 · Report as offensive
Profile Darrell
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,418,681
RAC: 0
United States
Message 1870052 - Posted: 29 May 2017, 2:56:45 UTC - in response to Message 1870035.  

Read the article also, and after your post checked this website statistics page:

https://boincstats.com/en/stats/0/project/detail/user

It confirms the amounts stated in the article. The third graph begs the question of what happened from April 21 to April 22nd to cause a loss of 39,000 active users?

Then I checked the Boinc combined user statistics to see if it was just users switching projects:

https://boincstats.com/en/stats/-1/project/detail/user

It also shows the same drop in users in April. Wonder what happened.
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1870052 · Report as offensive
Profile Mr. Kevvy Crowdfunding Project Donor*Special Project $250 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 3567
Credit: 1,114,826,392
RAC: 3,319
Canada
Message 1870089 - Posted: 29 May 2017, 10:33:23 UTC - in response to Message 1870052.  

It confirms the amounts stated in the article. The third graph begs the question of what happened from April 21 to April 22nd to cause a loss of 39,000 active users?


Just prior to this there was a purge being done of bad team pages; due to no check whether the team founder had any credit/work done (which was also fixed) the team pages were being used as a free hosting service by spammers, mostly for the type of things that can't get legitimate hosting and need to be sent in spam e-mails. This may have been the purge at completion of the accounts that created those pages.
ID: 1870089 · Report as offensive
Profile Darrell
Volunteer tester
Avatar

Send message
Joined: 14 Mar 03
Posts: 267
Credit: 1,418,681
RAC: 0
United States
Message 1870112 - Posted: 29 May 2017, 14:32:27 UTC - in response to Message 1870089.  
Last modified: 29 May 2017, 14:34:26 UTC

Just prior to this there was a purge being done of bad team pages; due to no check whether the team founder had any credit/work done (which was also fixed) the team pages were being used as a free hosting service by spammers, mostly for the type of things that can't get legitimate hosting and need to be sent in spam e-mails. This may have been the purge at completion of the accounts that created those pages.


I could see that as a valid explanation for the drop of 33,000 active Seti users (34% drop). But that does not explain the drop of 286,000 BOINC users (39% drop) at the same time. Even though the BOINC projects run the same BOINC server code {but not the same version}, there is no central processing unit that would cause all of the projects to execute the same purge at the same time. It more like one or two huge farms of computers suddenly went offline. Or is it some glitch in the statistics page?
... and still I fear, and still I dare not laugh at the Mad Man!

Queen - The Prophet's Song
ID: 1870112 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 21006
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1870116 - Posted: 29 May 2017, 14:50:36 UTC

Some projects did do a purge of "spam" accounts and duplicate accounts, but that certainly doesn't account for the vast drop - I would be looking at the way the statistics are generated and collated.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1870116 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6533
Credit: 196,805,888
RAC: 57
United States
Message 1870133 - Posted: 29 May 2017, 16:16:39 UTC

SETI@home has been at ~150,000 active users for many years.
I'd say the largest drop came from the change from SETI@home classic to the BOINC platform.
My team had hundreds of active users in the classic days. Less than 50 changed over to BOINC and only 13 are active today.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1870133 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1870159 - Posted: 29 May 2017, 19:25:35 UTC - in response to Message 1870133.  

So, Do we have more data than volunteers?

Tom
A proud member of the OFA (Old Farts Association).
ID: 1870159 · Report as offensive
rob smith Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer moderator
Volunteer tester

Send message
Joined: 7 Mar 03
Posts: 21006
Credit: 416,307,556
RAC: 380
United Kingdom
Message 1870165 - Posted: 29 May 2017, 20:06:48 UTC

That is not a simple one to answer - if you take work units as the unit of data, and humans involved as the unit of volunteers then of course we have far more data than volunteers.
If the question you are asking is "Is there more data available than the current volunteer base can process?" then the answer is probably yes, but that is a different question. However it is not a bad thing, the project should always have a reserve of data ("tapes") to process, otherwise even a small blip in the supply chain would result in there being no data available to process - something that has been seen before, and no doubt caused some volunteers to drop out.
Bob Smith
Member of Seti PIPPS (Pluto is a Planet Protest Society)
Somewhere in the (un)known Universe?
ID: 1870165 · Report as offensive
Profile Advent42
Avatar

Send message
Joined: 23 Mar 17
Posts: 175
Credit: 4,015,683
RAC: 0
Ireland
Message 1870168 - Posted: 29 May 2017, 20:37:15 UTC - in response to Message 1870165.  

That is not a simple one to answer - if you take work units as the unit of data, and humans involved as the unit of volunteers then of course we have far more data than volunteers.
If the question you are asking is "Is there more data available than the current volunteer base can process?" then the answer is probably yes, but that is a different question. However it is not a bad thing, the project should always have a reserve of data ("tapes") to process, otherwise even a small blip in the supply chain would result in there being no data available to process - something that has been seen before, and no doubt caused some volunteers to drop out.


Also my understanding is that with the "Break Through Listen" coming on stream....there will be quite a substantial amount of data coming down the pipe line......I may have got that wrong....but if not, then it makes sense....
Also, as mobile phone apps have become more popular....it would appear less work units are getting processed....as compared to a desktop or laptop even.....
ID: 1870168 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1870181 - Posted: 29 May 2017, 22:31:40 UTC - in response to Message 1870168.  

Also, as mobile phone apps have become more popular....it would appear less work units are getting processed....as compared to a desktop or laptop even.....
Sadly, this. It to me is somewhat of a similar correlation to what we are seeing in retail, big box traditional retail is dying a slow (but seems to be accelerating) death due to pressure from the online retailers who are catering to 'lazy consumers', having found ways to now even offer same day delivery on a growing number of items, and similarly PCs and notebooks are going in the same direction. Not that either brick and mortar stores or PC's will go away anytime in the foreseeable future, but both will continue to lose market share, and on our side of things, even if mobile BOINC apps become more common and powerful (if they even exist, as I have no idea, I don't pay attention to that side of things, I'm a PC guy) I seriously doubt that they could process tasks at anywhere near the rate that a decent powered PC could due to all their limitations- battery, processor, effect on system performance would be a few guesses. Just my .02 on it.

ID: 1870181 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 5124
Credit: 276,046,078
RAC: 462
Message 1870185 - Posted: 30 May 2017, 2:45:48 UTC - in response to Message 1870181.  
Last modified: 30 May 2017, 2:59:29 UTC

Also, as mobile phone apps have become more popular....it would appear less work units are getting processed....as compared to a desktop or laptop even.....
Sadly, this. It to me is somewhat of a similar correlation to what we are seeing in retail, big box traditional retail is dying a slow (but seems to be accelerating) death due to pressure from the online retailers who are catering to 'lazy consumers', having found ways to now even offer same day delivery on a growing number of items, and similarly PCs and notebooks are going in the same direction. Not that either brick and mortar stores or PC's will go away anytime in the foreseeable future, but both will continue to lose market share, and on our side of things, even if mobile BOINC apps become more common and powerful (if they even exist, as I have no idea, I don't pay attention to that side of things, I'm a PC guy) I seriously doubt that they could process tasks at anywhere near the rate that a decent powered PC could due to all their limitations- battery, processor, effect on system performance would be a few guesses. Just my .02 on it.


Android Bonic / Seti applications exist that will run on your smart cellphone. It is available via Google Play. The setup includes a large number of different research choices, including Seti. As I understand it, it won't process unless the phone is plugged into a charger, the battery is above 90% and the screen is turned off. As far as I can tell, it is running two tasks on a 4 cpu phone (I think it is 4 cpus). And the progress is minuscule. :( And I just realized I had an "unactivated" Android phone laying around doing nothing, so I just installed the Bonic/Seti on it, and it is waiting till it is charged up to get going. Doesn't have to have a phone number to be used :)

But hey, I'm the guy who played with processing Bonic/Seti on a couple three different netbooks and an AMD A-10 for a while... and they were not great producers either :)

Tom
A proud member of the OFA (Old Farts Association).
ID: 1870185 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6533
Credit: 196,805,888
RAC: 57
United States
Message 1870246 - Posted: 31 May 2017, 2:13:19 UTC - in response to Message 1870185.  

I believe someone made a statement regarding the volume of Breakthrough Listen initiative data, the current method of processing, and number of volunteer systems. To the effect that backlog would occur.

Personally I don't see a backlog of data as a problem unless storing the data sets becomes an issue.
It wasn't that long ago that we were processing the data faster than it could be collected. So they had turned the application scanning resolution to the max. Which let data be analyzed with more detail.
However a few years later an issue of processing data faster than it could be delivered from the telescope occurred.
I seem to recall that there was a period of 30 or 60 days in an October or November time frame that no new data was being split.

Given a large portion of the money over 10 years for Breakthrough Listen is going towards the telescope time.
Time will be required to analyze the data unless funding is reorganized or another initiative is made to fund the backend/processing of the data.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the BP6/VP6 User Group today!
ID: 1870246 · Report as offensive
Profile Gary Charpentier Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 25 Dec 00
Posts: 29414
Credit: 53,134,872
RAC: 32
United States
Message 1870254 - Posted: 31 May 2017, 3:45:24 UTC - in response to Message 1870246.  

I believe someone made a statement regarding the volume of Breakthrough Listen initiative data, the current method of processing, and number of volunteer systems. To the effect that backlog would occur.

Personally I don't see a backlog of data as a problem unless storing the data sets becomes an issue.
It wasn't that long ago that we were processing the data faster than it could be collected. So they had turned the application scanning resolution to the max. Which let data be analyzed with more detail.
However a few years later an issue of processing data faster than it could be delivered from the telescope occurred.
I seem to recall that there was a period of 30 or 60 days in an October or November time frame that no new data was being split.
Which followed a year+ of the telescope being offline for maintenance IIRC.
Given a large portion of the money over 10 years for Breakthrough Listen is going towards the telescope time.
Time will be required to analyze the data unless funding is reorganized or another initiative is made to fund the backend/processing of the data.

Right now we are only getting Greenbank from Breakthrough Listen, soon we should also be getting data from Parks. Not much after that the Square kilometer should come online. It is an exciting time to be searching.

Unfortunately people are turning off their PC's at home and just using smartphones. While the phone can crunch, it isn't a 1000 core GPU, it is a 4 core slow non-optimized general CPU with tiny caches. I do have one ARM crunching, a Raspberry PI 3, and it seems to have settled in to a RAC about 300, but it also crunches some Einstein units with a RAC there of 100. I don't think the 20 or so on SetiBeta makes much difference, but I do have something else running on it 24/7 so you would get higher numbers ~15% if you set one up. Also IIRC when I had a Pentium II @566 MHz going it got a RAC of around 600 for comparison.

The real issue is we, earth, don't yet have enough computational power to search all the data collected to the level we are for the narrow band that Seti@home is looking in. IIRC Seti@home only scans 2.5MHz centered on the water hole. Many of the telescopes collect data several octaves wider. Note it isn't thrown away, but SERENDIP can only do a search for really strong signals, it wont be detecting leakage. In some respects that is a shame as we may have already received an ET, but we just don't have the ability to find the needle in the cosmic haystack.

So to answer the question, yes Seti can use more computers.
ID: 1870254 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6324
Credit: 106,370,077
RAC: 121
Russia
Message 1870270 - Posted: 31 May 2017, 6:47:25 UTC

yes
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1870270 · Report as offensive
Profile Robbie Klinkenberg
Volunteer tester

Send message
Joined: 7 Sep 16
Posts: 5
Credit: 505,953
RAC: 0
Netherlands
Message 1870304 - Posted: 31 May 2017, 12:33:54 UTC - in response to Message 1870185.  
Last modified: 31 May 2017, 12:35:12 UTC


Android Bonic / Seti applications exist that will run on your smart cellphone. It is available via Google Play. The setup includes a large number of different research choices, including Seti. As I understand it, it won't process unless the phone is plugged into a charger, the battery is above 90% and the screen is turned off.

As far as I can tell, it is running two tasks on a 4 cpu phone (I think it is 4 cpus)

Tom


You can change these options in the setting menu in the app, you may need to activate "advance settings" or something similar to that.
You can also change the number of cores that are used, I think the default is 2 so if you want to run on all cores you need to select 4.

Personally I have an old Samsung Galaxy S4 mini running 1 task of SETI and 1 task of Einstein.
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8099963

And sometimes I run my own Nexus 6P on SETI beta, which has 8 cores and does run 8 tasks. (currently no task running)
https://setiweb.ssl.berkeley.edu/beta/show_host_detail.php?hostid=80690

Running times vary but it usually it takes a lot of time and you don't get much credit for your time but you just have to let it go and not worry about it.
ID: 1870304 · Report as offensive
Profile Jord
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 15183
Credit: 4,362,181
RAC: 3
Netherlands
Message 1870333 - Posted: 31 May 2017, 16:27:48 UTC - in response to Message 1870304.  

Personally I have an old Samsung Galaxy S4 mini running 1 task of SETI and 1 task of Einstein.
https://setiathome.berkeley.edu/show_host_detail.php?hostid=8099963

My Huawei G700 runs tasks in about 200,000 - 300,000 seconds. The first GUPPI took a whopping 330,000 seconds, numbers I hadn't seen since running Classic. I have since also changed the amount of CPU cores down to 2, as running 3 the CPU was 49C hot, running 2 just 37C. During times when it's hot already from the weather, that matters.

One thing though, due to changes to the operating system the present Seti science applications do no longer work on Android 7. They err immediately.
ID: 1870333 · Report as offensive
Chris Oliver Project Donor
Avatar

Send message
Joined: 4 Jul 99
Posts: 72
Credit: 134,288,250
RAC: 15
United Kingdom
Message 1870902 - Posted: 3 Jun 2017, 15:22:20 UTC

I personally know of 11 friends who are former hard core crunchers for Seti and they invested a lot of money on high performance multi GPU equipped rigs. But they felt their contributions to the project were not appreciated after the introduction of Creditnew (also known as credit screw!). They found themselves crunching the same amount of tasks as the previous week but for half the credit and got disheartened. They are still crunching for other projects with more generous credit schemes.
ID: 1870902 · Report as offensive
EdwardPF
Volunteer tester

Send message
Joined: 26 Jul 99
Posts: 389
Credit: 236,772,605
RAC: 374
United States
Message 1870903 - Posted: 3 Jun 2017, 15:31:56 UTC - in response to Message 1870902.  

+1

As ridiculous as it seems ... I believe it is real ... (sigh) (shrug)

EdF
ID: 1870903 · Report as offensive
Profile Dimly Lit Lightbulb 😀
Volunteer tester
Avatar

Send message
Joined: 30 Aug 08
Posts: 15399
Credit: 7,423,413
RAC: 1
United Kingdom
Message 1870953 - Posted: 3 Jun 2017, 21:53:00 UTC - in response to Message 1870902.  

I personally know of 11 friends who are former hard core crunchers for Seti and they invested a lot of money on high performance multi GPU equipped rigs. But they felt their contributions to the project were not appreciated after the introduction of Creditnew (also known as credit screw!). They found themselves crunching the same amount of tasks as the previous week but for half the credit and got disheartened. They are still crunching for other projects with more generous credit schemes.

An example of credit new Vs not credit new:



Member of the People Encouraging Niceness In Society club.

ID: 1870953 · Report as offensive
Profile Juan Jiménez

Send message
Joined: 15 May 99
Posts: 9
Credit: 15,674,908
RAC: 0
Germany
Message 1871160 - Posted: 4 Jun 2017, 21:16:27 UTC

I'll take up some of the slack. Just brought 7000+ cores online on SETI@Home. The reason why is hilarious, but unfortunately I can't share it here.
ID: 1871160 · Report as offensive
1 · 2 · 3 · Next

Message boards : Number crunching : Do we have more data than volunteers?


 
©2022 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.