vlar data and breakthrough listen

Message boards : Number crunching : vlar data and breakthrough listen
Message board moderation

To post messages, you must log in.

AuthorMessage
DonS

Send message
Joined: 15 May 16
Posts: 7
Credit: 1,540,164
RAC: 2
Canada
Message 1793695 - Posted: 5 Jun 2016, 16:23:03 UTC

I'm new again to setiathome and an curious about the .vlar extensions I'm getting and the Breakthrough Listen setiathome data and how I might know if I'm running it's jobs (as opposed to what the project is about which I've just started to read on).

I'm googling left and googling right but not getting a clear picture of what these are or if they're related. Info or pointers to pages perhaps?

Don
ID: 1793695 · Report as offensive
Profile tullio
Volunteer tester

Send message
Joined: 9 Apr 04
Posts: 8797
Credit: 2,930,782
RAC: 1
Italy
Message 1793696 - Posted: 5 Jun 2016, 16:30:13 UTC - in response to Message 1793695.  

Go to Number Crunching.
Tullio
ID: 1793696 · Report as offensive
Profile Angela Special Project $75 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 16 Oct 07
Posts: 13130
Credit: 39,854,104
RAC: 31
United States
Message 1793741 - Posted: 5 Jun 2016, 19:51:54 UTC

I am moving this thread to Number Crunching as I believe that more interested parties will participate in discussion of this topic there.
ID: 1793741 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1793748 - Posted: 5 Jun 2016, 20:24:03 UTC - in response to Message 1793695.  

I'm new again to setiathome and an curious about the .vlar extensions I'm getting and the Breakthrough Listen setiathome data and how I might know if I'm running it's jobs (as opposed to what the project is about which I've just started to read on).

I'm googling left and googling right but not getting a clear picture of what these are or if they're related. Info or pointers to pages perhaps?

Don



Welcome back Don

I'm going to link 2 places for you.

The first is Eric announcement on the Breakthrough listen

http://setiathome.berkeley.edu/forum_thread.php?id=79411

Next is his description of what the file name means

http://setiathome.berkeley.edu/forum_thread.php?id=79411&postid=1778453


blc3_2bit_guppi_57451_19304_HIP62472_0003

blc3: Breakthrough Listen
2bit: bits per real sample (4 bit per complex sample)
guppi: File format (Greenbank Ultimate Pulsar Processing Instrument)
57451: Modified Julian Date
19304: Seconds past midnight
HIP62472: Object Name
0003: Observation sequence number


When this first came out, they were primarily sent to the CPU as the data was more complex than what the GPU applications at the time could handle.

Since the initial announcement, they have been release to the graphic cards.

I'm not going to define that VLAR stand for, someone else can do. ( I tend to confuse the terms)

What is know is that the GreenBank data is more complex than the Arecibo data as it is a long time period observation of a specific point the sky (rather than a quick scan)

As such, the data is bigger and more complex requiring the applications to do more work.

This is just a quick and dirty version.
ID: 1793748 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1793752 - Posted: 5 Jun 2016, 20:39:39 UTC - in response to Message 1793748.  

When this first came out, they were primarily sent to the CPU as the data was more complex than what the GPU applications at the time could handle.

What is know is that the GreenBank data is more complex than the Arecibo data as it is a long time period observation of a specific point the sky (rather than a quick scan)

As such, the data is bigger and more complex requiring the applications to do more work.

This is just a quick and dirty version.

And sadly, wrong. Can we have a go at stamping out this urban myth, please?

The amount of work that needs to be done is pretty darn near identical. But it's work which is not well suited to the design of, specifically, NVidia GPUs. They tend to take longer to achieve the same outcome as other computational resources.
ID: 1793752 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1793754 - Posted: 5 Jun 2016, 20:46:19 UTC - in response to Message 1793752.  

. But it's work which is not well suited to the design of, specifically, NVidia GPUs. They tend to take longer to achieve the same outcome as other computational resources.


On this point, I will say that statement isn't necessarily true. Depending on the class of Nvidia GPU and the application, they can and do better than some of their counter parts.
ID: 1793754 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1793755 - Posted: 5 Jun 2016, 20:48:45 UTC - in response to Message 1793754.  
Last modified: 5 Jun 2016, 20:53:36 UTC

. But it's work which is not well suited to the design of, specifically, NVidia GPUs. They tend to take longer to achieve the same outcome as other computational resources.

On this point, I will say that statement isn't necessarily true. Depending on the class of Nvidia GPU and the application, they can and do better than some of their counter parts.

On Breakthrough Listen VLAR tasks? Do elaborate, please - we'd all like to achieve that happy outcome. (Under Windows, for the majority)

(Edit - I'm in the middle of putting together a new installer, to assist users to deploy optimised apps. If such an app exists, and has been tested for accuracy and reliability, I'd want to consider including it)
ID: 1793755 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34249
Credit: 79,922,639
RAC: 80
Germany
Message 1793757 - Posted: 5 Jun 2016, 20:54:50 UTC - in response to Message 1793755.  

. But it's work which is not well suited to the design of, specifically, NVidia GPUs. They tend to take longer to achieve the same outcome as other computational resources.

On this point, I will say that statement isn't necessarily true. Depending on the class of Nvidia GPU and the application, they can and do better than some of their counter parts.

On Breakthrough Listen VLAR tasks? Do elaborate, please - we'd all like to achieve that happy outcome. (Under Windows, for the majority)


Its related to the pulse find kernels which can be tuned with OpenCL apps via -sbs and -period_iterations_num switch.

Raistmer explained it in another thread.


With each crime and every kindness we birth our future.
ID: 1793757 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1793758 - Posted: 5 Jun 2016, 20:57:21 UTC - in response to Message 1793757.  

Thanks Mike,

You are quicker than I am, lol...
ID: 1793758 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1793760 - Posted: 5 Jun 2016, 21:03:12 UTC - in response to Message 1793757.  

. But it's work which is not well suited to the design of, specifically, NVidia GPUs. They tend to take longer to achieve the same outcome as other computational resources.

On this point, I will say that statement isn't necessarily true. Depending on the class of Nvidia GPU and the application, they can and do better than some of their counter parts.

On Breakthrough Listen VLAR tasks? Do elaborate, please - we'd all like to achieve that happy outcome. (Under Windows, for the majority)


Its related to the pulse find kernels which can be tuned with OpenCL apps via -sbs and -period_iterations_num switch.

Raistmer explained it in another thread.

For the avoidance of doubt, are we talking about message 1793638?

I'll try and work it out in the morning - I ended up in a pub at the end of my walk :-)
ID: 1793760 · Report as offensive
Profile Mike Special Project $75 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 01
Posts: 34249
Credit: 79,922,639
RAC: 80
Germany
Message 1793761 - Posted: 5 Jun 2016, 21:08:25 UTC - in response to Message 1793760.  

. But it's work which is not well suited to the design of, specifically, NVidia GPUs. They tend to take longer to achieve the same outcome as other computational resources.

On this point, I will say that statement isn't necessarily true. Depending on the class of Nvidia GPU and the application, they can and do better than some of their counter parts.

On Breakthrough Listen VLAR tasks? Do elaborate, please - we'd all like to achieve that happy outcome. (Under Windows, for the majority)


Its related to the pulse find kernels which can be tuned with OpenCL apps via -sbs and -period_iterations_num switch.

Raistmer explained it in another thread.

For the avoidance of doubt, are we talking about message 1793638?

I'll try and work it out in the morning - I ended up in a pub at the end of my walk :-)


Yes, its one of them.


With each crime and every kindness we birth our future.
ID: 1793761 · Report as offensive
Al Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Avatar

Send message
Joined: 3 Apr 99
Posts: 1682
Credit: 477,343,364
RAC: 482
United States
Message 1793799 - Posted: 5 Jun 2016, 23:13:50 UTC - in response to Message 1793760.  


I'll try and work it out in the morning - I ended up in a pub at the end of my walk :-)

Good on ya! You certainly deserved it!

ID: 1793799 · Report as offensive
Profile William
Volunteer tester
Avatar

Send message
Joined: 14 Feb 13
Posts: 2037
Credit: 17,689,662
RAC: 0
Message 1793909 - Posted: 6 Jun 2016, 9:48:08 UTC

For the sake of DonS before his question gets buried in unrelated discussions:
[and partially just rephrasing what Zalster posted]

You can distinguish between Arecibo and breakthrough listen data [tasks] by the name of the tasks.
If there's 'guppi' in the name, you are crunching breakthrough listen, if not you're looking at Arecibo data (what we've been doing for the past 17 years and still keep plodding through).
Guppi stands for 'Greenbank Ultimate Pulsar Processing Instrument'. The Greenbank telescope does mainly close observations of suns that may have planets around them (therefore more likely to have aliens). Arecibo depends on what the researcher is looking at - we only piggyback. That data may be sweeping the sky.

VLAR stands for 'very low angle range'
Imagine a telescope pointing at a star for a night. as the star moves across the sky (or rather the earth moves...) the telescope needs to move with it. the difference between the position at the start and the end of the data makes the angle range (AR). Arecibo data can come in mid-AR (the bulk) with ARs in the region of 0.4 [if you look at a finished task e.g. this one you can see the AR printed near the top.]
There are also VHAR - very high AR, 2 and above and VLAR with an AR below 0.12 [it's derived from the beam width these days so may differ for guppi].

As you can see in the task linked the AR for guppi is usually very very small - as I said they tend to point at a star, so you have minimal movement.


I think that covered the questions.

Now, to the discussion that ensued.

When Greenbank data was first discussed by Eric, the assumption was, this would be very big tasks possibly lasting weeks if not months.
When we finally got guppis it turned out they have been parceled into much the same size as we are used to from Arecibo.

Now, VLAR on NV. Back in 2009 it was found that the signal search algorithms as NVidia ported them to CUDA aren't really suited for the data structure of VLARs. Basically it takes an disproportional amount of processing power. The GPUs we had back then didn't cope very well, for some people slowing down the display as to make the host totally unresponsive. As GPUs got bigger, the problem lessened.
Still the complaints - and the ensuing workarounds [like automated task abort]- led to the project introducing checks on the AR, marking VLAR as such and not distributing them to NV GPUs.

Now, the picture has shifted. Besides having much more powerful GPUs we also have OpenCL applications. And the SoG app is far better suited to VLARs then the current CUDA app. [AFAIK by processing some stuff on the CPU, but you'd have to ask the developer i.e. Raistmer] The downside of the OpenCL apps on NV is that they basically block a CPU core too.
Whether or not you are prepared to sacrifice some CPU to better GPU is up to you. We are still waiting for better preferences on that issue, to better accommodate what and how people like to run. Since that is BOINC territory, it isn't exactly easy to achieve.


So, guys and gals, if you have anything to add to enlighten DonS or more detail to provide, go ahead.
If, however, you wish to discuss anything from the size of guppis to the suitability of apps please open a dedicated thread. This is a newbie question and he deserves a simple and precise answer not a technical discussion/argument he doesn't understand atm.
A person who won't read has no advantage over one who can't read. (Mark Twain)
ID: 1793909 · Report as offensive
Profile Mr. Kevvy Crowdfunding Project Donor*Special Project $250 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 3776
Credit: 1,114,826,392
RAC: 3,319
Canada
Message 1793910 - Posted: 6 Jun 2016, 9:58:49 UTC - in response to Message 1793909.  
Last modified: 6 Jun 2016, 10:01:24 UTC

The Greenbank telescope does mainly close observations of suns that may have planets around them (therefore more likely to have aliens).


That was the initial idea (to observe Kepler objects) but doesn't seem to have carried through to the Breakthrough Listen observation plan:

1. Green Bank Radio Telescope

World’s deepest searches for artificial signals in five key samples (Northern Hemisphere):

All 43 stars within 5 parsecs, at 1-15 GHz. First-ever complete SETI survey within 5 parsecs. Sensitive to “Earth-leakage” levels of radio transmission.
1000 stars of all spectral-types (OBAFGKM). Within 50 parsecs. 1-15 GHz.
One Million Nearby Stars. In 2016, first 5,000 stars; 1 minute exposure (1-15 GHz)
Centers of 100 nearby galaxies: spirals, ellipticals, dwarfs, irregulars (1-15 GHz)
Exotic Stars: 20 White Dwarfs, 20 Neutron stars, 20 black holes

2. Parkes Radio Telescope

World’s deepest searches for artificial signals in six key samples (Southern Hemisphere):

All 43 stars (at south declinations) within 5 parsecs, at 1-15 GHz. First-ever complete SETI survey within 5 parsecs. Sensitive to “Earth-leakage” levels of radio transmission.
1000 stars (south) of all spectral-types (OBAFGKM). Within 50 parsecs. (1-4 GHz)
One Million Nearby Stars (south). In 2016-2017, first 5,000 stars; 1 minute exposure (1-4 GHz)
Galactic plane and Center (1-4 GHz)
Centers of 100 nearby galaxies (south declinations): spirals, ellipticals, dwarfs, irregulars (1-4 GHz)
Exotica: 20 White Dwarfs, 20 Neutron stars, 20 black holes

ID: 1793910 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1793911 - Posted: 6 Jun 2016, 10:07:41 UTC - in response to Message 1793909.  

by processing some stuff on the CPU

To inhibit new urban myth from developing: SoG uses less CPU in processing than non-SoG. And both had no increases in share of CPU involvement from first development of ATi MultiBeam app.
To make picture precise and uncover used approach: first PoT processed on CPU entirely (cause first PoT unavoidably will update best signals for obvious reason). Then CPU used to re-assure signal existance that both makes app more stable to false positives (if GPU thinks signal exists but CPU disagrees result will be corrected w/o going to inconclusives) and allows to simplify corresponding kernels.
But improvement in Pulse processing, especially on low FFT sizes (that constitute VLAR) achieved not because of that approach mostly but because of additional parallelization and more even load of GPU compute resources.
More additional parallelizations still possible.
ID: 1793911 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1793912 - Posted: 6 Jun 2016, 10:12:55 UTC - in response to Message 1793910.  


That was the initial idea (to observe Kepler objects) but doesn't seem to have carried through to the Breakthrough Listen observation plan

Hard to make any conclusion w/o list of Kepler objects.
If star reside in 5 parsecs from Sun it doesn't make it "non-Kepler" nor ensure it has planets.
In short - both lists most probably intersect.
ID: 1793912 · Report as offensive
Profile Mr. Kevvy Crowdfunding Project Donor*Special Project $250 donor
Volunteer moderator
Volunteer tester
Avatar

Send message
Joined: 15 May 99
Posts: 3776
Credit: 1,114,826,392
RAC: 3,319
Canada
Message 1793917 - Posted: 6 Jun 2016, 10:30:12 UTC - in response to Message 1793912.  
Last modified: 6 Jun 2016, 10:31:52 UTC

In short - both lists most probably intersect.


Only slightly... there are (edit: were prior to the May announcement) 460 stars with Kepler object planets, the furthest being 8kLy away, but there are 600M stars within 5kLy from earth, so less than one millionth contain Kepler planets.

Thus, if not looking at confirmed planetary systems we may as well stay with random searches as the search has been essentially randomized.
ID: 1793917 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1793930 - Posted: 6 Jun 2016, 11:29:16 UTC - in response to Message 1793917.  

Still not clear how these candidates will be selected:

1000 stars of all spectral-types (OBAFGKM). Within 50 parsecs. 1-15 GHz.
One Million Nearby Stars. In 2016, first 5,000 stars; 1 minute exposure (1-15 GHz)

Obviously total number of stars at those distances much greater.

Let's hope being listed as having planetary system will be lead reason for selection.
ID: 1793930 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14644
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1793940 - Posted: 6 Jun 2016, 11:43:15 UTC - in response to Message 1793917.  
Last modified: 6 Jun 2016, 11:46:03 UTC

... there are (edit: were prior to the May announcement) 460 stars with Kepler object planets ...

ROFL: "The 2016 May 10 confirmation of 1,284 more Kepler exoplanets has exceeded the technical abilities for this webpage to display."

I make that 1644 distinct host stars, at distances between 35.7 and 2700 parsecs - 116 to 8806 light years - from earth (but including many at unknown distances).

Edit: sources
http://kepler.nasa.gov/Mission/discoveries/
http://exoplanetarchive.ipac.caltech.edu/cgi-bin/TblView/nph-tblView?app=ExoTbls&config=planets
ID: 1793940 · Report as offensive
DonS

Send message
Joined: 15 May 16
Posts: 7
Credit: 1,540,164
RAC: 2
Canada
Message 1794584 - Posted: 9 Jun 2016, 0:42:52 UTC

> vlar

So I know what vlar means now but what's the significance of it?

> guppi

Great I'll be watching for these types of files.

This is great info:

blc3_2bit_guppi_57451_19304_HIP62472_0003
blc3: Breakthrough Listen
2bit: bits per real sample (4 bit per complex sample)
guppi: File format (Greenbank Ultimate Pulsar Processing Instrument)
57451: Modified Julian Date
19304: Seconds past midnight
HIP62472: Object Name
0003: Observation sequence number
ID: 1794584 · Report as offensive

Message boards : Number crunching : vlar data and breakthrough listen


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.