Message boards :
Technical News :
Moving on... (Apr 08 2013)
Message board moderation
Previous · 1 · 2 · 3 · Next
Author | Message |
---|---|
Bernie Vine Send message Joined: 26 May 99 Posts: 9957 Credit: 103,452,613 RAC: 328 |
A "NTPCKR" dedicated server, is the way to go. As discussed a while ago there is an "offline" nitpicker that it being used, just not talked about for some reason. |
HAL9000 Send message Joined: 11 Sep 99 Posts: 6534 Credit: 196,805,888 RAC: 57 |
Are you saying that you are not able to acquire enough data from Arecibo to continue to distribute work at the improved rates? One you guys get data from Arecibo. Do you have to go to the colo to plug in drives or do you just copy the raw data the server(s) over the remote admin interface? SETI@home classic workunits: 93,865 CPU time: 863,447 hours Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[ |
Gary Charpentier Send message Joined: 25 Dec 00 Posts: 30969 Credit: 53,134,872 RAC: 32 |
The public ntpckr IIRC was to be used with http://boinc.berkeley.edu/trac/wiki/BossaIntro. The public one, if it runs, will generate waterfall plots. Humans eyeballs need to look at the waterfall plots to figure out if it is BS or ET. Seti doesn't have the $$$ to do the training to teach you how the tell the difference yet or to implement it real time for 10,000 eyeballs to look at each one in parallel. However that doesn't mean that the data isn't being looked at from time to time. |
Bernie Vine Send message Joined: 26 May 99 Posts: 9957 Credit: 103,452,613 RAC: 328 |
I see, can you give us mere mortals a link to the thread that contains this information? |
Josef W. Segur Send message Joined: 30 Oct 99 Posts: 4504 Credit: 1,414,761 RAC: 0 |
Hey, weren't we supposed to start seeing some data from somewhere else, Green Bank I think? Whatever happened with that? See Andrew Siemion's posts in the New paper on SETI observations of the Kepler field thread. They still intend that first 40 Terabytes of Green Bank data to be processed here, but haven't had time to get splitters coded and tested. With the Arecibo data more or less fully occupying the old download capacity, there also wasn't much motivation to add that other data source. But one of the GPUUG donation drive targets was getting enough drives and shipping cases to bring GBT data to Berkeley, which I took as an indication they intend to record more data there. Joe |
BassieXp Send message Joined: 5 Jun 05 Posts: 14 Credit: 1,408,518 RAC: 0 |
I've recently watched the 10yr anniversary video's again and from what i could tell. The way they do it now is that there's a machine, server, desktop or something which run's intermittent , where the nitpicker is a piece of software which should be able to run the results as they are comming in. For those who are interessted in the video i believe is was in Eric Korpela's presentation. (just looked it up it's around 53m:45s). Just on the subject the presentation from Jeff Cobb and Matt Lebofsky is also verry interesting as it's about the backend in 2009. http://setiathome.berkeley.edu/sah_10yr_anniversary.php |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
Are you saying that you are not able to acquire enough data from Arecibo to continue to distribute work at the improved rates? Given the amount of inbound traffic shown on the Cricket graphs from time to time, some of us are assuming it is due to data being transferred to the colo servers from the lab. "Time is simply the mechanism that keeps everything from happening all at once." |
Cosmic_Ocean Send message Joined: 23 Dec 00 Posts: 3027 Credit: 13,516,867 RAC: 13 |
I've recently watched the 10yr anniversary video's again and from what i could tell. Pretty much. But first it has to get through 13 years of back-log before it can analyze in real-time as more results come in. In the past, the reason it hasn't run in a while is because either the dedicated machine for it got re-purposed to replace a failing/dead machine or two or the database server didn't have enough disk I/O or spare memory capacity to handle the extra load.. or both simultaneously. Once the database gets tweaked some more to be able to handle some more additional heavy load, and there is a dedicated machine to run it, I'm sure it will happen. Though just as a thought, could get a dedicated machine for it, load it with CPUs and RAM, and make an offline database from one of the weekly back-ups and let it chew on that locally for a couple months, and then when it gets caught up, let it catch up on the main database for a week or so, and then once it is current, it can do real-time with ease. Linux laptop: record uptime: 1511d 20h 19m (ended due to the power brick giving-up) |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
I've recently watched the 10yr anniversary video's again and from what i could tell. Another thought that I believe has been bandied about before, would be to farm out the ntpckr process to us crunchers as a separate set of tasks, such as MB and AP are. Could have a choice of MB, AP, or NP...... I think the Seti processing collective could munch through the ntpckr backlog in rather short order. This would, of course, require a lot of work to write the programming to split, distribute, and process the data on our remote hosts. "Time is simply the mechanism that keeps everything from happening all at once." |
Highlander Send message Joined: 5 Oct 99 Posts: 167 Credit: 37,987,668 RAC: 16 |
It's a pity that ALMA records data at frequency range from 30 to 950 GHz. In theory this would be a fantastic data-source (data are public available). But vy nice for "day-dreaming". - Performance is not a simple linear function of the number of CPUs you throw at the problem. - |
kittyman Send message Joined: 9 Jul 00 Posts: 51477 Credit: 1,018,363,574 RAC: 1,004 |
It's a pity that ALMA records data at frequency range from 30 to 950 GHz. In theory this would be a fantastic data-source (data are public available). But vy nice for "day-dreaming". Well, I have no doubt that Eric will make other arrangements should the data being recorded at Arecibo be insufficient to further his search, and ours. "Time is simply the mechanism that keeps everything from happening all at once." |
contalis Send message Joined: 5 Dec 01 Posts: 28 Credit: 27,237,533 RAC: 73 |
Another thought that I believe has been bandied about before, would be to farm out the ntpckr process to us crunchers as a separate set of tasks, such as MB and AP are. My understanding is that the ntpckr needs to be processed locally, though IDK if that's speculation by others or from official sources. |
contalis Send message Joined: 5 Dec 01 Posts: 28 Credit: 27,237,533 RAC: 73 |
The public one, if it runs, will generate waterfall plots. Humans eyeballs need to look at the waterfall plots to figure out if it is BS or ET. Seti doesn't have the $$$ to do the training to teach you how the tell the difference yet or to implement it real time for 10,000 eyeballs to look at each one in parallel isn't this what Seti Live is doing: http://setilive.org/ |
BMaytum Send message Joined: 3 Apr 99 Posts: 104 Credit: 4,382,041 RAC: 2 |
The public one, if it runs, will generate waterfall plots. Humans eyeballs need to look at the waterfall plots to figure out if it is BS or ET. ... I'd guess that's what Seti Live is/was attempting to do. Admittedly I didn't spend much time there or at the(ir?) related http://setiquest.org/join-the-quest/software/readme website. Initially I thought I might grab SetiQuest's open source SonATA "waterfall image" software to try on one of my (non-Ubuntu) linux PCs. But I chose not to after inferring from some blogs there that the initial multi-hundreds or thousands of eyeballs looking at waterfalls starting in 2011(?) had declined exponentially since then. I also wondered how they had blown through the $100,000 TED funding already.... Anyway, right after the recent successful S@H colocation, I too thought: "Great, now the SSL scientists can get on with NTPCKR so we longtime crunchers can see some candidates". But in reading the most recent posts in this thread, I'm beginning to wonder what's really going to happen to digesting the millions of S@H results, and when? Sabertooth Z77, i7-3770K@4.2GHz, GTX680, W8.1Pro x64 P5N32-E SLI, C2D E8400@3Ghz, GTX580, Win7SP1Pro x64 & PCLinuxOS2015 x64 |
wulf 21 Send message Joined: 18 Apr 09 Posts: 93 Credit: 26,337,213 RAC: 43 |
Another thought that I believe has been bandied about before, would be to farm out the ntpckr process to us crunchers as a separate set of tasks, such as MB and AP are. Exactly my thought. The reason why the raw data crunching is so perfectly fitted for distributed computing is that a lot of processing time is required for a fairly low amount of data. Your computer is running thousands of tests on the same 0.36 KB of data. (a lot of different frequencies/resolutions/chirp rates). Then just a handful of hits are sent back. Ntpckr on the other hand needs to do a fairly low amount of processing on a high amount of data. For a single result all that needs to be done is deciding which pixel(s) it applies to (a decision which a local server can do faster than sending it out to a user). Then, to start scoring, you need ALL the results that have ever been watched at this pixel. And all that needs to be done is calculating a score from the values that are in the results table, which again can be done really fast. And then again for the RFI checking, you need ALL the results that have been watched about the time, the result you want to check have been watched. And the simple test that has to be done is if there were a lot of results on the same frequency before and after the result (i.e. in different spots of the sky) or a lot of results at the same time but at other frequencys (i.e. the signal is broadband). Both is considered RFI. So, just 2 fast tests on a lot of data. Summing up, if you would do persistency checking or RFI checking as a distributed computing task, preparing the WUs for the users, scheduling them and sending them over the internet would take more time then just doing the checks locally. It's just not fitted tor distributed computing. |
Mary Send message Joined: 22 Mar 12 Posts: 1 Credit: 363,143 RAC: 0 |
Why did my screen turn red with green flags all over it about a week or so ago? |
David S Send message Joined: 4 Oct 99 Posts: 18352 Credit: 27,761,924 RAC: 12 |
Hey, Matt! We haven't heard from you in a couple months now. Is everything going so well you don't need to tell us anything? How about an update on some other stuff? You guys must be working on something now that the servers are more or less behaving themselves. Maybe you can tell us what's up with the nitpckrs and the other things at the bottom of the server page. David Sitting on my butt while others boldly go, Waiting for a message from a small furry creature from Alpha Centauri. |
Bernie Vine Send message Joined: 26 May 99 Posts: 9957 Credit: 103,452,613 RAC: 328 |
We haven't heard from you in a couple months now. Yes, I wonder why?? |
Alinator Send message Joined: 19 Apr 05 Posts: 4178 Credit: 4,647,982 RAC: 0 |
We haven't heard from you in a couple months now. SAH firefighting, other SSL duties to perform, music, a life..... Not necessarily in that order. :-D |
Matt Lebofsky Send message Joined: 1 Mar 99 Posts: 1444 Credit: 957,058 RAC: 0 |
I apologize for the lack of updates. I was out of the lab for a month, then out of town the past two weekends.. So I've fallen out of the habit. I'm around (mostly) this summer so I'll try to get a random update out at least this week and then more afterward... There's always *something* to report... - Matt -- BOINC/SETI@home network/web/science/development person -- "Any idiot can have a good idea. What is hard is to do it." - Jeanne-Claude |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.