Posts by Mr. Kevvy


log in
1) Message boards : SETI@home Science : Scientific Value of Processing (Message 1763324)
Posted 4 days ago by Profile Mr. KevvyProject donor
If you go into Account and then SETI@Home Preferences you can edit them and set them to:

SETI@home v7: no
AstroPulse v7: yes
SETI@home v8: no

And your computer will only receive AstroPulse work. Caveat: there is not always a run of AstroPulse work with any available, and because it "pays" more credit for the same amount of processing, a lot of people have set their machines to AstroPulse-only and when there is any it is quickly assigned.
2) Message boards : SETI@home Science : Scientific Value of Processing (Message 1763294)
Posted 4 days ago by Profile Mr. KevvyProject donor
By the way i tried clicking the link you provided but it was blocked on my end.


It's using a self-signed SSL certificate, so this needs to be allowed in your browser.

If humans are literally the ONLY intelligent life in the entire universe (which I don't believe, but lets just assume so for question's sake), what other ares could these "triplets" we find help?


Triplets could(?) discover pulsars although the spike duration appears to be too short. But there is an entire area of SETI@Home designed to discover astrophysical phenomena like this: Astropulse. So even if there's nothing intelligent out there, it is still valid and its data could still produce useful discoveries.
3) Message boards : SETI@home Science : Scientific Value of Processing (Message 1763195)
Posted 4 days ago by Profile Mr. KevvyProject donor
SETI@Home's Dr. Eric Korpela and Dan Wertheimer completed a project to map neutral hydrogen distribution in the Milky Way using data on the SETI@Home tapes. I could be wrong but I don't think that this was based on our processing, however it was still using the same data that wouldn't be there otherwise.

Edit: If SETI@Home shut down today, I don't doubt the data would be handed over to other group(s) who would perform a far more comprehensive correlation and re-observation than has been done until present. Don't get me started on NTPCkr again... ;^)
4) Message boards : Number crunching : Running a test: An ARM with a cracked screen. (Message 1762302)
Posted 7 days ago by Profile Mr. KevvyProject donor
I have one like that too: 7518526.

Its SIM slot failed making it useless as a phone, so I bought it from my missus as it can still connect to WiFi. I put it behind the bedroom TV and forgot about it. I eventually found that it had fallen behind the dresser and been crushed, probably when the dresser was moved. But I dusted it off and got it working again, and found it took a licking and keeps on ticking; I'm amazed that the touchscreen still works under all the crackle.

I hope if I ever discover anything that it's this device that does it... there's a lesson in there about not wasting or overlooking potential.

Here it is crunching away as of a couple minutes ago:

5) Message boards : Number crunching : Proposal/idea for a near-real-time, low-resource, inexpensive NTPCkr (Message 1762076)
Posted 8 days ago by Profile Mr. KevvyProject donor
The type of queries that NTPCkr needs to run.. needs access to the entire database..for one single query, that could take hours or even days to finish running.


It doesn't seem that way... again it takes 5,000 CPU days to correlate 2,113 actual days' results, so 2.64 or more cores could do it in real time. The queries must be returning smaller results and I expect that as much as possible of that database is in RAM thus the huge memory requirement. I would imagine that the sky map is "pixellized" so that each object has a hard-and-fast location that can be indexed for instantaeous retrieval, rather than everything having a long decimal RA + Dec. location that requires extensive searching to find nearby objects that may be matches... it would just be too slow.

This would be an ideal question for the SAH team to provide more info on...

Well I would be interested if and when it's decided.


I will definitely pass the word. :^)

Considering the cost of RAM, have things moved forward enough now that its reasonable to run an application like this from an SSD? I read an article earlier where 3 Samsung M.2 950 pro SSDs were run in RAID on an Asus board getting insane bandwidth of around 3 GB/s. That's about a 1/4 of the speed of DDR3 but at a much smaller fraction of the cost.


The original spec. for NTPCkr was for 512GB of RAM and 10TB of SSD, and as you noted since RAIDed SSD is so fast I'm wondering why both. Then again that spec. is for that NTPCkr to do it all.

I think Mr. Kevvy has a good plan, but then... what do I know?


Enough that I say thank you. :^)

Honestly, I think there is a general lack of urgency. You would think that someone would have just "floated a trial balloon" to see if the hardware was doable if they really wanted it.


I agree in that I think they are just too busy elsewhere and will be so for the immediate future with all of the changes happening. The "cloud" NTPCkr happened because an intern worked on it, but there is a substantial investment in development for a local one that I doubt can be fulfilled right now. Still, may as well float this idea if it can be done for a small fraction of the cost... may be an ideal project for the next intern or post-grad.
6) Message boards : Cafe SETI : Beet's give us a caption #63 (Message 1762066)
Posted 8 days ago by Profile Mr. KevvyProject donor
On reconsideration I think I will change my response to "Mirror, me roar..."

Missed it by that much. :^)
7) Message boards : Cafe SETI : Beet's give us a caption #63 (Message 1761982)
Posted 8 days ago by Profile Mr. KevvyProject donor


"Mirroar, mirroar..."
8) Message boards : Number crunching : SETI@home Version 8 for GPU (Message 1761939)
Posted 8 days ago by Profile Mr. KevvyProject donor
Here are the Lunatics installers. You'll want 0.44.
9) Message boards : Cafe SETI : Raccoon Update XXIII - All Are Welcome In The Critter Cafe (Message 1761898)
Posted 8 days ago by Profile Mr. KevvyProject donor
Raccoon tries to catch the subway at Spadina station


Mask Transit?

<<ducks>>
10) Message boards : Number crunching : Proposal/idea for a near-real-time, low-resource, inexpensive NTPCkr (Message 1761584)
Posted 9 days ago by Profile Mr. KevvyProject donor
If you ever wanted to cringe at a price tag... go look at server boards that can handle 512-1024 GB of RAM. THEN...go look up how much it would cost to get the ECC Registered RAM to get at least 512GB of it (in the module size needed for the board.. probably 8 or 16GB modules, times however many). That's painful.


Both of those were in the price spec. at the bottom. As Chris noted the RAM would be in 16x32GB sticks. Here is an example dual-Xeon board rackmount for the quoted cost. It actually has 1TB of max. RAM capacity in 16 DIMM slots whereas the one I had seen originally had 512GB, and also has 10GBit Ethernet.

I guess someone from the SAH team would be the only ones who could say one way or another if the capped "catch-up" rate of 64 queries/sec. to the science server would cause an issue, but as indicated the BOINC database can be seen to get 1,200+ queries/sec. on the server status page (when I looked at it right now it was 1,360.)
11) Message boards : Cafe SETI : The joke thread Part 4. (Message 1761422)
Posted 10 days ago by Profile Mr. KevvyProject donor
I have this strange sense of deja vu...
12) Message boards : Number crunching : Proposal/idea for a near-real-time, low-resource, inexpensive NTPCkr (Message 1761389)
Posted 10 days ago by Profile Mr. KevvyProject donor
Too late. Already on the way.


I doubt that is going to be used long-term as it would be prohibitively expensive (I had a gander at Amazon's rates recently.) Also, as indicated, even with a run of a few weeks it's anything but "Near Time" ie you can click on your recently completed work after a few minutes to see how it matches up with others' in NTPCkr.
13) Message boards : Number crunching : Proposal/idea for a near-real-time, low-resource, inexpensive NTPCkr (Message 1761382)
Posted 10 days ago by Profile Mr. KevvyProject donor
This thread is intended as a discussion to bat ideas around and get feedback. I've posted it in Number Crunching, rather than SETI@Home Science where the other NTPCkr thread is, because this is a computing rather than astrophysical topic, and this forum seems to be where the most technically savvy folks with the greatest knowledge of SETI@Home's infrastructure can be found.

I hope that it proves fruitful, as I think that a working NTPCkr or equivalent is very important to the future of this project. If is seems to pass muster, or if any flaws found with it seem to be reasonably solvable, it could certainly be proposed to the SETI@Home team for consideration.

Note: Please do not reply quoting this entire post unless you are inserting commentary throughout! I will become rather annoyed. :^p

First, we need some preliminaries, of course, for new volunteers:

What is NTPCkr?
NTPCkr is Near-Time Persistency Checker. Simply, it's a server/computer that takes our crunched results and quickly matches them up (correlates them) with the billions of other results on file to see if multiple interesting signals have been found at one point in the sky. Ideally, we also could access it through our browsers and see where work units we've crunched are located on a sky map, and whether other interesting results have been found there.

Why is NTPCkr so important to SETI@Home?
NTPCkr's importance is two-fold:

1) Science: By virtue of being near-real-time, NTPCkr offers to project scientists much faster responsiveness should a transient signal be detected, so that it can be investigated further in a minimal amount of time before it is gone. Responsiveness is now easier and thus this has become a priority due to SETI@Home's access to the Green Bank Telescope, which can be pointed at interesting locations, in contrast to Arecibo's constraint of only listening at whatever location the telescope happens to be "pointing" to for other research.

2) Project participation and long-term success: One of the statistics I've seen the project team use is the unfortunate 8% retain rate of SETI@Home volunteers. This means that 92% of everyone who ever did any project work is now gone, or at least isn't doing any more work. Over the years I've seen several BOINC and other distributed computing projects fail or stagger because of lack of participant enthusiasm, and the biggest contributing factor is always that participants don't think that their work is valuable to the project and being used. With the flood of new Breakthrough Listen data from Green Bank and possibly elsewhere requiring maximum participation, providing a captivating way of showing volunteers that their results are being analyzed and that they possibly contain interesting data would not only prevent them from leaving, but may also convince those who have left to return.

What's passing for NTPCkr now?
Currently, NTPCkr requires the entire science database to be exported and then the job run on a supercomputer or cloud computing facility. This takes weeks, so it should be just called "PCkr" as it's anything but Near-Time. (Perhaps Far-Time, which would make it FTPCkr, so would ideally be run on the IBM computing cluster in Poughkeepsie... and props to anyone who gets the reference.) The cost estimate for a NTPCkr owned by and local to the SETI@Home facility to replace this was initially $30-40K. I hope to demonstrate that it can be done for far less cost.

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Synopsis
Doing a complete run of the entire SETI@Home science database to find persistent signals more than once is unnecessary and inefficient. Instead, only one initial run is required, because, as the result is a database itself, further runs can build on it incrementally (much like backing up a hard disk only copying new/changed files since the last backup) only adding new results since the last run, rather than starting from scratch. These further runs can be done on a much smaller/cheaper computer at the SETI@Home facility than initially estimated, and eventually this process would catch up to the newest results in the science database, so would fulfill the "Near Time" requirement of correlating results almost as soon as they are received.

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Methodology


paddym is the SETI@Home science server for the SETI@Home science database of MB (Multi-Beam) results which NTPCkr would be looking at. While I don't know the exact number of results it has, we know that the total credit in SETI@Home's results is about 3.2 x 1011. This grows by about 1.5 x 108 credits per day. Thus assuming the MB/AP credit ratio hasn't changed too drastically averaged over time, the science database contains 3.2 x 1011/1.5 x 108 ≈ 2,130 days worth of work. We can thus estimate the number of results paddym holds, knowing that about 92,000 results are returned per hour (which means 25 writes/second to the science database.) This works out to (2.2 x 106 per day x 2,133 days' worth) / 2 results per work unit ~ 2.3 x 109 results in total.



The current time/date is noted as the checkpoint time for use later, and the SETI@Home science database is exported and sent to a cloud or supercomputing facility to run the NTPCkr job. Note that in this time the SETI@Home science database has grown as it has been collecting results continuously (I show new results as halftone at the bottom.) This initial run needs to be done elsewhere as it is computationally unfeasible to perform with SETI@Home's in-house computing resources (it also can't be distributed to volunteers as each would need enough of a percentage of the science database in local storage that there is not enough network bandwidth to distribute it.) The actual time requirement for the job to run is 35 days. The processing requirement for this job is about 5,000 CPU days... ie a quad-core desktop would take about 5,000/4 = 1,250 days — 3 years and 3 months — to complete it.



The resulting NTPCkr database from the run is returned to SETI@Home and installed in the NTPCkr computer. With the 35 day run time, plus an estimated week of transport and setup each way, 49 days have elapsed, with the SETI@Home science database growing continuously during this time.



The cloud or supercomputing facility is no longer required. The SETI@Home science database is then copied to NTPCkr so that the complete database is available locally as otherwise the correlations would be too slow.



NTPCkr is now started up and runs the same correlation job that was run on the cloud or supercomputing facility, correlating its local replica copy of the science database. The critical difference is that NTPCkr only runs the job on results that arrived between the checkpoint time/date noted above and the current time/date.



NTPCkr has now finished additions to its result database. However, it hasn't caught up with the science database yet, because in the time it took to do this plenty of new results arrived.



NTPCkr now sets the checkpoint time as the previous current time, and runs the job again. However from this point forward copying the entire science database over is no longer required. Instead, NTPCkr directly queries paddym for the new results since checkpoint and adds them to its local copy of the science database, then correlates them into the NTPCkr database.
This process repeats with NTPCkr converging on the state of the science database with each iteration. The time to convergence can be easily determined... no calculus required!
We know in the initial run that 5,000 CPU days correlated 2,133 days' worth of science database results, so results accumulate at the rate of 5,000/2,133 = 2.34 CPU days per day. Thus, a NTPCkr with only 3 or more CPU cores could keep with the science database results as they arrive.
We need to allow for NTPCkr outages requiring catch-up to the last checkpoint, sub-optimal utilization and other issues, and that the NTPCkr computer is also expected to serve out local and web queries. I would therefore expect a minimum of a six-core Xeon.
There is initially a 49-day offset between the science database and the NTPCkr database. This is 49 x 2.34 = 114.66 CPU days of computation.
Every day the science database adds 2.34 CPU days of results to correlate, and NTPCkr's six-core Xeon reduces this by 6 CPU days, a net reduction of 6-2.34 = 3.66 days' offset.
NTPCkr would thus catch up to the science database in 114.66/3.66 ≈ 31.3 days.
During this time it can be limited to querying the science database (6 / 2.34) x 25 = 64 queries/sec. (The 25 is the number of new results/second written to the science database.) This should not present an issue, considering the BOINC database receives 20x as many queries/sec. with fewer resources.



After the 31.3 days, NTPCkr would then be near-real-time, checkpointing perhaps once per minute or whatever is determined to be a proper interval. At this point, it could be linked with muarae1 to serve web queries from anyone interested in checking results for correlation. if NTPCkr crashed or was otherwise cut off from the science database, it would catch up in 2.34 / 6 = 0.39 of the time offset, so for example if it was down for a day it would catch up in just over nine hours.

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Proposed Costing/parts list

The initial NTPCkr spec. indicated 10TB of SSD and 512GB RAM. Given the differing methodology, with only 64 queries/sec. to the science database it doesn't seem that SSDs would be required. (This would also be 180 MBits/sec. even if complete work units were being extracted, which they aren't, so the built-in GBit networking on the mainboard would be more than sufficient.) A single six-core CPU has been demonstrated to be enough, but eight-cores are only a couple of hundred dollars more so would be worth the difference. As well to facilitate further upgrades and mitigate possible lack of resources, a dual-CPU board would be ideal. Not knowing how the requirements would change, the original 512GB RAM spec. should also be kept.

SuperMicro rackmount with dual-Xeon LGA2011 mainboard and 950W PSU: $1,850
Xeon eight-core CPU: $700
512GB memory (32GB DDR4 ECC DIMM server memory @ $300 x 16) = $4,800 (well over half the cost... it appears that the price-fixing convictions haven't stopped RAM from being way too expensive!)
12TB hard disk (3 TB NAS SATA III HD @ $120 x 5 [one as a parity drive]) = $600

Total: $7,950. (This is between one-fifth and one-quarter the cost of full-correlation NTPCkr estimated at $30-40K. It also matches the actual cost of equivalent SETI@Home servers that I know of.)

Time estimate for raising this amount: One week with matching fundraising.
14) Message boards : Number crunching : Its breaking my Heart but I may have to bow out of SETI@Home .. (Message 1760803)
Posted 12 days ago by Profile Mr. KevvyProject donor
“It is the greatest of mistakes to do nothing because you can only do little. Do what you can.”
--- Sydney Smith
15) Message boards : SETI@home Science : First Signs of the NTPCkr (Message 1760743)
Posted 12 days ago by Profile Mr. KevvyProject donor
I have no doubt that Matt could easily lay out the specs for such a m/c, but where would the money come from to fund its purchase?


Already made it clear to the team several times that I'd be more than happy to raise the funds via matching fundraiser or equivalent as per Matt's spec. and cost estimate, so that isn't an issue. With the v8 transition, building instruments at Green Bank, integrating Breakthrough Listen and all of their other duties, I think that they're just too busy right now.

So, I will remind them from time to time.... gently. :^)

I'm working on a writeup of an idea for NTPCkr that I will shortly post in Number Crunching, perhaps this weekend...
16) Message boards : Cafe SETI : R.I.P. Abe Vigoda (for real this time) (Message 1759474)
Posted 16 days ago by Profile Mr. KevvyProject donor
Fish sleeps with the Tessios.

Abe Vigoda, the sad-faced actor who emerged from a workmanlike stage career to find belated fame in the 1970s as the earnest mobster Tessio in “The Godfather” and the dyspeptic Detective Phil Fish on the hit sitcom “Barney Miller,” died on Tuesday morning in Woodland Park, N.J. He was 94, having outlived by about 34 years an erroneous report of his death that made him a cult figure.

His daughter Carol Vigoda Fuchs, told The Associated Press that Mr. Vigoda had died in his sleep at her home.


I am working through the complete Barney Miller and just got through all of his episodes only days ago, and wondered when he would be gone for real. It's amazing that he was playing a retiree over forty years ago yet was still around. So long, Abe, for the last time.
17) Message boards : Number crunching : Abandoned Tasks (Message 1758628)
Posted 19 days ago by Profile Mr. KevvyProject donor
You have some marked 'Abandoned' I see.


Same thing here across several machines at the same time.
18) Message boards : Number crunching : AVG detecting IDP.AREW.Generic (Message 1758475)
Posted 20 days ago by Profile Mr. KevvyProject donor
Anybody else seeing that?


Yes, or similar false positive.

Edit: Options>Advanced Settings>Exceptions, Add Exception, select Folder and add the BOINC program folder.
19) Message boards : Number crunching : Lunatics Windows Installer v0.44 - new release for v8 (required upgrade) (Message 1758433)
Posted 20 days ago by Profile Mr. KevvyProject donor
After installing the client reports missing files, but I am crunching away normally.

22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/setiathome_8.00_windows_intelx86.exe not found 22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/setigraphics_8.00_windows_intelx86.exe not found 22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/setiathome_8.00_windows_intelx86__cuda50.exe not found 22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/setiathome_8.00_windows_intelx86__cuda42.exe not found 22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/cudart32_42_9.dll not found 22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/cufft32_42_9.dll not found 22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/setiathome_7.00_windows_intelx86__cuda42.exe not found 22/01/2016 7:30:47 PM | SETI@home | File projects/setiathome.berkeley.edu/setiathome_7.00_windows_intelx86__cuda50.exe not found


In order to clear out any old junk, I detached/reattached to clear the folder when v8 GPU client was released and ran stock with no app_info.xml until installing Lunatics 0.44, so nothing I did (I think!) But it seems to be working.

Edit: After quitting/restarting the client (which I am doing often tweaking app_info) seems to have gone away.
20) Message boards : Number crunching : Lunatics Windows Installer v0.44 - new release for v8 (required upgrade) (Message 1758166)
Posted 20 days ago by Profile Mr. KevvyProject donor
And by lunchtime Friday. Very well done to those that had a hand in it, Richard, William, and others.


Absolutely... I thought I'd at least have to wait a few days. I guess it helps that the unofficial developers have become the official developers. :^) Excellent job and thanks!


Next 20

Copyright © 2016 University of California