Monitoring inconclusive GBT validations and harvesting data for testing

Message boards : Number crunching : Monitoring inconclusive GBT validations and harvesting data for testing
Message board moderation

To post messages, you must log in.

Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 36 · Next

AuthorMessage
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1825550 - Posted: 20 Oct 2016, 5:01:45 UTC - in response to Message 1825539.  

saved it for a gander. [Edit:] looks like whatever my weekend trouble was, isn't there now, and things are on the slow mend. Food for thought on how things can go wacky.

Maddening, isn't it, when things go sideways but there's no trail to follow to get to the root of the problem. The week before last, two of my machines, one Win7 and one Win Vista, had multiple BSOD crashes over two or three days. I think they might have tied into the run of AP tasks we had at that time. However, that all happened during the time the Replica DB was offline and I never got a chance to see the task detail, since they were all purged by the time the DB came back. So it goes.
ID: 1825550 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1825553 - Posted: 20 Oct 2016, 5:18:23 UTC - in response to Message 1825550.  

saved it for a gander. [Edit:] looks like whatever my weekend trouble was, isn't there now, and things are on the slow mend. Food for thought on how things can go wacky.

Maddening, isn't it, when things go sideways but there's no trail to follow to get to the root of the problem. The week before last, two of my machines, one Win7 and one Win Vista, had multiple BSOD crashes over two or three days. I think they might have tied into the run of AP tasks we had at that time. However, that all happened during the time the Replica DB was offline and I never got a chance to see the task detail, since they were all purged by the time the DB came back. So it goes.


Just for some extra masochism, I opened a modern best practices guide for C++ '11 standards, as part of preparation for x42 along with the build system is a heavy refactoring. The pending necessity for that relates to pluginising performance code, moving closer to thread safe practices, and better use of compiler features, all requiring significant cleanup. That part wont be a quick ór painless process ;)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1825553 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1825787 - Posted: 21 Oct 2016, 3:48:14 UTC

I see that Petri Special x41p_zi3k has now shown up in my Inconclusives list. I'm guessing that this is the latest version.

Workunit 2296123852 (blc5_2bit_guppi_57432_31271_HIP57866_0021.28970.0.17.26.204.vlar)
Task 5222702934 (S=5, A=0, P=25, T=0, G=0) x41p_zi3k, Cuda 8.00 special
Task 5222702935 (S=1, A=0, P=29, T=0, G=0) v8.19 (opencl_ati5_nocal) windows_intelx86
Task 5224114544 (S=5, A=0, P=25, T=0, G=0) v8.00 windows_intelx86

Since this is a -9 overflow, it has the usual sequencing issues. Setting that aside though, I just looked at the first 25 Pulses that both the x41p_zi3k and opencl_ati5_nocal app picked up. It appears that the peaks of 23 of those 25 line up pretty nicely, to 4 decimal places, but on the 14th and 25th ones they seem to disagree.

x41p_zi3k: Pulse: peak=9.912649, time=45.9, period=23.26, d_freq=1232805777.41, score=1.029, chirp=-35.007, fft_len=2k
opencl_ati5_nocal: Pulse: peak=9.968046, time=45.9, period=23.35, d_freq=1232805777.41, score=1.035, chirp=-35.007, fft_len=2k

x41p_zi3k: Pulse: peak=10.29642, time=45.9, period=25.32, d_freq=1232799131.5, score=1.066, chirp=-41.499, fft_len=2k
opencl_ati5_nocal: Pulse: peak=10.68464, time=45.9, period=25.41, d_freq=1232799131.5, score=1.106, chirp=-41.499, fft_len=2k

Is that significant enough to do some deeper digging? My host has the potential tiebreaker and will be running it as SoG r3528.
ID: 1825787 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 1825796 - Posted: 21 Oct 2016, 4:40:08 UTC - in response to Message 1825787.  

Is that significant enough to do some deeper digging? My host has the potential tiebreaker and will be running it as SoG r3528.


The peak differences on the first signal set are on the order of what was reduced going from v6 to v7, so quanitifably can account for in the realm of 10% inconslusives to pending, so quite significant. If possible, would be good to know which is closer in peaks to the defacto reference, which is Win32 Stock CPU. If not a 'bug'per se, Could just be a case of one, the other, or both trying to overoptimise the averages, inadvertently increasing error growth. The work reducing error growth on stock CPU was partially my own, therefore I have a good feel for that particular characteristic.

The second case looks a little more concerning, with the large difference implying one or another may have a logical flaw outright (but not different enough to imply it's a hardware or environmental glitch)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 1825796 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1825798 - Posted: 21 Oct 2016, 5:01:58 UTC - in response to Message 1825796.  

Okay, well, I figure my host will be running it in about 6 or 7 hours with SoG r3528, so perhaps that will provide some more info. I expect I'll sleep right through it. ;^)
ID: 1825798 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1825807 - Posted: 21 Oct 2016, 6:14:51 UTC - in response to Message 1825787.  


x41p_zi3k: Pulse: peak=9.912649, time=45.9, period=23.26, d_freq=1232805777.41, score=1.029, chirp=-35.007, fft_len=2k
opencl_ati5_nocal: Pulse: peak=9.968046, time=45.9, period=23.35, d_freq=1232805777.41, score=1.035, chirp=-35.007, fft_len=2k

x41p_zi3k: Pulse: peak=10.29642, time=45.9, period=25.32, d_freq=1232799131.5, score=1.066, chirp=-41.499, fft_len=2k
opencl_ati5_nocal: Pulse: peak=10.68464, time=45.9, period=25.41, d_freq=1232799131.5, score=1.106, chirp=-41.499, fft_len=2k

Is that significant enough to do some deeper digging? My host has the potential tiebreaker and will be running it as SoG r3528.

Yep, it's significant. Difference in periods means you stumble upon same bug as with that test case I tried to pursue before.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1825807 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1825897 - Posted: 21 Oct 2016, 15:54:58 UTC - in response to Message 1825807.  


x41p_zi3k: Pulse: peak=9.912649, time=45.9, period=23.26, d_freq=1232805777.41, score=1.029, chirp=-35.007, fft_len=2k
opencl_ati5_nocal: Pulse: peak=9.968046, time=45.9, period=23.35, d_freq=1232805777.41, score=1.035, chirp=-35.007, fft_len=2k

x41p_zi3k: Pulse: peak=10.29642, time=45.9, period=25.32, d_freq=1232799131.5, score=1.066, chirp=-41.499, fft_len=2k
opencl_ati5_nocal: Pulse: peak=10.68464, time=45.9, period=25.41, d_freq=1232799131.5, score=1.106, chirp=-41.499, fft_len=2k

Is that significant enough to do some deeper digging? My host has the potential tiebreaker and will be running it as SoG r3528.

Yep, it's significant. Difference in periods means you stumble upon same bug as with that test case I tried to pursue before.

A bug in the OpenCL apps or the Petri Special?

There's still no consensus on that WU. My host, on SoG r3528, didn't validate against any of the previous 3 hosts, so it'll be sent out to yet another cruncher. Mine returned 30 Pulses but no Spikes, although the first 29 of the Pulses appear to line up very nicely with the 29 that the opencl_ati5_nocal host returned.

Anyway, the WU is still out there for whoever wants to grab it for offline testing.
ID: 1825897 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1825906 - Posted: 21 Oct 2016, 16:09:02 UTC - in response to Message 1825897.  
Last modified: 21 Oct 2016, 16:17:32 UTC


A bug in the OpenCL apps or the Petri Special?

Petri Special. Would be good to reproduce it in offline run and provide to Petri for fixing.

I saved task and 4 available stderrs.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1825906 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1826041 - Posted: 22 Oct 2016, 3:27:18 UTC - in response to Message 1825897.  
Last modified: 22 Oct 2016, 3:41:10 UTC

There's still no consensus on that WU. My host, on SoG r3528, didn't validate against any of the previous 3 hosts, so it'll be sent out to yet another cruncher. Mine returned 30 Pulses but no Spikes, although the first 29 of the Pulses appear to line up very nicely with the 29 that the opencl_ati5_nocal host returned.

That WU, 2296123852, finally validated with the 5th host running a Linux CPU task. The reported signal counts, 5 Spikes and 25 Pulses, apparently matched up sufficiently with the stock Windows CPU app, which was awarded the canonical result. Everybody who came to the party got 60.49 credits, however, since it was a late stage overflow.

Now, this evening's new quadruple Inconclusive, another late stage overflow with 4 different results, is brought to us exclusively by SoG (SSE3xj Win32 Build 3528), 3 stock, one Anonymous Platform (mine), running on 4 different Nvidia GPUs.

Workunit 2298592042 (blc3_2bit_guppi_57451_21431_HIP63121_0009.12921.416.17.26.230.vlar)
Task 5227915685 (S=15, A=0, P=15, T=0, G=0) v8.19 (opencl_nvidia_SoG) windows_intelx86
Task 5227915686 (S=12, A=0, P=18, T=0, G=0) SSE3xj Win32 Build 3528
Task 5231314430 (S=16, A=0, P=14, T=0, G=0) v8.19 (opencl_nvidia_SoG) windows_intelx86
Task 5233097746 (S=13, A=0, P=17, T=0, G=0) v8.19 (opencl_nvidia_SoG) windows_intelx86

I think my host, the second one, is the only one using any command line parameters. So it would seem that just the default processing sequences within SoG for different GPUs are sufficient to cause inconsistencies that fail to pass validation not once, not twice, but three times.

Once again, it will take a 5th host to settle this one.

EDIT: Further complicating this one is that, for three of the four hosts, the Stderr appears to show additional processing after the signal counts were first written. This results in one additional Spike being detected and the signal counts being written a second time, now with a total of 31 signals.
ID: 1826041 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1826067 - Posted: 22 Oct 2016, 7:32:31 UTC - in response to Message 1826041.  
Last modified: 22 Oct 2016, 7:36:40 UTC

There's still no consensus on that WU. My host, on SoG r3528, didn't validate against any of the previous 3 hosts, so it'll be sent out to yet another cruncher. Mine returned 30 Pulses but no Spikes, although the first 29 of the Pulses appear to line up very nicely with the 29 that the opencl_ati5_nocal host returned.

That WU, 2296123852, finally validated with the 5th host running a Linux CPU task. The reported signal counts, 5 Spikes and 25 Pulses, apparently matched up sufficiently with the stock Windows CPU app, which was awarded the canonical result. Everybody who came to the party got 60.49 credits, however, since it was a late stage overflow.

The weakness of current validator.
It can't distinguish between right and wrong w/o additional knowledge human being has.
Pulses in the same chirp range should be identical in their periods. Difference in periods designates some bug in processing unless there are so many nearly identical by power pulses that noise in power estimation could shift period detection too.
That could be revealed by offline run with threshold-reporting build.


Now, this evening's new quadruple Inconclusive, another late stage overflow with 4 different results, is brought to us exclusively by SoG (SSE3xj Win32 Build 3528), 3 stock, one Anonymous Platform (mine), running on 4 different Nvidia GPUs.

Workunit 2298592042 (blc3_2bit_guppi_57451_21431_HIP63121_0009.12921.416.17.26.230.vlar)
Task 5227915685 (S=15, A=0, P=15, T=0, G=0) v8.19 (opencl_nvidia_SoG) windows_intelx86
Task 5227915686 (S=12, A=0, P=18, T=0, G=0) SSE3xj Win32 Build 3528
Task 5231314430 (S=16, A=0, P=14, T=0, G=0) v8.19 (opencl_nvidia_SoG) windows_intelx86
Task 5233097746 (S=13, A=0, P=17, T=0, G=0) v8.19 (opencl_nvidia_SoG) windows_intelx86

I think my host, the second one, is the only one using any command line parameters. So it would seem that just the default processing sequences within SoG for different GPUs are sufficient to cause inconsistencies that fail to pass validation not once, not twice, but three times.

Once again, it will take a 5th host to settle this one.

EDIT: Further complicating this one is that, for three of the four hosts, the Stderr appears to show additional processing after the signal counts were first written. This results in one additional Spike being detected and the signal counts being written a second time, now with a total of 31 signals.


Did you spot any differencies in common part of reported signals?


So it would seem that just the default processing sequences within SoG for different GPUs are sufficient to cause inconsistencies

Absolutely. More than that. In theory, 2 identical hosts with slightly different setup can do the same. Cause there is asynchronously reported and synchronously reported signals inside task. So, they asynchronous between each other. Differencies in the time of checkpoint for example can do that.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1826067 · Report as offensive
TBar
Volunteer tester

Send message
Joined: 22 May 99
Posts: 5204
Credit: 840,779,836
RAC: 2,768
United States
Message 1826075 - Posted: 22 Oct 2016, 8:57:12 UTC - in response to Message 1826041.  
Last modified: 22 Oct 2016, 9:19:16 UTC

I ran this WU on my CPU & CUDA 75 App and received;

Current WU: blc3_2bit_guppi_57451_21431_HIP63121_0009.12921.416.17.26.230.vlar.wu
---------------------------------------------------
Running default app with command : MBv8_8.05r3344_sse41_x86_64-apple-darwin
      932.74 real       929.17 user         1.68 sys
Elapsed Time: ………………………………… 933 seconds
---------------------------------------------------
Running app with command : setiathome_8.11_x86_64-apple-darwin__cuda75_mac -device 0
      232.32 real        22.41 user        19.16 sys
Elapsed Time : ……………………………… 232 seconds
Speed compared to default : 402 %
-----------------
Comparing results
Result      : Strongly similar,  Q= 99.32%
---------------------------------------------------
Done with blc3_2bit_guppi_57451_21431_HIP63121_0009.12921.416.17.26.230.vlar.wu.


This pretty much agrees with the 940M although the signals are out of order. The 940M is the one from that group with the Normal looking stderr,
Spike count:    16
Autocorr count: 0
Pulse count:    14
Triplet count:  0
Gaussian count: 0


The CPU results;
WU true angle range is :  0.006866
Spike: peak=25.07172, time=8.830, d_freq=1616892188.79, chirp=0, fft_len=128 
Pulse: peak=4.935277, time=45.82, period=11.3, d_freq=1616892188.79, score=1.036, chirp=0, fft_len=128 
Spike: peak=24.58112, time=8.830, d_freq=1616892194.83, chirp=0.68414, fft_len=128 
Pulse: peak=5.212198, time=45.82, period=11.3, d_freq=1616892220.13, score=1.094, chirp=0.68414, fft_len=128 
Spike: peak=24.98177, time=8.830, d_freq=1616892182.75, chirp=-0.68414, fft_len=128 
Pulse: peak=10.68985, time=45.82, period=26.49, d_freq=1616892251.54, score=1.003, chirp=1.3696, fft_len=128 
Spike: peak=24.30722, time=8.830, d_freq=1616892176.69, chirp=-1.3696, fft_len=128 
Spike: peak=24.18143, time=51.54, d_freq=1616885326.11, chirp=-1.4635, fft_len=128k
Spike: peak=24.28919, time=51.54, d_freq=1616885326.11, chirp=-1.4686, fft_len=128k
Pulse: peak=4.842633, time=45.82, period=10.41, d_freq=1616892282.88, score=1.019, chirp=2.0537, fft_len=128 
Spike: peak=25.59976, time=8.828, d_freq=1616892254.01, chirp=-2.7391, fft_len=64 
Pulse: peak=7.826777, time=45.82, period=15.63, d_freq=1616892152.7, score=1.106, chirp=-2.7391, fft_len=64 
Spike: peak=26.02031, time=8.828, d_freq=1616892135.64, chirp=4.1074, fft_len=64 
Pulse: peak=8.642861, time=45.82, period=19.7, d_freq=1616892287.56, score=1.014, chirp=4.1074, fft_len=64 
Spike: peak=29.18505, time=8.828, d_freq=1616892241.94, chirp=-4.1074, fft_len=64 
Pulse: peak=8.873488, time=45.82, period=15.63, d_freq=1616892090.01, score=1.254, chirp=-4.1074, fft_len=64 
Spike: peak=29.35182, time=8.828, d_freq=1616892147.73, chirp=5.4769, fft_len=64 
Pulse: peak=12.26473, time=45.82, period=26.95, d_freq=1616892350.31, score=1.124, chirp=5.4769, fft_len=64 
Spike: peak=32.18671, time=8.828, d_freq=1616892229.85, chirp=-5.4769, fft_len=64 
Pulse: peak=12.73880, time=45.82, period=29.87, d_freq=1616892027.26, score=1.164, chirp=-5.4769, fft_len=64 
Spike: peak=32.19514, time=8.828, d_freq=1616892159.82, chirp=6.8465, fft_len=64 
Pulse: peak=13.76811, time=45.82, period=25.83, d_freq=1616892413.06, score=1.264, chirp=6.8465, fft_len=64 
Spike: peak=34.41929, time=8.828, d_freq=1616892217.76, chirp=-6.8465, fft_len=64 
Pulse: peak=13.95272, time=45.82, period=24.11, d_freq=1616891964.52, score=1.284, chirp=-6.8465, fft_len=64 
Spike: peak=34.36495, time=8.828, d_freq=1616892171.91, chirp=8.216, fft_len=64 
Pulse: peak=13.51195, time=45.82, period=24.11, d_freq=1616892475.8, score=1.243, chirp=8.216, fft_len=64 
Spike: peak=35.75184, time=8.828, d_freq=1616892205.67, chirp=-8.216, fft_len=64 
Pulse: peak=14.20186, time=45.82, period=25.52, d_freq=1616891901.77, score=1.304, chirp=-8.216, fft_len=64 
Spike: peak=24.49516, time=8.830, d_freq=1616892177.97, chirp=8.9002, fft_len=128 
Pulse: peak=10.72781, time=45.82, period=26.67, d_freq=1616891870.4, score=1.006, chirp=-8.9002, fft_len=128 
SETI@Home Informational message -9 result_overflow

Best spike: peak=35.75184, time=8.828, d_freq=1616892205.67, chirp=-8.216, fft_len=64 
Best autocorr: peak=16.141, time=40.09, delay=4.7128, d_freq=1616889947.25, chirp=-0.15866, fft_len=128k
Best gaussian: peak=0, mean=0, ChiSq=0, time=-2.123e+11, d_freq=0,
	score=-12, null_hyp=0, chirp=0, fft_len=0 
Best pulse: peak=14.20186, time=45.82, period=25.52, d_freq=1616891901.77, score=1.304, chirp=-8.216, fft_len=64 
Best triplet: peak=0, time=-2.123e+11, period=0, d_freq=0, chirp=0, fft_len=0
ID: 1826075 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1826140 - Posted: 22 Oct 2016, 17:24:19 UTC - in response to Message 1826067.  
Last modified: 22 Oct 2016, 17:37:15 UTC

Did you spot any differencies in common part of reported signals?

The signals that are common among the 4 reports seem to match up, but the interleaving of Spikes and Pulses varies and, of course, each host ended up with a different count for each type of signal.

I think the images below are probably the easiest way to compare the four. Trying to cut and paste into a single text file for side-by-side comparison in a readable format would take me half the morning. :^)

All signals:


Spikes only:


Pulses only:


I still feel that once an overflow is detected, there needs to be some common standard among all the apps as to how and what signals are reported, even if that means backing up and reprocessing some portion of the WU. No matter how optimized the app is, if a WU has to be resent to one, two, three or more additional hosts, that app efficiency goes out the window. It's better to have the first host (or two) do some extra work than to require additional hosts to process the entire WU all over again.
ID: 1826140 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1826151 - Posted: 22 Oct 2016, 18:02:43 UTC - in response to Message 1826140.  

No matter how optimized the app is, if a WU has to be resent to one, two, three or more additional hosts, that app efficiency goes out the window.

Depends on frequency of such events and speedup over non-overflowed tasks.


It's better to have the first host (or two) do some extra work than to require additional hosts to process the entire WU all over again.

With complete implementation of SoG path this issue should go away almost completely.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1826151 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1826248 - Posted: 23 Oct 2016, 5:30:26 UTC - in response to Message 1826151.  
Last modified: 23 Oct 2016, 5:32:11 UTC

No matter how optimized the app is, if a WU has to be resent to one, two, three or more additional hosts, that app efficiency goes out the window.

Depends on frequency of such events and speedup over non-overflowed tasks.

I wouldn't think that the optimization gain for non-overflowed tasks should be affected by any partial reprocessing of overflowed tasks. However, your point about the frequency of overflow tasks that require additional hosts to validate, versus the ones that validate with just the first two hosts, got me digging into my archives. I like to see hard numbers when questions like that come up.

I went back to last weekend, on the assumption that most of those WUs would have validated by now, and looked at all the overflow tasks for a 24-hour period for my two boxes that are running SoG r3528. I found a total of 212 overflows, which I tried to break out into "Instant" overflows and "Late Stage" overflows. For my purposes, I've considered anything that ran for under a minute as "Instant", while the rest (which all appeared to run for more than 2 minutes) are classified as "Late Stage". (In counting hosts, I count only those with tasks that validated or were declared invalid. Host tasks that had computation errors, were aborted or abandoned are omitted.)

Total "Instant" overflows: 185
Validated with 2 hosts: 170 (91.9%)
Validated with 3 hosts: 10 (5.4%)
Validated with 5 hosts: 1 (0.5%)
Still pending (2 hosts): 4 (2.2%)

Total "Late Stage" overflows: 27
Validated with 2 hosts: 23 (85.2%)
Validated with 3 hosts: 4 (14.8%)

Also, while I have nothing in my records that would specifically confirm what app the hosts other than my own were running, I did notice that 8 of the "Instant" overflows that took 3 hosts to validate involved at least one host that I know is currently running a version of Petri's Special: 8015199 and 8043690 (Laurent Domisse), 7907890 (WezH), 8018093 (Mr. Kevvy), and 7475713 (petri33). It seems like a legitimate assumption that all were running the same app a week ago during the period I sampled.

All in all, at least for that 24-hour period, it looks like a fairly high percentage of overflow WUs validated on the first try. I had gotten the impression recently that a higher percentage was failing to validate on the first try, at least for the Late Stage overflows, but maybe that's just because I've stumbled across some fairly egregious examples in the last several days. Perhaps, if I have time, I'll try to sample a more recent period, though that may just end up with a higher percentage still in a pending state. Who knows?
ID: 1826248 · Report as offensive
Profile Raistmer
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 16 Jun 01
Posts: 6325
Credit: 106,370,077
RAC: 121
Russia
Message 1826255 - Posted: 23 Oct 2016, 6:47:51 UTC - in response to Message 1826248.  
Last modified: 23 Oct 2016, 6:48:47 UTC

You missed my point perhaps.
To estimate performance degradation this "number of overflows" per 24h should be normilized on "number of total tasks" per the same 24h.
And then, to correctly estimate the amount of overhead one should compare with some other existed app that has lower number of inconclusives on overflow.
Only then some judgement if current approach provides speedup or resends kill any speedup could be made.
SETI apps news
We're not gonna fight them. We're gonna transcend them.
ID: 1826255 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1826334 - Posted: 23 Oct 2016, 17:15:57 UTC - in response to Message 1826255.  

You missed my point perhaps.
To estimate performance degradation this "number of overflows" per 24h should be normilized on "number of total tasks" per the same 24h.
And then, to correctly estimate the amount of overhead one should compare with some other existed app that has lower number of inconclusives on overflow.
Only then some judgement if current approach provides speedup or resends kill any speedup could be made.

I still don't see where any additional processing that might be triggered once an overflow situation has been detected would have any impact on non-overflow tasks. It shouldn't matter if overflows constitute 1% of all tasks or 50%.

However, what does seem relevant to me is the ratio of overflow tasks that still validate on the first attempt versus those that require resends to additional hosts. Would the additional processing to generate a standardized signal sequence for all overflow tasks be greater, in total, than the processing that would be saved by not sending some of those tasks to additional hosts?

So, what I was attempting to quantify with those overflow numbers, was where a reasonable cutoff might be. Just looking at the "Late Stage" overflows in my sample, if it required only an additional 10% overhead (only on overflow tasks) to eliminate the resending of tasks for 15% of the overflow WUs, then I would say that it's worth it. On the other hand, if the additional overhead exceeds 20%, then it probably wouldn't be. That's something of an oversimplification, of course, and I really don't know how representative that particular sample set is of overall overflow ratios, but I think it could at least be a starting point.
ID: 1826334 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1826375 - Posted: 23 Oct 2016, 22:54:34 UTC

Someone should been able to grab this 1, Workunit 2301590234, as it's still in progress.

Cheers.
ID: 1826375 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34744
Credit: 261,360,520
RAC: 489
Australia
Message 1826383 - Posted: 24 Oct 2016, 0:03:13 UTC

Sorry, I missed the edit window, but here's another 1, Workunit 2294646450.

Cheers.
ID: 1826383 · Report as offensive
Profile Jeff Buck Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 11 Feb 00
Posts: 1441
Credit: 148,764,870
RAC: 0
United States
Message 1826416 - Posted: 24 Oct 2016, 4:32:48 UTC - in response to Message 1826383.  

Sorry, I missed the edit window, but here's another 1, Workunit 2294646450.

Cheers.

I see that one was another quadruple Inconclusive that did validate with the 5th host (not counting Lionel's abortion) and everybody (except Lionel) got credit. I did manage to grab the WU if anyone wants it for offline testing.

I've also got a new quadruple in my own list this evening.

Workunit 2301658991 (blc3_2bit_guppi_57432_25217_HIP57328_0003.31678.831.18.27.117.vlar)
Task 5234418931 (S=6, A=0, P=13, T=11, G=0) SSE3xj Win32 Build 3528
Task 5234418932 (S=10, A=0, P=10, T=10, G=0) v8.00 windows_intelx86
Task 5236649323 (S=10, A=0, P=10, T=10, G=0) x41p_zi3k, Cuda 8.00 special
Task 5237476192 (S=1, A=25, P=1, T=3, G=0) v8.03 x86_64-apple-darwin
ID: 1826416 · Report as offensive
Kiska
Volunteer tester

Send message
Joined: 31 Mar 12
Posts: 302
Credit: 3,067,762
RAC: 0
Australia
Message 1826437 - Posted: 24 Oct 2016, 7:33:22 UTC - in response to Message 1826416.  

Sorry, I missed the edit window, but here's another 1, Workunit 2294646450.

Cheers.

I see that one was another quadruple Inconclusive that did validate with the 5th host (not counting Lionel's abortion) and everybody (except Lionel) got credit. I did manage to grab the WU if anyone wants it for offline testing.

I've also got a new quadruple in my own list this evening.

Workunit 2301658991 (blc3_2bit_guppi_57432_25217_HIP57328_0003.31678.831.18.27.117.vlar)
Task 5234418931 (S=6, A=0, P=13, T=11, G=0) SSE3xj Win32 Build 3528
Task 5234418932 (S=10, A=0, P=10, T=10, G=0) v8.00 windows_intelx86
Task 5236649323 (S=10, A=0, P=10, T=10, G=0) x41p_zi3k, Cuda 8.00 special
Task 5237476192 (S=1, A=25, P=1, T=3, G=0) v8.03 x86_64-apple-darwin


I have saved this data file on to Google Drive, persistent storage.
https://drive.google.com/file/d/0B0-3oeXJF8g0RmVLMDFibkZJUmc/view?usp=sharing
ID: 1826437 · Report as offensive
Previous · 1 . . . 24 · 25 · 26 · 27 · 28 · 29 · 30 . . . 36 · Next

Message boards : Number crunching : Monitoring inconclusive GBT validations and harvesting data for testing


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.