Pondering Stealth Results

Message boards : Number crunching : Pondering Stealth Results
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 939108 - Posted: 11 Oct 2009, 2:38:58 UTC
Last modified: 11 Oct 2009, 2:43:58 UTC

A week into be Jocelyn-less, I'm wondering what work-arounds people have to view the results of their crunched work units.

Issue is, even when Jocelyn comes back, probably 75% of your own results for the week will have already been cleaned off the reporting database, so we won't be able to browse granularly the points, check error WU's, drill down into triplet & gaussian counts, etc. Only the WU's where Wingmen haven't reported, or were reported in the previous 24h will be viewable.

Ideally (inside Boinc), the property page of completed & uploaded WU's would list some of this stuff, before actual reporting takes place (which we can optionally control anyhow), but it seems the data isn't really in the "_0" files. (Too bad the Boinc standard didn't accomodate a <summary><\summary> freeform field).

I looked at the descriptions of BoincSpy and BoincWatch, but they don't sound like they do what I'm after, which is to approximately duplicate the results page locally minus all aspects of the wingman (afterall, the data was more-or-less here and processed in the first place, and yes I do know misc validation, calculations, & analysis do take place server side -- and that your local view isn't necessarily what the Server will grant either).

Cheers.
ID: 939108 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 939111 - Posted: 11 Oct 2009, 3:04:12 UTC - in response to Message 939108.  

A week into be Jocelyn-less, I'm wondering what work-arounds people have to view the results of their crunched work units.

Issue is, even when Jocelyn comes back, probably 75% of your own results for the week will have already been cleaned off the reporting database, so we won't be able to browse granularly the points, check error WU's, drill down into triplet & gaussian counts, etc. Only the WU's where Wingmen haven't reported, or were reported in the previous 24h will be viewable.

Ideally (inside Boinc), the property page of completed & uploaded WU's would list some of this stuff, before actual reporting takes place (which we can optionally control anyhow), but it seems the data isn't really in the "_0" files. (Too bad the Boinc standard didn't accomodate a <summary><\summary> freeform field).

I looked at the descriptions of BoincSpy and BoincWatch, but they don't sound like they do what I'm after, which is to approximately duplicate the results page locally minus all aspects of the wingman (afterall, the data was more-or-less here and processed in the first place, and yes I do know misc validation, calculations, & analysis do take place server side -- and that your local view isn't necessarily what the Server will grant either).

Cheers.


Have you tried BoincView? It has a logging function as well. Altho I am unsure if it records the data you are wanting.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 939111 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 939124 - Posted: 11 Oct 2009, 6:40:03 UTC - in response to Message 939111.  

Thx Hal, I looked at screenshots of BoincView and Mike's Boinc.Net, and everything else I could easily find on Google Images. Nothing seems to capture the key result pieces and nothing seems to even capture the non-key pieces in an easy running log format. Oh well... I guess the addon imagineers figure that without wingman data, no sense in doing the other stuff.
ID: 939124 · Report as offensive
Profile FalconFly
Avatar

Send message
Joined: 5 Oct 99
Posts: 394
Credit: 18,053,892
RAC: 0
Germany
Message 939129 - Posted: 11 Oct 2009, 8:01:16 UTC - in response to Message 939124.  

I am missing such functions since the days of SETIwatch (logging top Gaussian/Pulse/Triplet for every single WorkUnit into local database).

By now I originally had expected that some BOINC-wide logging would exist (either per Project Site or with a dedicated BOINC tool). But apart from isolated and project-specialized tools requiring manual work (CPDN) or Projects like Rosetta (detailled results summary) usually nothing is shown concerning results.

Maybe we see something like that when NTPCKR is fully operational but I'm not too hopeful because of other priorities.
ID: 939129 · Report as offensive
Ianab
Volunteer tester

Send message
Joined: 11 Jun 08
Posts: 732
Credit: 20,635,586
RAC: 5
New Zealand
Message 939134 - Posted: 11 Oct 2009, 8:45:15 UTC - in response to Message 939108.  

Personally, I just get mildly annoyed that I cant check exactly what my machines are doing easily. But I can live with it. Backup server will be online in due course and things will be back to normal.

Ian
ID: 939134 · Report as offensive
MarkJ Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 17 Feb 08
Posts: 1139
Credit: 80,854,192
RAC: 5
Australia
Message 939143 - Posted: 11 Oct 2009, 10:24:30 UTC - in response to Message 939111.  

Have you tried BoincView? It has a logging function as well. Altho I am unsure if it records the data you are wanting.


Here is one BOINCview prepared earlier...

<log>
<tool>
  <name>BoincView</name>
  <version>1.4.2</version>
</tool>
<result>
  <name>02se09ab.22745.10701.13.10.121_0</name>
  <final_cpu_time>43.5000000000</final_cpu_time>
  <completed_time>0.000000</completed_time>
  <last_access>2455116.2152365740</last_access>
  <app>
    <name>setiathome_enhanced</name>
    <major_version>6</major_version>
    <minor_version>08</minor_version>
  </app>
  <client>
  </client>
</result>
</log>
<platform_name>windows_intelx86</platform_name>
<core_client_major_version>6</core_client_major_version>
<core_client_minor_version>10</core_client_minor_version>
<core_client_release>13</core_client_release>
<cpu_sched_period>0</cpu_sched_period>
<cpu_sched_work_done_this_period>0.000000</cpu_sched_work_done_this_period>
<user_run_request>1</user_run_request>
<user_network_request>2</user_network_request>
<work_fetch_no_new_work>0</work_fetch_no_new_work>
<cpu_benchmarks_pending>0</cpu_benchmarks_pending>
<network_status>2</network_status>
<task_suspend_reason>0</task_suspend_reason>
<network_suspend_reason>8</network_suspend_reason>
<host_venue>home</host_venue>
<host_info>
  <timezone>39600</timezone>
  <domain_name>Maul</domain_name>
  <ip_addr>10.1.1.6</ip_addr>
  <host_cpid>ca0239ef733b8e8ffec44c814777de32</host_cpid>
  <p_vendor>GenuineIntel</p_vendor>
  <p_model>Intel(R) Core(TM) i7 CPU         920  @ 2.67GHz</p_model>
  <p_capabilities>x86 Family 6 Model 26 Stepping 4</p_capabilities>
  <p_ncpus>8</p_ncpus>
  <p_fpops>2674765617.7786598200</p_fpops>
  <p_iops>6859032871.2105770100</p_iops>
  <p_membw>125000000.0000000000</p_membw>
  <p_calculated>12899560724.4375000000</p_calculated>
  <m_nbytes>3210784768.0000000000</m_nbytes>
  <m_cache>262144.0000000000</m_cache>
  <m_swap>5188280320.0000000000</m_swap>
  <d_total>320061046784.0000000000</d_total>
  <d_free>310871326720.0000000000</d_free>
  <os_name>Microsoft Windows XP</os_name>
  <os_version>Professional x86 Edition, Service Pack 3, (05.01.2600.00)</os_version>
  <accelerators></accelerators>
</host_info>
<time_stats>
  <last_update>12899714984.647375</last_update>
  <on_frac>0.5927040000</on_frac>
  <active_frac>0.9998940000</active_frac>
  <connected_frac>0.4993970000</connected_frac>
  <cpu_efficiency>1.0000000000</cpu_efficiency>
</time_stats>
<net_stats>
  <bwup>16368.3884250000</bwup>
  <bwdown>174996.6496500000</bwdown>
</net_stats>
<project>
  <master_url>http://setiathome.berkeley.edu/</master_url>
  <project_name>SETI@home</project_name>
  <sym_store></sym_store>
  <user_name>MarkJ</user_name>
  <team_name></team_name>
  <host_venue>home</host_venue>
  <email_hash>*****</email_hash>
  <cross_project_id>a489f8719b30db60d6c793d095111a3f</cross_project_id>
  <user_total_credit>7061755.077247</user_total_credit>
  <user_expavg_credit>28199.765717</user_expavg_credit>
  <user_create_time>1203242636.000000</user_create_time>
  <rpc_seqno>6239</rpc_seqno>
  <hostid>4848148</hostid>
  <host_total_credit>1662005.065958</host_total_credit>
  <host_expavg_credit>11303.514473</host_expavg_credit>
  <host_create_time>1237553440.000000</host_create_time>
  <nrpc_failures>0</nrpc_failures>
  <master_fetch_failures>0</master_fetch_failures>
  <min_rpc_time>12899706750.834875</min_rpc_time>
  <disk_usage>185117339.000000</disk_usage>
  <debt>69979.078527</debt>
  <long_term_debt>-254812.488178</long_term_debt>
  <resource_share>25.000000</resource_share>
  <duration_correction_factor>0.266899</duration_correction_factor>
  <sched_rpc_pending>0</sched_rpc_pending>
  <ams_resource_share>0.000000</ams_resource_share>
  <last_rpc_time>0.000000</last_rpc_time>
  <project_files_downloaded_time>0.000000</project_files_downloaded_time>
  <gui_urls>
<gui_url>
  <name>Message boards</name>
  <description>Correspond with other users on the SETI@home message boards</description>
  <url>http://setiathome.berkeley.edu/forum_index.php</url>
</gui_url>
<gui_url>
  <name>Help</name>
  <description>Ask questions and report problems</description>
  <url>http://setiathome.berkeley.edu/forum_help_desk.php</url>
</gui_url>
<gui_url>
  <name>Your account</name>
  <description>View your account information and credit totals</description>
  <url>http://setiathome.berkeley.edu/show_user.php?userid=8823491</url>
</gui_url>
<gui_url>
  <name>Your preferences</name>
  <description>View and modify your SETI@home account profile and preferences</description>
  <url>http://setiathome.berkeley.edu/home.php</url>
</gui_url>
<gui_url>
  <name>Your results</name>
  <description>View your last week (or more) of computational results and work</description>
  <url>http://setiathome.berkeley.edu/results.php?userid=8823491</url>
</gui_url>
<gui_url>
  <name>Your computers</name>
  <description>View a listing of all the computers on which you are running SETI@Home</description>
  <url>http://setiathome.berkeley.edu/hosts_user.php?userid=8823491</url>
</gui_url>
<gui_url>
  <name>Donate</name>
  <description>Donate to SETI@home</description>
  <url>http://setiathome.berkeley.edu/sah_donate.php</url>
</gui_url>
  </gui_urls>
</project>

BOINC blog
ID: 939143 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 939149 - Posted: 11 Oct 2009, 11:47:35 UTC - in response to Message 939143.  

Thx All

Have you tried BoincView? It has a logging function as well. Altho I am unsure if it records the data you are wanting.


I will look closer at BoincView and BoincSpy by installing them (along with .net 2.0 and other stuff I had been hoping to avoid on my cruncher system, oh well).

An old project called BoincLogX looked promising, but after installing it doesn't list things like Spikes, Gaussians, & triplets. Also, it's from 2006, and I had to hand edit the INI file since it's GUI directory picker has no notion of the virtual directories used by Boinc (not a problem once you figure out why nothing in it seemed to work)(plus it seems to periodically poll the work directory which really is very expensive and very out of lock step with the the multi cores of today not to mention the speed at which CUDA can now complete and move on to the next WU).
ID: 939149 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 939198 - Posted: 11 Oct 2009, 17:56:28 UTC - in response to Message 939149.  

Thx All

Have you tried BoincView? It has a logging function as well. Altho I am unsure if it records the data you are wanting.

I will look closer at BoincView and BoincSpy by installing them (along with .net 2.0 and other stuff I had been hoping to avoid on my cruncher system, oh well).

An old project called BoincLogX looked promising, but after installing it doesn't list things like Spikes, Gaussians, & triplets. Also, it's from 2006, and I had to hand edit the INI file since it's GUI directory picker has no notion of the virtual directories used by Boinc (not a problem once you figure out why nothing in it seemed to work)(plus it seems to periodically poll the work directory which really is very expensive and very out of lock step with the the multi cores of today not to mention the speed at which CUDA can now complete and move on to the next WU).

Probably the best approach is run with No New Tasks turned on so the core client doesn't report completed work too soon, save a copy of client_state.xml before each time you allow new tasks. The stderr sections in those saved files are the same as shown on a Task details page, and the <fpops_cumulative> value can be converted to an approximation of the credit claim (not exact since we don't know the server-side credit_multiplier value for any specific "sent" date).

The interval at which BoincLogX checks files can be adjusted. I'm not sure its default checking is particularly burdensome compared to BOINC Manager which does a complete set of RPCs every second, I run without the Manager 99.9% of the time but do have BoincLogX always running. I don't have a fast multicore system though. BoincLogX doesn't save the fpops_cumulative value so those particularly interested in credit might be disappointed, but it does save all signals it extracts from the MB result files; there's no built-in display but the csv files are there for viewing in a spreadsheet program.

The ideal would of course be a version of the BOINC core client which would log the pertinent details before deleting them. But that's a feature that only a minority of users would want or understand so is not likely to be added by the BOINC devs.
                                                                Joe
ID: 939198 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 939225 - Posted: 11 Oct 2009, 20:45:38 UTC - in response to Message 939198.  

Thx J.S, all points valid and well taken.

The interval at which BoincLogX checks files can be adjusted. I'm not sure its default checking is particularly burdensome compared to BOINC Manager which does a complete set of RPCs every second


I would think the directory dredge burden takes over after you have a few hundred WU's in inventory (certainly before a few thousand), or at least I would hope the RPCs are efficient in internally tracking state changes (and yes, I know BMgr starts to wobble itself around 2000 WUs, so the RPCs are not heaven either on frequent large data sets, although I have a feeling most of that wobble is in the UI control itself). If LogX had used a filter driver (yes, much harder code), then that might have been the best of both sides.


The ideal would of course be a version of the BOINC core client which would log the pertinent details before deleting them.


Yes! The tricky part is how to meaningfully represent the succint progress or the succint summation of interest per WU when every project is different (the notion of "credits" is basically a universal leveler which abstracts away that problem). That's kind of why I suggested this inside the existing Task Tab property page (or a single column at the end) as a free form data field (ie: XML of course, but stripped at display time), at least then human eyes could scan it (which is what I do today by clicking down into the web Results page when it's up).

So what I was hoping for was a local log of the STATE files from the \slot directories. IE: after a WU completes but before Upload takes place, a process fetched and parsed a copy (at least before the next WU wipes it out). The complicator as said before is now how do you present that meaningfully... a tabbed pivot table is kind of heavy weight.

But that's a feature that only a minority of users would want or understand.


And that's the stumper for me. I would think at least with this project (S@H), well over 80% ARE interested in knowing if one of their WU's hit triple digits for Gaussians, Triplets, Pulses, and Spikes. Sort of a local NTPCkr. I see tools which are close designed for Seti Classic, but were never rev'd for Boinc.

ID: 939225 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 939398 - Posted: 12 Oct 2009, 16:55:12 UTC - in response to Message 939225.  

And that's the stumper for me. I would think at least with this project (S@H), well over 80% ARE interested in knowing if one of their WU's hit triple digits for Gaussians, Triplets, Pulses, and Spikes. Sort of a local NTPCkr. I see tools which are close designed for Seti Classic, but were never rev'd for Boinc.

According to BOINCstats, there are more than 1 million participants at SETI@Home.

Last time I asked, I was told that 8,800 of those had posted in the forums at least once.

No idea how many read the forums, but I'd bet that number is surprisingly small.

Our picture of how BOINC and SETI@Home should work is colored by the fact that we in the forums represent less than 1% of all SETI@Home crunchers.
ID: 939398 · Report as offensive
Profile 52 Aces
Avatar

Send message
Joined: 7 Jan 02
Posts: 497
Credit: 14,261,068
RAC: 67
United States
Message 939444 - Posted: 12 Oct 2009, 21:26:00 UTC - in response to Message 939398.  
Last modified: 12 Oct 2009, 21:40:14 UTC

According to BOINCstats, there are more than 1 million participants at SETI@Home.


Take a closer look. More than 900,000 of those have a current monthly credit of zero ... which in my book is dead (or at least profoundly hibernating). They'd probably have no opinion on if the "advanced" UI view were completely eliminated.

Our picture of how BOINC and SETI@Home should work is colored by the fact that we in the forums represent less than 1% of all SETI@Home crunchers.


Point is very well taken, but don't undervalue that 1% by body count too much, as they probably represent 40% of the cobblestones and 75% of the "tinker & ponder" time spent this year by the active S@H community.
ID: 939444 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 939453 - Posted: 12 Oct 2009, 21:51:08 UTC - in response to Message 939444.  

Our picture of how BOINC and SETI@Home should work is colored by the fact that we in the forums represent less than 1% of all SETI@Home crunchers.


Point is very well taken, but don't undervalue that 1% by body count too much, as they probably represent 40% of the cobblestones and 75% of the "tinker & ponder" time spent this year by the active S@H community.

The design goals for BOINC are not for the "tinker and ponder" set but the set-and-forget user.

1 million is high, but so is 8,800 -- it's the number who have posted just once.

There are many who are not forum-fanatics.
ID: 939453 · Report as offensive
CryptokiD
Avatar

Send message
Joined: 2 Dec 00
Posts: 150
Credit: 3,216,632
RAC: 0
United States
Message 939529 - Posted: 13 Oct 2009, 3:02:03 UTC - in response to Message 939453.  

there are also many who read the forums but do not post. lurkers as we tend to call them.
ID: 939529 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 939544 - Posted: 13 Oct 2009, 4:09:44 UTC - in response to Message 939529.  

there are also many who read the forums but do not post. lurkers as we tend to call them.

I am aware of the term, and the behaviour. The project doesn't know how many people read the forums, only those who log in and post.
ID: 939544 · Report as offensive
CryptokiD
Avatar

Send message
Joined: 2 Dec 00
Posts: 150
Credit: 3,216,632
RAC: 0
United States
Message 939551 - Posted: 13 Oct 2009, 4:43:24 UTC - in response to Message 939544.  

i dont know what type of forum software we use, but on some, perhaps most bbs, the forum software is sophisticated enough to distinguish anon's from registered users based on ip addy's, maybe cookies? and such. so a registered user that is not logged in will not be counted as an anon when they browse before logging in to post. this type of bbs software can show you the number of unique guests, regular guests, registered users, registered users that are not logged in, etc.

i dont know if seti@home forum software can do this or not.
ID: 939551 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 939553 - Posted: 13 Oct 2009, 4:47:54 UTC - in response to Message 939551.  

i dont know what type of forum software we use, but on some, perhaps most bbs, the forum software is sophisticated enough to distinguish anon's from registered users based on ip addy's, maybe cookies? and such. so a registered user that is not logged in will not be counted as an anon when they browse before logging in to post. this type of bbs software can show you the number of unique guests, regular guests, registered users, registered users that are not logged in, etc.

i dont know if seti@home forum software can do this or not.

The forum software is unique to (and integrated with) BOINC.

I asked for the statistics, and Eric said "we don't know how many read without posting."

Could they figure something out through the Apache logs? Maybe.
ID: 939553 · Report as offensive

Message boards : Number crunching : Pondering Stealth Results


 
©2026 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.