John

When the Time Comes, I look forward to flying down to Meet Tony and helping with the Bonfire Seen From Space!

I still have the wood and many more are prime candidates

The wife is going to go visit Sister in April... I would be happy if you have tools to help rebuild the cylinders on the backhoe and buy More Beer, for the Bonfire... In Fact I would be "Honored" flying down for the Bonfire Seen From Space..

In my younger days cussed at a 12 footer that I was "told" to rebuild... LOL...

what's all this bonfire business about then? anything in particular being burt? the first thing that comes to mind is old, dead computers lol

also, pappa, what about the classic servers

John

When the Time Comes, I look forward to flying down to Meet Tony and helping with the Bonfire Seen From Space!

I still have the wood and many more are prime candidates

The wife is going to go visit Sister in April... I would be happy if you have tools to help rebuild the cylinders on the backhoe and buy More Beer, for the Bonfire... In Fact I would be "Honored" flying down for the Bonfire Seen From Space..

In my younger days cussed at a 12 footer that I was "told" to rebuild... LOL...

Lee this is weird!

(The Nice thing is that there is an) "educational discount"

Numbered 1-12 Tests (click "reply" to view the BBCode):<br />
1 <br />
2 <br />

John

When the Time Comes, I look forward to flying down to Meet Tony and helping with the Bonfire Seen From Space!

R/

Al

I still have the wood and many more are prime candidates

It has been stated that Matt L "Sorta" needs some New hardware to help... UCB / Seti is a Sun Shop and in order for you to develope a Portal for the users there would need to be some mutial form of communcations (access to the overworked database). So connectivy aside (now that seti classic has closed there is a bit more left over). It could be doable..

From some of my earlier posts in finding out what really runs Seti and a couple of threads attempting to help get another "Sun V40z" through Donations Sorry I could not miss the chance... This with Matt's statement about needed 2+ terabytes to backup Databases (which would also mean Outages would decrease in time or disappear)... You might be able get UCB/Seti to buy into this...

Pappa, do you know if one of the classic servers is going to function as a slave DB server that was talked about ages ago, but promptly abandoned due to performance reasons?

also the link you posted just before you quoted john doesn't appear for some reason, it didn't appear in my reply either, so i'm guessing it's BBCode being helpful again!

It has been stated that Matt L "Sorta" needs some New hardware to help... UCB / Seti is a Sun Shop and in order for you to develope a Portal for the users there would need to be some mutial form of communcations (access to the overworked database). So connectivy aside (now that seti classic has closed there is a bit more left over). It could be doable..

From some of my earlier posts in finding out what really runs Seti and a couple of threads attempting to help get another "Sun V40z" through Donations Sorry I could not miss the chance... This with Matt's statement about needed 2+ terabytes to backup Databases (which would also mean Outages would decrease in time or disappear)... You might be able get UCB/Seti to buy into this...

page.php?otherattribute=othervalue&format=xml

John

When the Time Comes, I look forward to flying down to Meet Tony and helping with the Bonfire Seen From Space!

R/

Al

I don't think they are going to want to put the results info into the stats-dump. They currently dump the teams, users and hosts data as compressed (.gz) XML files. The teams.gz file is the smallest, currently at 3.5 MB. Users.gz is at 24.2 MB, and hosts.gz is a whopping 98.9 MB. IF they were to also generate a 'results.gz' file, it would likely be 100's of MB in size. Not only would it constipate the BOINC database while it was being generated, the bandwidth required to distribute it would quickly become an issue.

Now, all this said, the BEST (in my opinion) place for data-collection about results completed is client-side.

Now, I know there are various logging-programs out there now, and maybe some of them would implement this (if it is not already being worked on):

• The logging app on your computer could communicate with the BOINC daemon running on that computer. using the same mechanism that BOINCManager uses, namely sockets. Optionally, it could also do the same for any other computer running on your LAN that you are running BOINC on (similar to BOINCView).
  
• The logging app could then, for every result the computer(s) process, 'log' any data you wish to a database.
  

• The logging app could then, at a configurable time once every day, 'visit' YOUR
  results page and grab the final piece of the puzzle -- 'credit granted' for each result that was validated that day. It is possible to do this from the HTML page, but having an XML version would make life much easier here. One would only need to grab the results page, and not the pages on the work units or results, because the work units can be matched up by time-sent and time reported (and if needed, cpu time). Since it grabs THAT USER's page ONLY, and only does so once a day, I doubt that S@H (or any of the other projects, for that matter) would have a problem with it (unless, of course, it becomes TOO popular, and EVERYONE starts running it).
  

• Optionally, if one wants a centralized point of data collection for analysis and comparasion purposes, you could have each logging app (should the user opt into it) 'call home' once a day, and dump that day's changed data into a central database (similar to something Ken had going for a while on SetiQueue).
  

I am thinking that this might be a great tool for teams to use. And it has the advantages of not involving ANY project-resources except, of course, for the once a day 'results grab' (to get granted-credit data for each result). This puts the burden of stats collection where it rightly belongs.

To: Willy and Toby & other posters to this thread;

Thanks for your input.

Here is the reason behind my question. I am the CEO of a software Company that would like further the goals of Berkley, BOINC, and their distributed computing projects. I am not a programmer myself, thus my very non-technical questions. I have written a high-level whitepaper on developing a web-based portal that would have a dedicated resource or resources devoted to creating a portal that will take all of the information distributed by BOINC and give you (the user) the ability to create your own customized portal with several pages of data with the information YOU want to see. It will also (in the beginning give you the ability to perform other analysis that is not available on any of the other third party web-sites I have researched.

In order to finish the whitepaper for presentation to the Board and approval, I have been posting these rather simplistic questions. There will also be a new “patented” feature that will I know will display certain information that (almost) all users will be excited about and attempt to reinstate some of the feature that vanished with classic SETI

Now having said that, will there be a problem with a complete XML dump at hourly intervals for the purpose of updating each user’s portal and give allow me to track donated CPU time?

Perhaps I should contact Berkeley directly?

Many Thanks,

John

Do this and your IP will surely be blocked by Berkeley. The XML outputs are here to prevent the scraping of HTML pages.

Not if you are only doing a single user's results and only once or twice a day. The sites that were blocked were scraping thousands of user/team stats which are already available in XML format. However HTML scraping is still kind of a nasty way of getting data. How about BoincLogX? This is a client-side utility that logs information about work units as they are processed by your client.

John, take a look at my post in your other thread regarding the matter, you would be best to contact Dr. David Anderson as the result(s) data isn't part of the exported XML stats
you might also want to contact Andy K about how his Pending Credit "calculator" works, as i'm sure it would use a lot of the techquiues you're interested in (but requesting UCB to publish an XML version of pages is the best option)

• The logging app on your computer could communicate with the BOINC daemon running on that computer. using the same mechanism that BOINCManager uses, namely sockets. Optionally, it could also do the same for any other computer running on your LAN that you are running BOINC on (similar to BOINCView).
  
• The logging app could then, for every result the computer(s) process, 'log' any data you wish to a database.
  
• The logging app could then, at a configurable time once every day, 'visit' YOUR
  results page and grab the final piece of the puzzle -- 'credit granted' for each result that was validated that day. It is possible to do this from the HTML page, but having an XML version would make life much easier here. One would only need to grab the results page, and not the pages on the work units or results, because the work units can be matched up by time-sent and time reported (and if needed, cpu time). Since it grabs THAT USER's page ONLY, and only does so once a day, I doubt that S@H (or any of the other projects, for that matter) would have a problem with it (unless, of course, it becomes TOO popular, and EVERYONE starts running it).
  
• Optionally, if one wants a centralized point of data collection for analysis and comparasion purposes, you could have each logging app (should the user opt into it) 'call home' once a day, and dump that day's changed data into a central database (similar to something Ken had going for a while on SetiQueue).
  

I am thinking that this might be a great tool for teams to use. And it has the advantages of not involving ANY project-resources except, of course, for the once a day 'results grab' (to get granted-credit data for each result). This puts the burden of stats collection where it rightly belongs.

Hmm... maybe I might work on something like this... I am bored, of late, and have been needing a good project to keep me busy until UNT starts back up for the Spring semester.

Perhaps I should contact Berkeley directly?

Many Thanks,

John

Do this and your IP will surely be blocked by Berkeley. The XML outputs are here to prevent the scraping of HTML pages.

Not if you are only doing a single user's results and only once or twice a day. The sites that were blocked were scraping thousands of user/team stats which are already available in XML format. However HTML scraping is still kind of a nasty way of getting data. How about BoincLogX? This is a client-side utility that logs information about work units as they are processed by your client.

Do this and your IP will surely be blocked by Berkeley. The XML outputs are here to prevent the scraping of HTML pages.

Well, I have just gone over the BOINC source code, and I just don't see this function as being available. Perhaps you could write a perl script that would download this page in html, strip out the html with regexps, then place the remaining data into a database. This is how I used to handle a stats site I ran.

Thanks for your response. I have this information in XML format, I am looking for the page that lists the total CPU time taken to return each work unit. The XML feeds this page:

http://setiathome.berkeley.edu/results.php?userid=2084190

My goal is to track time WU was sent, CPU time to complete WU, and Time WU was returned to the project.

Thanks for your help!

John

To all,

I'm sure somebody has an idea on how to pull user results down in XML format. There are too many smart people here.

Regards,

John

To All:

I am attempting to request by resuts and display them in an XML format.

The normal URL request in SETI is:

http://setiathome.berkeley.edu/results.php?userid=2084190

Can anyone (especially programmers) tell me how to request this same data in a XML output form?

Regards,

John Lee

http://setiathome.berkeley.edu/show_user.php?userid=2084190&format=xml

This what you are looking for?



<user>
<id>2084190</id>
<cpid>8788594b6382858ff50920512465d0e3</cpid>
<create_time>1007648011</create_time>
<name>Team: Lee</name>
<country>United States</country>
<total_credit>278624.629705</total_credit>
<expavg_credit>2861.471542</expavg_credit>
<expavg_time>1136093760.3394</expavg_time>
<teamid>118944</teamid>
<url>www.akuratus.com</url>
<has_profile>1</has_profile>
</user>

To all,

I'm sure somebody has an idea on how to pull user results down in XML format. There are too many smart people here.

Regards,

John

To All:

I am attempting to request by resuts and display them in an XML format.

The normal URL request in SETI is:

http://setiathome.berkeley.edu/results.php?userid=2084190

Can anyone (especially programmers) tell me how to request this same data in a XML output form?

Regards,

John Lee