Team-stats and Berkeley load

I have started to (re)build the SETI-stats of our team Seti@Netherlands to "their former glory".

The first action was to rebuild the "statsengine" that reads the Berkeley Data. The software that was used didn't work very good and needed a complete rebuild. This part has been completed and works very good now. (I spent special attention to not unnecessarily downloading the GZ files)

I have also altered/updated our forum sig stats update tool, there I found this handy URL:

http://setiathome.berkeley.edu/userw.php?id=3339763

I am thinking of using this URL / information to create more accurate/recent stats for our team. But what is acceptable usage?
I/we are thinking of this:
A S@NL member needs to have a RAC of more then 50 or will be passed by (position change). In our team that will amount to a 500 accounts.

Now, I don't want to strain the resources of the Berkeley network, so the "userw.php" is ideal and I have read that this php-code (even the entire system) caches the SQL-output (by the hour I believe).

Is it a problem when I would request this information (those aprox. 500 users) from 7:30 to 24:00 GMT+1 for every hour?

You may want to contact mattl at ssl dot berkeley dot edu with your intentions and ask if this will cause any problems just to be on the safe side.
He's the one that keeps an eye on the system for bots and such, and would be the one to take action.

Use the http://setiathome.berkeley.edu/stats/ directory instead, and please check the time stamp before downloading.

I already said:

(I spent special attention to not unnecessarily downloading the GZ files)

I am a programmer by profession, I am not the newby on the block.

Don't. Screen scraping is hard on the bandwith of the project and can get the IP used to do so bannished from the site.

I found nothing about excessive usage and banishing. I did find this:
http://boinc.berkeley.edu/trac/ticket/268

This mentions 5 to 10 requests per second!? I am talking about 1 request of aprox 400 bytes per 7 seconds!?
Bandwidth, well I don't even download the hosts.gz, that gives me "bandwidth credit" for aprox 1.000.000.000 userw.php files a day..

But stop.. I don't want to start out on the wrong foot here.. I could just have done it, and with what I have red (like in that ticket) I thought I wasn't doing weird things. I thought I was being nice to ask what is excepted. I didn't expect to get a slap on the wrist with a decent question.

Sorry I was a little abrupt. However, I do know that a screen scraper had their IP banished from the site yesterday. You do not get an email, you just suddenly can't get to the site anymore.

Apology accepted of course. I wanted to PM you.. but didn't know how to start... ;) We started out on the wrong foot. I really do understand the risc of the idea I am working out.. This "system" can/will be tunable to your wishes (as to what is acceptable) and since I believe in "asking before doing"... ;) (you understand.. ;)

OK.. then may I ask a last "worst case scenario" question?
What if, even with all my best intentions, software coding etc, our server does get blocked(*).
Obviously I'll find out practically immediately (assuming I have "the system up and running").
I hereby promise to create a "dead man's switch" in this part of the stats-code and inform "the S@NL board" on how to use it (just imagine I am on holiday when it happens..).
can I ask you (John McLeod VII) or mail Matt to release the IP number? Otherwise we would have to start pulling stunts to keep the daily stats update going....

(*) there always is the risk of making/creating a disastrous software bug resulting in...
I log practically everything, and check the logs regularly to see if everything goes as planned..
But a (complex) piece of software can be a bit like life sometimes.. you never know for 100% sure.. ;)