Export of credits.

Message boards : Number crunching : Export of credits.
Message board moderation

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Rob.B

Send message
Joined: 23 Jul 99
Posts: 157
Credit: 1,439,682
RAC: 0
United Kingdom
Message 886620 - Posted: 20 Apr 2009, 12:00:18 UTC

Any idea when the export of Seti credit totals to the likes of BoincStats is going to kick in? I'm over 20k adrift (even between the two figures for Seti on my seti account page) and that was before I left for work.

Rob.
ID: 886620 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 886626 - Posted: 20 Apr 2009, 12:45:15 UTC - in response to Message 886620.  

They published something last night, but that does not mean that it is up to date stats.

http://setiathome.berkeley.edu/stats/

ID: 886626 · Report as offensive
Rob.B

Send message
Joined: 23 Jul 99
Posts: 157
Credit: 1,439,682
RAC: 0
United Kingdom
Message 886659 - Posted: 20 Apr 2009, 14:21:05 UTC - in response to Message 886626.  

They published something last night, but that does not mean that it is up to date stats.

http://setiathome.berkeley.edu/stats/


Did they? My values show zero for yesterday and have not moved yet today (and that's on my SETI Boing account page).

No panic but looking at BoincStats filtered for SETI everyone is showing zero, so for once it's not just me.

Rob.

ID: 886659 · Report as offensive
Profile S@NL - Eesger - www.knoop.nl
Avatar

Send message
Joined: 7 Oct 01
Posts: 385
Credit: 50,200,038
RAC: 0
Netherlands
Message 886683 - Posted: 20 Apr 2009, 15:05:58 UTC

I have a similar thread going in "Questions and Answers : Web site"

The problem is not just for you, it's for everybody, teams, users.. all. (our stats does not include the hosts, but I suspect that one has the same issues..)

I think the cause is in the MySql problems Berkeley is having.. It looks like the export script is doing it's work fine, but is exporting from a "frozen table/database"..
The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS
ID: 886683 · Report as offensive
Profile Labbie
Avatar

Send message
Joined: 19 Jun 06
Posts: 4083
Credit: 5,930,102
RAC: 0
United States
Message 886698 - Posted: 20 Apr 2009, 15:58:11 UTC

IIRC, the stats export is run from the Replica DB. Maybe it hasn't been updated due to all the DB issues.

I've got almost 40K "missing". ;)


Calm Chaos Forum...Join Calm Chaos Now
ID: 886698 · Report as offensive
Rob.B

Send message
Joined: 23 Jul 99
Posts: 157
Credit: 1,439,682
RAC: 0
United Kingdom
Message 886716 - Posted: 20 Apr 2009, 17:08:45 UTC - in response to Message 886698.  

IIRC, the stats export is run from the Replica DB. Maybe it hasn't been updated due to all the DB issues.

I've got almost 40K "missing". ;)


Thats fine, I'm sure it will catch up eventualy. It's not a problem, I'm just curious as to what is going on. Nice to have an up-front answer.

Thanks.

Rob.
ID: 886716 · Report as offensive
Rob.B

Send message
Joined: 23 Jul 99
Posts: 157
Credit: 1,439,682
RAC: 0
United Kingdom
Message 886729 - Posted: 20 Apr 2009, 18:06:52 UTC

Another day with zero credits exported. Hey-Ho!
ID: 886729 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 886766 - Posted: 20 Apr 2009, 19:38:24 UTC - in response to Message 886729.  

Another day with zero credits exported. Hey-Ho!

Stats are exported from the replica database to limit loading, and from all other reports they had a very rocky weekend.

The stats are exported once per day (and picked up by the stat sites once per day). If that doesn't work tonight, I'm sure they'll be digging into it (unless they know the replica is bad and turned it off on purpose).
ID: 886766 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 886821 - Posted: 20 Apr 2009, 23:25:58 UTC - in response to Message 886716.  
Last modified: 20 Apr 2009, 23:54:44 UTC

IIRC, the stats export is run from the Replica DB. Maybe it hasn't been updated due to all the DB issues.

I've got almost 40K "missing". ;)


Thats fine, I'm sure it will catch up eventualy. It's not a problem, I'm just curious as to what is going on. Nice to have an up-front answer.

Thanks.

Rob.

See this post.
ID: 886821 · Report as offensive
Profile S@NL - Eesger - www.knoop.nl
Avatar

Send message
Joined: 7 Oct 01
Posts: 385
Credit: 50,200,038
RAC: 0
Netherlands
Message 887036 - Posted: 21 Apr 2009, 22:33:01 UTC

The new stats update is there, thanx for fixing this (also)!
The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS
ID: 887036 · Report as offensive
Profile S@NL - Eesger - www.knoop.nl
Avatar

Send message
Joined: 7 Oct 01
Posts: 385
Credit: 50,200,038
RAC: 0
Netherlands
Message 889148 - Posted: 28 Apr 2009, 12:31:53 UTC

Question,

Yesterday the data export was at 9:31:19 UTC, today the export was generated/finished at 09:38:18 UTC.

Usually the exports were around midnight UTC. So the difference is rather big..

When I look at the stats-folder I see that the tables.xml has the date: 28-Apr-2009 02:38

It almost looks like the "export server" (does that one exist?) is running in the wrong time-zone or something?


PS: Yesterday after the data sync, the accumulated credits for most teams/users etc was more then one day, so the exported data is "in sync", just the moment seems te have gotten out of sync..
The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS
ID: 889148 · Report as offensive
Profile arkayn
Volunteer tester
Avatar

Send message
Joined: 14 May 99
Posts: 4438
Credit: 55,006,323
RAC: 0
United States
Message 889151 - Posted: 28 Apr 2009, 12:47:19 UTC - in response to Message 889148.  

Question,

Yesterday the data export was at 9:31:19 UTC, today the export was generated/finished at 09:38:18 UTC.

Usually the exports were around midnight UTC. So the difference is rather big..

When I look at the stats-folder I see that the tables.xml has the date: 28-Apr-2009 02:38

It almost looks like the "export server" (does that one exist?) is running in the wrong time-zone or something?


PS: Yesterday after the data sync, the accumulated credits for most teams/users etc was more then one day, so the exported data is "in sync", just the moment seems te have gotten out of sync..


Berkeley is currently -7 UTC so that would be 02:38 PDT, the stats server does not have UTC on it.

ID: 889151 · Report as offensive
Profile S@NL - Eesger - www.knoop.nl
Avatar

Send message
Joined: 7 Oct 01
Posts: 385
Credit: 50,200,038
RAC: 0
Netherlands
Message 889190 - Posted: 28 Apr 2009, 15:00:26 UTC

OK.. but that changed then! I have (re)built a stats-system for my team. This went live at 28th of February. All that time the data export was created around or mostly a bit after midnight UTC. That seems to have changed to around 9:30 UTC since yesterday.

Not that I see this as a problem, I just thought it was odd, since there does not seem to be a reason in my view to change the export moment.

PS: I must confess that midnight UTC is preferable to me/our server since the data synchronisation takes place at night then (Dutch time = UTC+2 ;)
The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS
ID: 889190 · Report as offensive
Profile S@NL - Eesger - www.knoop.nl
Avatar

Send message
Joined: 7 Oct 01
Posts: 385
Credit: 50,200,038
RAC: 0
Netherlands
Message 889366 - Posted: 29 Apr 2009, 6:24:22 UTC
Last modified: 29 Apr 2009, 6:24:55 UTC

And got fixed again? ;) Last export: 22:16:03 28-04-2009 UTC (so the stats-sites won't show you had a great production day, because the interval was a bit more then half a day ;)

Some questions are floating around in my head..:

  • Is this the right sub-forum to post these kind of questions/remarks?
  • Is it appreciated that I mention/ask these kind of discontinuities?
  • Can one think (realisticly ;) that this got fixed, but "The Berkeley People" simply are to bussy to mention it?


The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS
ID: 889366 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 889555 - Posted: 29 Apr 2009, 17:58:28 UTC - in response to Message 889366.  

And got fixed again? ;) Last export: 22:16:03 28-04-2009 UTC (so the stats-sites won't show you had a great production day, because the interval was a bit more then half a day ;)

Some questions are floating around in my head..:

  • Is this the right sub-forum to post these kind of questions/remarks?
  • Is it appreciated that I mention/ask these kind of discontinuities?
  • Can one think (realisticly ;) that this got fixed, but "The Berkeley People" simply are to bussy to mention it?


I think Matt actually did mention it.
ID: 889555 · Report as offensive
Profile S@NL - Eesger - www.knoop.nl
Avatar

Send message
Joined: 7 Oct 01
Posts: 385
Credit: 50,200,038
RAC: 0
Netherlands
Message 889574 - Posted: 29 Apr 2009, 19:47:48 UTC

OK, I'll keep mentioning oddities.. (it's nice to think one contributes.. and then I don't just mean my RAC ;)

The first one.. and this has been asked before.. the 'number-crunching' section is the most active.. but I have in the past posted in 'Questions and Answers : Web site' too.. what is a good spot?
The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS
ID: 889574 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 889589 - Posted: 29 Apr 2009, 20:30:42 UTC - in response to Message 889574.  

OK, I'll keep mentioning oddities.. (it's nice to think one contributes.. and then I don't just mean my RAC ;)

The first one.. and this has been asked before.. the 'number-crunching' section is the most active.. but I have in the past posted in 'Questions and Answers : Web site' too.. what is a good spot?

Good question. I don't think the "Technical News" is the right place, although I think if you keep up with that section that most operational glitches get mentioned there sooner or later if there is anything worth saying.

We know from reading the technical news that the stats come from the backup (replica) database to ease loading on the primary.

We also know that when there is a problem in general, the stat updates are usually fixed last -- and we know that they often recopy the database during an outage instead of letting the normal "sync" process deal with it.

Q&A is manned with volunteers like us who may not know any more than we can observe. I don't think it's wrong to ask.

I've noted that questions asked here are often answered (more or less) by Matt in his (mostly-daily) technical news posts.
ID: 889589 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 889592 - Posted: 29 Apr 2009, 20:34:32 UTC - in response to Message 889190.  

OK.. but that changed then! I have (re)built a stats-system for my team. This went live at 28th of February. All that time the data export was created around or mostly a bit after midnight UTC. That seems to have changed to around 9:30 UTC since yesterday.

Not that I see this as a problem, I just thought it was odd, since there does not seem to be a reason in my view to change the export moment.

PS: I must confess that midnight UTC is preferable to me/our server since the data synchronisation takes place at night then (Dutch time = UTC+2 ;)

Wearing my developer's hat for a moment, I'd probably code my app. so it checked for an update (file timestamp changed) every couple of hours or so, and if it downloaded an update, wait something like 23 hours before checking again.

Changing that from two hours to 12 hours after a few tries would probably be more server-friendly.

That way it'd tend to download near an update but not load the connection much.
ID: 889592 · Report as offensive
Profile S@NL - Eesger - www.knoop.nl
Avatar

Send message
Joined: 7 Oct 01
Posts: 385
Credit: 50,200,038
RAC: 0
Netherlands
Message 889608 - Posted: 29 Apr 2009, 21:38:13 UTC - in response to Message 889592.  

Wearing my developer's hat for a moment, I'd probably code my app. so it checked for an update (file timestamp changed) every couple of hours or so, and if it downloaded an update, wait something like 23 hours before checking again.

Ahhh.. but for all those stats junkies out there an update is an update.. and they want it ;)

Changing that from two hours to 12 hours after a few tries would probably be more server-friendly.

That way it'd tend to download near an update but not load the connection much.

Ah.. the tables.xml is only a few hundred bytes ;)

And "all" those stats-sites do alleviate the stress on the Berkeley-site! I am certain that the "burden of exporting" is more then compensated by this..
The SETI@Home Gauntlet 2012 april 16 - 30| info / chat | STATS
ID: 889608 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 889612 - Posted: 29 Apr 2009, 21:58:40 UTC - in response to Message 889608.  

Wearing my developer's hat for a moment, I'd probably code my app. so it checked for an update (file timestamp changed) every couple of hours or so, and if it downloaded an update, wait something like 23 hours before checking again.

Ahhh.. but for all those stats junkies out there an update is an update.. and they want it ;)

Changing that from two hours to 12 hours after a few tries would probably be more server-friendly.

That way it'd tend to download near an update but not load the connection much.

Ah.. the tables.xml is only a few hundred bytes ;)

And "all" those stats-sites do alleviate the stress on the Berkeley-site! I am certain that the "burden of exporting" is more then compensated by this..

My understanding is that the export files are created once per day, so the goal would be to download just as soon after they are created as possible.

... and I'd want to do so without having to monitor the servers for any schedule changes at Berkeley.

I understand that some of the XML files are quite large, but it's been a long time and I no longer remember where they can be downloaded.
ID: 889612 · Report as offensive
1 · 2 · Next

Message boards : Number crunching : Export of credits.


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.