Why so much pending credit...

Message boards : Number crunching : Why so much pending credit...
Message board moderation

To post messages, you must log in.

Previous · 1 · 2 · 3 · Next

AuthorMessage
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 969447 - Posted: 9 Feb 2010, 14:21:10 UTC - in response to Message 969426.  

Seems there's quite a few folks like myself waiting for a pot-load of work to be validated. I'm not going to knock the process (S@H) but in reading through the forums, from what I gather, the Home Offices' approach seems a little disjointed. I'm sure I'll gain a better understanding of the big picture as I progress but God, with astronomical amount of comp time/crunching being expended there has to be a better way to streamline and speed up the process...Meanwhile, the wingmen as far as I can tell have flown south for the winter...and ET sits back in his recliner, smoking a cig and thinking "you've got to be kidding!"


The thing is, streamlining any process is a natural human intention to make best use of resources, but the fact of the matter is the data we're sifting through is from light-years away. Even if a signal is in your workunit, the signal is going to sit in a database before it can be re-observed and confirmed.

If anything, the ET sitting back in his recliner happened thousands of years ago and is mostly likely not even living anymore, though his progeny most likely is if the race didn't wipe themselves out. Their progeny might most likely be thinking, "It's ok, they'll get there eventually. They're still a young race."
ID: 969447 · Report as offensive
Matthew S. McCleary
Avatar

Send message
Joined: 9 Sep 99
Posts: 121
Credit: 2,288,242
RAC: 0
United States
Message 969455 - Posted: 9 Feb 2010, 14:57:34 UTC

Overall is down to 4.8 million now, but I'm at 63,700 and still climbing.
ID: 969455 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14655
Credit: 200,643,578
RAC: 874
United Kingdom
Message 969461 - Posted: 9 Feb 2010, 15:12:12 UTC - in response to Message 969455.  

Overall is down to 4.8 million now, but I'm at 63,700 and still climbing.

You're comparing apples with oranges. The figure on the Server Status page is the number of Tasks which have been returned and are awaiting validation: this thread is about the value of pending Credit. Each task represents a variable, but much larger than unity, number of credits.

Or are you saying that you have 63,700 tasks waiting to validate? Now that would be a backlog.....
ID: 969461 · Report as offensive
Dave

Send message
Joined: 29 Mar 02
Posts: 778
Credit: 25,001,396
RAC: 0
United Kingdom
Message 969474 - Posted: 9 Feb 2010, 15:47:24 UTC - in response to Message 969423.  

This is now at 4.8 mil.
Or have an option for impatient people to only process tasks that need to be validated

Gets my vote to


+/or just give them shorties. then slowly 'wean' them onto the other units. All sounds a bit too complicated to me now.
ID: 969474 · Report as offensive
Matthew S. McCleary
Avatar

Send message
Joined: 9 Sep 99
Posts: 121
Credit: 2,288,242
RAC: 0
United States
Message 969477 - Posted: 9 Feb 2010, 15:56:28 UTC - in response to Message 969461.  
Last modified: 9 Feb 2010, 16:03:01 UTC


Or are you saying that you have 63,700 tasks waiting to validate? Now that would be a backlog.....


That is precisely what I'm saying. Here is from just a few minutes ago, having clicked on my "pending credit" link under my account page.



ID: 969477 · Report as offensive
Profile Fred J. Verster
Volunteer tester
Avatar

Send message
Joined: 21 Apr 04
Posts: 3252
Credit: 31,903,643
RAC: 0
Netherlands
Message 969478 - Posted: 9 Feb 2010, 15:56:47 UTC - in response to Message 969474.  
Last modified: 9 Feb 2010, 16:01:08 UTC

Hi, it only takes a valid-result and some time and the validators have to work.
And a little patience . . .

In the meantime just keep on crunching.
ID: 969478 · Report as offensive
Matthew S. McCleary
Avatar

Send message
Joined: 9 Sep 99
Posts: 121
Credit: 2,288,242
RAC: 0
United States
Message 969480 - Posted: 9 Feb 2010, 16:05:18 UTC - in response to Message 969478.  

Hi, it only takes a valid-result and some time and the validators have to work.
And a little patience . . .

In the meantime just keep on crunching.


I'm not sure who this was directed at -- probably me. I'm not being impatient; I'm well aware this is how the process works. However, I've never before seen my pending credit this high -- usually it's no more than 2-3x my RAC, which is a rule of thumb I've seen around here several times. My intent here is not to kvetch about how much pending credit I have but rather to point out there may be something wrong on the backend.
ID: 969480 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14655
Credit: 200,643,578
RAC: 874
United Kingdom
Message 969487 - Posted: 9 Feb 2010, 16:29:12 UTC - in response to Message 969477.  

Or are you saying that you have 63,700 tasks waiting to validate? Now that would be a backlog.....

That is precisely what I'm saying. Here is from just a few minutes ago, having clicked on my "pending credit" link under my account page.


No, that's telling you how many cobblestones (aka credits) you have pending.

You can't have .58 of a task pending, or anywhere else for that matter!
ID: 969487 · Report as offensive
Matthew S. McCleary
Avatar

Send message
Joined: 9 Sep 99
Posts: 121
Credit: 2,288,242
RAC: 0
United States
Message 969489 - Posted: 9 Feb 2010, 16:45:06 UTC - in response to Message 969487.  


You can't have .58 of a task pending, or anywhere else for that matter!


Oh, okay, I see what you're saying now.

That said, though it's nowhere near as bad as having 67k tasks pending, 67000 cobblestones pending is still several standard deviations higher than I've ever seen before.
ID: 969489 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14655
Credit: 200,643,578
RAC: 874
United Kingdom
Message 969493 - Posted: 9 Feb 2010, 16:49:42 UTC - in response to Message 969489.  

You can't have .58 of a task pending, or anywhere else for that matter!

Oh, okay, I see what you're saying now.

That said, though it's nowhere near as bad as having 67k tasks pending, 67000 cobblestones pending is still several standard deviations higher than I've ever seen before.

Well, can't disagree with that comment.

If it's any consolation, mine says "Pending credit: 123,553.51", but I'm not worrying - they'll all turn in eventually.
ID: 969493 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 969499 - Posted: 9 Feb 2010, 17:01:53 UTC - in response to Message 969489.  


You can't have .58 of a task pending, or anywhere else for that matter!


Oh, okay, I see what you're saying now.

That said, though it's nowhere near as bad as having 67k tasks pending, 67000 cobblestones pending is still several standard deviations higher than I've ever seen before.

The process that does the validating was off for a day or two. That is most likely the reason of the rapid increase in pending credit. Mine that is normally around 50,000 got up to about 85,000. Not that is really matters.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 969499 · Report as offensive
Profile Pappa
Volunteer tester
Avatar

Send message
Joined: 9 Jan 00
Posts: 2562
Credit: 12,301,681
RAC: 0
United States
Message 969533 - Posted: 10 Feb 2010, 5:18:25 UTC
Last modified: 10 Feb 2010, 5:20:32 UTC

I have started to post before... RAC and Pending Credit depend on the Average Turnaround Times as seen in the Server Statitics. It is based on an old assumption.

Pre Cuda I worked through a lot of spreadsheet data to determine that a crazy formula could tell what was happening.

Pending Credit * 0.90 / ATR = RAC

So the assumption is that 90% of my returned results get through daily.
ATR = Average Turnaround Time (from the Server Status page).

One thing that affects the crazy game is outages and users that Very Large Caches. Otherwise it would be a single user that bombed a WU or an Overclocked machine that crashed thousands of WU's...

I started collecting information at about 8:00am PST daily on the 14th of January to test the assumption to see if it still hold true. So other than a few bobbles it is interesting to say the least.

To run the test, you only need a couple items of information, a piece of paper and a calculator. Then you can see some of the things that happen over a period of time.

What you want to record, "daily" at the same time every day from your Account Page.
Total credit
Recent average credit
Pending Credit

Then from the server status page
Result turnaround time (last hour average)

Hopefully tomorrow I can reconstruct what I started to write and will post (using my data).

Regards
Please consider a Donation to the Seti Project.

ID: 969533 · Report as offensive
Profile perryjay
Volunteer tester
Avatar

Send message
Joined: 20 Aug 02
Posts: 3377
Credit: 20,676,751
RAC: 0
United States
Message 969588 - Posted: 10 Feb 2010, 16:35:17 UTC

I'm not going to dig around to find it but I do remember not too long ago some of the bigger crunchers were having a race to see who would get to 1 million pending credits first. Kinda puts it all in perspective doesn't it?

We all hit a stretch of bad wingmen who either disappear or have problems with their machines. That and server problems that cause a lot of ghost work units to go out. It all works out in the end.

This VLAR storm we just went through is hitting me. I've got a bunch of them moved over from my GPU by my reschedular that are gonna be close to their timeout times. I aborted a bunch of them but they were just replaced by a bunch more. I average about 6 a day returned on my two CPUs so it won't take me too long to get them done but I don't like making my wingmen wait so long for results.


PROUD MEMBER OF Team Starfire World BOINC
ID: 969588 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14655
Credit: 200,643,578
RAC: 874
United Kingdom
Message 969613 - Posted: 10 Feb 2010, 18:26:46 UTC - in response to Message 969493.  

You can't have .58 of a task pending, or anywhere else for that matter!

Oh, okay, I see what you're saying now.

That said, though it's nowhere near as bad as having 67k tasks pending, 67000 cobblestones pending is still several standard deviations higher than I've ever seen before.

Well, can't disagree with that comment.

If it's any consolation, mine says "Pending credit: 123,553.51", but I'm not worrying - they'll all turn in eventually.

Now gone down to "Pending credit: 122,480.30".

Somebody asked me in a PM how I got that figure. On your Account page, there's a link for pending credit. Just scroll to the very bottom, and it's there.
ID: 969613 · Report as offensive
Profile hiamps
Volunteer tester
Avatar

Send message
Joined: 23 May 99
Posts: 4292
Credit: 72,971,319
RAC: 0
United States
Message 969660 - Posted: 10 Feb 2010, 23:30:23 UTC - in response to Message 969613.  

You can't have .58 of a task pending, or anywhere else for that matter!

Oh, okay, I see what you're saying now.

That said, though it's nowhere near as bad as having 67k tasks pending, 67000 cobblestones pending is still several standard deviations higher than I've ever seen before.

Well, can't disagree with that comment.

If it's any consolation, mine says "Pending credit: 123,553.51", but I'm not worrying - they'll all turn in eventually.

Now gone down to "Pending credit: 122,480.30".

Somebody asked me in a PM how I got that figure. On your Account page, there's a link for pending credit. Just scroll to the very bottom, and it's there.

I have 236,000 + should be a great bounce...
Official Abuser of Boinc Buttons...
And no good credit hound!
ID: 969660 · Report as offensive
Profile ccappel
Avatar

Send message
Joined: 27 Jan 00
Posts: 362
Credit: 1,516,412
RAC: 0
United States
Message 969791 - Posted: 11 Feb 2010, 16:23:41 UTC - in response to Message 969330.  
Last modified: 11 Feb 2010, 16:25:07 UTC

Each time work is returned, the quota is doubled.

or have an option for impatient people to only process tasks that need to be validated.

That one gets my vote.

But if that option was available and everyone did it, then there would end up being not enough tasks waiting to be validated (until the task got to high priority mode). Someone has to go first in the wingman pair.
"Life is a tragedy for those who feel, and a comedy for those who think."

"I never get into an argument that I cannot win."
ID: 969791 · Report as offensive
Matthew S. McCleary
Avatar

Send message
Joined: 9 Sep 99
Posts: 121
Credit: 2,288,242
RAC: 0
United States
Message 969808 - Posted: 11 Feb 2010, 17:41:01 UTC - in response to Message 969660.  


I have 236,000 + should be a great bounce...


I hear you. I'm at 81k now. Looking forward to the logjam breaking.
ID: 969808 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 969827 - Posted: 11 Feb 2010, 18:33:26 UTC - in response to Message 969791.  
Last modified: 11 Feb 2010, 18:34:01 UTC

Each time work is returned, the quota is doubled.

or have an option for impatient people to only process tasks that need to be validated.

That one gets my vote.

But if that option was available and everyone did it, then there would end up being not enough tasks waiting to be validated (until the task got to high priority mode). Someone has to go first in the wingman pair.

Actually, wouldn't that mean that the second WU couldn't be issued until it "needed validation" and that doesn't happen until the first one is done?
ID: 969827 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 969853 - Posted: 11 Feb 2010, 20:06:46 UTC - in response to Message 969827.  
Last modified: 11 Feb 2010, 20:11:11 UTC

Each time work is returned, the quota is doubled.

or have an option for impatient people to only process tasks that need to be validated.

That one gets my vote.

But if that option was available and everyone did it, then there would end up being not enough tasks waiting to be validated (until the task got to high priority mode). Someone has to go first in the wingman pair.

Actually, wouldn't that mean that the second WU couldn't be issued until it "needed validation" and that doesn't happen until the first one is done?

That's is sorta how I had it worked out in my head when I posted that.
1. A task would be put in a queue for unprocessed work.
2. The task would be assigned to a user.
3. The task would be completed or timed out.
4. If the task was complete put the task in a needs validation queue. Else put the task back in the unprocessed work queue.
5. The task would be assigned to a user for validation processing.
6. The task would be completed or timed out.
7. If the task was complete check the task for validation. Else put the task back in the needs validation queue.
8. If the task fails validation put the task in the needs validation queue.
This would require more back end stuff to happen and in general more monkey motion. Which is probably why it isn't done this way.

EDIT:
To have users select what that would want to process there would probably need to be options in the "Run only the selected applications" settings like
SETI@home Enhanced: yes
SETI@home Enhanced Validation: yes
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 969853 · Report as offensive
1mp0£173
Volunteer tester

Send message
Joined: 3 Apr 99
Posts: 8423
Credit: 356,897
RAC: 0
United States
Message 969860 - Posted: 11 Feb 2010, 21:19:27 UTC - in response to Message 969853.  

Each time work is returned, the quota is doubled.

or have an option for impatient people to only process tasks that need to be validated.

That one gets my vote.

But if that option was available and everyone did it, then there would end up being not enough tasks waiting to be validated (until the task got to high priority mode). Someone has to go first in the wingman pair.

Actually, wouldn't that mean that the second WU couldn't be issued until it "needed validation" and that doesn't happen until the first one is done?

That's is sorta how I had it worked out in my head when I posted that.
1. A task would be put in a queue for unprocessed work.
2. The task would be assigned to a user.
3. The task would be completed or timed out.
4. If the task was complete put the task in a needs validation queue. Else put the task back in the unprocessed work queue.
5. The task would be assigned to a user for validation processing.
6. The task would be completed or timed out.
7. If the task was complete check the task for validation. Else put the task back in the needs validation queue.
8. If the task fails validation put the task in the needs validation queue.
This would require more back end stuff to happen and in general more monkey motion. Which is probably why it isn't done this way.

EDIT:
To have users select what that would want to process there would probably need to be options in the "Run only the selected applications" settings like
SETI@home Enhanced: yes
SETI@home Enhanced Validation: yes

As I read your description, that's pretty much what BOINC does now. If a task validates, it transitions out of BOINC and into the science database. If it doesn't validate, or if it times out, a new result is created and assigned. In fact, that's how we get "new work" when the splitters are off.

In order to make this actually "do" something, you'd have to give those reissued work units priority, and that would have to extend to the BOINC client.

Work scheduling in the client is already quite complex without adding yet another variable -- and without creating "edges" in the algorithm that put deadlines at risk.

The only positive result: people who should be able to figure out that pending credit is just a fact of life would stop fretting over pending credit.

... and if that's actually needed, the simplest, safest way to fix that (and the "old client, 0 credit" issue is to issue three work units, and accept a quorum of two.
ID: 969860 · Report as offensive
Previous · 1 · 2 · 3 · Next

Message boards : Number crunching : Why so much pending credit...


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.