963 Multibeam tasks with "Validation Pending"

Message boards : Number crunching : 963 Multibeam tasks with "Validation Pending"
Message board moderation

To post messages, you must log in.

AuthorMessage
Greg Tippitt
Volunteer tester

Send message
Joined: 19 Apr 04
Posts: 24
Credit: 20,455,837
RAC: 39
United States
Message 1655969 - Posted: 23 Mar 2015, 9:22:46 UTC

When checking on BOINC-Stats, I noticed that I wasn't getting much credit from SETI, so I checked my accounts page on SETI and found that I have 960+ tasks with "Validation Pending". How long should I expect to wait for other machines to run these tasks?

I normally have my BOINC processing scattered across a dozen projects, so that my PCs don't go idle waiting for work when some projects are without tasks to send out. While SETI-AP was down, some of my other projects were down as well, so I installed the SETI Linux optimized apps for SETI@Home V7 for AMD64 with SSE3 and for my pre-Fermi CUDA cards. I have my app_info.xml set up to run setiathome_v7 tasks using both MBv7_7.05r1848_sse3_linux64 on the AMD CPU cores and setiathome_x41g_x86_64-pc-linux-gnu_cuda32 on Nvidia GPU cores. I had a couple of errors at first because I forgot to set the execute permission on the two downloaded programs, but otherwise they have been chugging along eating MB tasks. Some of the completed tasks get credit on occasionally, but most just seem to be sitting with "Validation Pending."

My tasks:

http://setiathome.berkeley.edu/results.php?userid=1865708&offset=800&show_names=0&state=4&appid=
ID: 1655969 · Report as offensive
Grant (SSSF)
Volunteer tester

Send message
Joined: 19 Aug 99
Posts: 13736
Credit: 208,696,464
RAC: 304
Australia
Message 1655977 - Posted: 23 Mar 2015, 9:59:25 UTC - in response to Message 1655969.  

How long should I expect to wait for other machines to run these tasks?

That depends on each WU. Some have a deadline of a few weeks, others a couple of months. Some people take only a few hours to crunch their work, others take weeks.


Some of the completed tasks get credit on occasionally, but most just seem to be sitting with "Validation Pending."

The more work you process, the larger the Validation Pending will be.
My pendings are around 770 at the moment. The big crunchers would have 10's of thousands pending.
From the time you start crunching, or make significant changes to your hardware (add or remove a system, add a new GPU etc) it generally takes 6-8 weeks for your Credit to stabilise around it's nominal value. And that's when there aren't any Seti system issues.
Grant
Darwin NT
ID: 1655977 · Report as offensive
Richard Haselgrove Project Donor
Volunteer tester

Send message
Joined: 4 Jul 99
Posts: 14650
Credit: 200,643,578
RAC: 874
United Kingdom
Message 1655985 - Posted: 23 Mar 2015, 10:21:26 UTC

I've got something over 600 pending. Half of them were completed in the last two days, two-thirds in the last four days - most of those will clear quickly, since the average turnround time for MB is around a day and a half.

But after that, I have a long tail of delayed tasks, stretching back into January. Some will be people deliberately holding as large a cache as possible, others people who have suffered a computer failure - some may be people who thought they'd dip in to see what the project was all about, and decided it wasn't for them. There's nothing you can do to speed up the process, and it's not worth worrying about.

But keep an eye on your results - if a significant proportion turn into 'validation inconclusive' or 'invalid', it could indicate a problem in your machine.
ID: 1655985 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1655999 - Posted: 23 Mar 2015, 11:15:58 UTC
Last modified: 23 Mar 2015, 11:16:33 UTC

The kitties have 5,364 MB v7 in pending.
As well as 565 AP v7.
And for some reason, still 10 AP v6 hanging around.

Nothing that unusual around here.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1655999 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1656004 - Posted: 23 Mar 2015, 11:59:39 UTC

OK this is directed to Matt and the Seti team ... WHY do you set such long timeout for tasks?

Yea I know when I first joined Seti, I had a 24 k modem, sure I wanted to get some tasks when I connected to last me a week until I connected again.

Well times have changed the 4 to 6 week return times are you kidding me? Computers these day are "connected". Be it a traveling man that visits lots of airports, a guy that like to watch videos on uTube. They are either on their home network or on wifi somewhere. Ask yourself "when have I been without internet for 4 days"

Point is if a computer does not connect to Seti in 10 days, what are the chances of getting those 4-6week tasks back from them them? My guess would be in the 0.5% range.

So they just sit in your database waiting for them to timeout.
ID: 1656004 · Report as offensive
kittyman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Jul 00
Posts: 51468
Credit: 1,018,363,574
RAC: 1,004
United States
Message 1656007 - Posted: 23 Mar 2015, 12:23:35 UTC - in response to Message 1656004.  
Last modified: 23 Mar 2015, 12:28:21 UTC

Short answer...
Because there are legions of very old, slow computers that still contribute a significant amount of work to the project.

And the project, I believe Eric in particular, has made it clear that it is not their intention to do anything to disenfranchise anybody who wishes to participate if it is at all possible.

That is why.
"Freedom is just Chaos, with better lighting." Alan Dean Foster

ID: 1656007 · Report as offensive
Greg Tippitt
Volunteer tester

Send message
Joined: 19 Apr 04
Posts: 24
Credit: 20,455,837
RAC: 39
United States
Message 1656011 - Posted: 23 Mar 2015, 12:41:55 UTC

Many thanks to each of you for your answers. I'm just being impatient after seeing my overall daily credit drop by 90% since making changes. I had read about problems with Seti-AP if it tried to run on pre-Fermi Nvidia GPUs. I was a bit paranoid that perhaps I was generating worthless results for SETI-MB, but I was not seeing too many with error or invalid results.

So that they didn't waste bandwith by asking for work when no tasks are available, a week ago, I paused new work fetching for the following projects:

SkyNet-POGS, Asteroids, Lattice, GPUgrid, and Citizen Science Grid DNA tasks.

The daily credit for these remaining projects all increased rapidly as they the projects that gave me new tasks:

POEM, Rosetta, World Community Grid, Enigma, and Milkway.

The highest ranking project that was getting the most CPU work and all GPU work was SETI, but it was showing only a modest increase in daily credit. From past experience, I know Lattice and GPUgrid give me more credit for hour of work, but SETI credit per hour of work is fairly comparable to the other projects for which I ran tasks. I was suspicious of the 90% drop in credit, because Nvidia GPU tasks normally account for 90% of my credit. I wanted to make sure I wasn't wasting those GPU cycles and electricity by generating garbage results for SETI tasks.

As the weather warms up, I must soon stop running my systems because of heat. They have heated my home this winter while running fully busy, but soon I'll be forced to turn them off during the day time because of the heat. While they could still run fully busy, I didn't want to waste the remaining cool days.

Greg
ID: 1656011 · Report as offensive
Keith J. LaGue
Volunteer tester
Avatar

Send message
Joined: 17 May 99
Posts: 59
Credit: 40,441,387
RAC: 0
United States
Message 1656042 - Posted: 23 Mar 2015, 15:07:06 UTC

Many participants contribute for the science aspect of this project...research, development, practical applications etc...

Recognition and credit is farthest most from our minds.


Keifer
ID: 1656042 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 1656050 - Posted: 23 Mar 2015, 15:50:20 UTC - in response to Message 1656004.  

OK this is directed to Matt and the Seti team ... WHY do you set such long timeout for tasks?

Yea I know when I first joined Seti, I had a 24 k modem, sure I wanted to get some tasks when I connected to last me a week until I connected again.

Well times have changed the 4 to 6 week return times are you kidding me? Computers these day are "connected". Be it a traveling man that visits lots of airports, a guy that like to watch videos on uTube. They are either on their home network or on wifi somewhere. Ask yourself "when have I been without internet for 4 days"

So they just sit in your database waiting for them to timeout.


I can answer this for you as I have talked to Dr. Korpela himself about this.

In general, the Project Scientists want anyone, anywhere, to be able to contribute to SETI@home. There are still places in the U.S. that are not connected via broadband, as well as many other areas of the world. People from some economic classes are unable to afford the latest and greatest computers as well.

As such, they are very much pandering to the lowest common denominator so as to not exclude anyone who wishes to join the SETI@home effort. This is a sentiment I fully support the project on.

Point is if a computer does not connect to Seti in 10 days, what are the chances of getting those 4-6week tasks back from them them? My guess would be in the 0.5% range.


I would like to see additional information on your question to make a more informed decision, rather than come to an 'agreeable' answer based off a talking point.
ID: 1656050 · Report as offensive
Profile Brent Norman Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester

Send message
Joined: 1 Dec 99
Posts: 2786
Credit: 685,657,289
RAC: 835
Canada
Message 1656168 - Posted: 23 Mar 2015, 23:02:38 UTC - in response to Message 1656050.  
Last modified: 23 Mar 2015, 23:08:20 UTC

Yea I know of rural issues. I ran a satellite transmitter and network to provide internet to rural communities (until it got taken over, and shortly shut down because they had such a high overhead) which (if they were the lucky few) got a wireless tower to feed the whole community. Then woo hoo a whopping 1Mb/256 connection ... that's for the community, not 1 person. Sure we could and DID provide a LOT more, but satellite space is not cheap, so you get what you pay for.

Ok, back on subject, I'm not saying you Eric, Matt, or anyone else doesn't know what they are doing.

But, on a 10 day cache ... ok a slow computer that takes 4 days computing time to complete a task, that's 14 days to get it back, fine whew. So to get to 6 weeks that is 96 hours out of 1008 hours, not bad, roughly 10% daily computing time.

THAT IS ONE TASK AND THEY HAVE 99 MORE THAT THEY GOT THE SAME DAY. Ok, being conservative, they only got 30, but still ... 29 or 99 lost tasks.


Point is if a computer does not connect to Seti in 10 days, what are the chances of getting those 4-6week tasks back from them them? My guess would be in the 0.5% range.


I think we both don't know the real answer to that question. I would guess only Matt and Eric (and a few others) know that. But hey If they want to email me the database, I will tell you too :)
ID: 1656168 · Report as offensive
OzzFan Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 9 Apr 02
Posts: 15691
Credit: 84,761,841
RAC: 28
United States
Message 1656178 - Posted: 23 Mar 2015, 23:59:55 UTC - in response to Message 1656168.  

But, on a 10 day cache ... ok a slow computer that takes 4 days computing time to complete a task, that's 14 days to get it back, fine whew. So to get to 6 weeks that is 96 hours out of 1008 hours, not bad, roughly 10% daily computing time.

THAT IS ONE TASK AND THEY HAVE 99 MORE THAT THEY GOT THE SAME DAY. Ok, being conservative, they only got 30, but still ... 29 or 99 lost tasks.


Slower hosts are not going to have 99 tasks in a 10 day cache. Remember that the cache is time based, so if the host takes over 240 hours to crunch a single workunit, it will only get a single workunit in its 10 day cache. 120 hours and it will only have 2 tasks in a ten day cache. etc. etc.

These slower systems not clogging up the database as much as you think. It is the faster systems that download several dozen, then the owner loses interest and never bothers to complete them that are a bigger problem (relatively speaking, as I don't think there are many of those either).
ID: 1656178 · Report as offensive
Profile Wiggo
Avatar

Send message
Joined: 24 Jan 00
Posts: 34761
Credit: 261,360,520
RAC: 489
Australia
Message 1656179 - Posted: 24 Mar 2015, 0:04:31 UTC

Only 963 pending MB's?

You're not trying. :-D

Over 1100 here from my 2 rigs. ;-)

Cheers.
ID: 1656179 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1656190 - Posted: 24 Mar 2015, 0:49:57 UTC - in response to Message 1656178.  

But, on a 10 day cache ... ok a slow computer that takes 4 days computing time to complete a task, that's 14 days to get it back, fine whew. So to get to 6 weeks that is 96 hours out of 1008 hours, not bad, roughly 10% daily computing time.

THAT IS ONE TASK AND THEY HAVE 99 MORE THAT THEY GOT THE SAME DAY. Ok, being conservative, they only got 30, but still ... 29 or 99 lost tasks.


Slower hosts are not going to have 99 tasks in a 10 day cache. Remember that the cache is time based, so if the host takes over 240 hours to crunch a single workunit, it will only get a single workunit in its 10 day cache. 120 hours and it will only have 2 tasks in a ten day cache. etc. etc.

These slower systems not clogging up the database as much as you think. It is the faster systems that download several dozen, then the owner loses interest and never bothers to complete them that are a bigger problem (relatively speaking, as I don't think there are many of those either).

I was going to say my slowest host runs about 5-6 days for a normal task & caches 4-5 at a time. So a task it downloads today may not make it back for 20 days. However I do have all of my hosts check in at all of their projects once a day at 2PM (local time). That is mostly so that I get a nice stats graph on each machine.

Shorter limits don't really make any sense for this project. Given none of the data is time sensitive. Any form of host or user reliability metric used on the server side to limit host work would take time to develop & wouldn't make up for reliable hosts that just vanish.
The only real way to effectively reduce the number of pending tasks the OP is seeing would be to further reduce the limits for hosts. Perhaps allowing no cache at all and only the number of tasks for the number of devices in their system. I think many of us remember the days prior to tools, such as SETI Spy, for caching how fun only having 1 task per CPU was.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1656190 · Report as offensive
Profile Donald L. Johnson
Avatar

Send message
Joined: 5 Aug 02
Posts: 8240
Credit: 14,654,533
RAC: 20
United States
Message 1656223 - Posted: 24 Mar 2015, 3:33:24 UTC - in response to Message 1656178.  

But, on a 10 day cache ... ok a slow computer that takes 4 days computing time to complete a task, that's 14 days to get it back, fine whew. So to get to 6 weeks that is 96 hours out of 1008 hours, not bad, roughly 10% daily computing time.

THAT IS ONE TASK AND THEY HAVE 99 MORE THAT THEY GOT THE SAME DAY. Ok, being conservative, they only got 30, but still ... 29 or 99 lost tasks.

Slower hosts are not going to have 99 tasks in a 10 day cache. Remember that the cache is time based, so if the host takes over 240 hours to crunch a single workunit, it will only get a single workunit in its 10 day cache. 120 hours and it will only have 2 tasks in a ten day cache. etc. etc.

These slower systems not clogging up the database as much as you think. It is the faster systems that download several dozen, then the owner loses interest and never bothers to complete them that are a bigger problem (relatively speaking, as I don't think there are many of those either).

There seems to be an unstated assumption here, that all hosts run 24/7 and/or are crunching whenever they are not doing something else. That ain't necessarily so, and that's why some hosts take 2 hours to crunch 1 task, and a similarly-capable host may take 2 days.
Donald
Infernal Optimist / Submariner, retired
ID: 1656223 · Report as offensive
Profile cliff
Avatar

Send message
Joined: 16 Dec 07
Posts: 625
Credit: 3,590,440
RAC: 0
United Kingdom
Message 1656244 - Posted: 24 Mar 2015, 4:15:41 UTC - in response to Message 1656178.  

But, on a 10 day cache ... ok a slow computer that takes 4 days computing time to complete a task, that's 14 days to get it back, fine whew. So to get to 6 weeks that is 96 hours out of 1008 hours, not bad, roughly 10% daily computing time.

THAT IS ONE TASK AND THEY HAVE 99 MORE THAT THEY GOT THE SAME DAY. Ok, being conservative, they only got 30, but still ... 29 or 99 lost tasks.


Slower hosts are not going to have 99 tasks in a 10 day cache. Remember that the cache is time based, so if the host takes over 240 hours to crunch a single workunit, it will only get a single workunit in its 10 day cache. 120 hours and it will only have 2 tasks in a ten day cache. etc. etc.

These slower systems not clogging up the database as much as you think. It is the faster systems that download several dozen, then the owner loses interest and never bothers to complete them that are a bigger problem (relatively speaking, as I don't think there are many of those either).


I've just has a wingman with a cache of 8 thousand WU PLUS, and a 14 day turn around. And as I checked up on the 3 WU we had in common 1 timed out, then another..
So I aborted the 3rd one and got some new work.. Frankly it would seem to me that the system in question should NEVER have been able to get 8k+ tasks in the 1st place, and if a load were lost tasks, the total shouldn't have allowed the WU stack to grow to that size.

Either the operator [anon] or the system is probably defunct, but since its anon there is no way for anyone other than s@H staff to contact the individual.

And I'm pretty damn sure they don't have time to check up on, what may turn out to be aconsiderable number of anon punters with large caches who are defaulting on their workload.

It would be a LOT better if a way was found to enforce a reasonable quota per system and stick to it, no re-sending of 'lost' tasks beyond a strictly defined limit, hopefully well below 8+k:-)

Regards,
Cliff,
Been there, Done that, Still no damm T shirt!
ID: 1656244 · Report as offensive
AndrewM
Volunteer tester

Send message
Joined: 5 Jan 08
Posts: 369
Credit: 34,275,196
RAC: 0
Australia
Message 1656270 - Posted: 24 Mar 2015, 5:28:44 UTC - in response to Message 1656244.  


I've just has a wingman with a cache of 8 thousand WU PLUS, and a 14 day turn around. And as I checked up on the 3 WU we had in common 1 timed out, then another..
So I aborted the 3rd one and got some new work.. Frankly it would seem to me that the system in question should NEVER have been able to get 8k+ tasks in the 1st place, and if a load were lost tasks, the total shouldn't have allowed the WU stack to grow to that size.

Either the operator [anon] or the system is probably defunct, but since its anon there is no way for anyone other than s@H staff to contact the individual.

And I'm pretty damn sure they don't have time to check up on, what may turn out to be aconsiderable number of anon punters with large caches who are defaulting on their workload.

It would be a LOT better if a way was found to enforce a reasonable quota per system and stick to it, no re-sending of 'lost' tasks beyond a strictly defined limit, hopefully well below 8+k:-)

Regards,


Hi cliff. Would you provide a link to the wingman with the "8 thousand WU PLUS" please?

You surely dont mean your wingman with the 8 cpu cores and 2 GPUs and the 1000 odd WUs.
Always nice to read other peoples opinions though.

So Regards,
ID: 1656270 · Report as offensive

Message boards : Number crunching : 963 Multibeam tasks with "Validation Pending"


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.