"this computer has reached a limit on tasks in progress"

Message boards : Number crunching : "this computer has reached a limit on tasks in progress"
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Rowe Family and Friends

Send message
Joined: 25 Dec 00
Posts: 17
Credit: 38,395,231
RAC: 67
New Zealand
Message 1696635 - Posted: 29 Jun 2015, 2:58:35 UTC

I have done a lot of reading on past posts on this topic but the last one with info was from early 2013, so im not sure if it is till valid or not

Here is the message from the log file:
29/06/2015 2:20:10 p.m. | SETI@home | Sending scheduler request: To fetch work.
29/06/2015 2:20:10 p.m. | SETI@home | Requesting new tasks for CPU and NVIDIA GPU
29/06/2015 2:20:13 p.m. | SETI@home | Scheduler request completed: got 0 new tasks
29/06/2015 2:20:13 p.m. | SETI@home | No tasks sent
29/06/2015 2:20:13 p.m. | SETI@home | No tasks are available for SETI@home Enhanced
29/06/2015 2:20:13 p.m. | SETI@home | No tasks are available for SETI@home v7
29/06/2015 2:20:13 p.m. | SETI@home | No tasks are available for AstroPulse v6
29/06/2015 2:20:13 p.m. | SETI@home | No tasks are available for AstroPulse v7
29/06/2015 2:20:13 p.m. | SETI@home | This computer has reached a limit on tasks in progress


And this is the computer in question: http://setiathome.berkeley.edu/show_host_detail.php?hostid=7601615

Now my issue lies in i wish to have 10 days worth of work.
I have set the minimum work buffer to 10.0 and the maximum additional work buffer to .01 as described in another thread here: http://boinc.berkeley.edu/dev/forum_thread.php?id=7451&nowrap=true#43374

But looking at other threads there is a maximum number of work units per CPU core and per GPU. This thread was from 2011 and had 50 per CPU and 400 per GPU.

Now this doesn't seem to hold true as i have 4 CPU cores processing and 2 NVIDIA Graphics cards. I have 100 tasks for the CPU and 200 for the GPU's, so that seems to be 25 per CPU and 100 per graphics card.

The thing is as i want 10 days worth of tasks (computer might be off the internet for a while) i seem to get enough CPU tasks to last most of the 10 days, but the GPUs will eat up their tasks in just over 2 days.

Whats the point in being able to set how much work you want to have it there are some "limits" posed on the CPU and GPU that arent displayed or explained anywhere?

i was under the impression from other threads that bonic learns how your CPU and GPU process tasks and gives out work based on that?
ID: 1696635 · Report as offensive
Profile Zalster Special Project $250 donor
Volunteer tester
Avatar

Send message
Joined: 27 May 99
Posts: 5517
Credit: 528,817,460
RAC: 242
United States
Message 1696637 - Posted: 29 Jun 2015, 3:06:53 UTC - in response to Message 1696635.  
Last modified: 29 Jun 2015, 3:18:04 UTC

There is a limit of 100 CPU task for 1 cpu chip despite however many cores it has

There is a 100 work unit limit for each GPU.

So for your system with 2 GPU and the 1 chip, the max is 300 work units

Why have the limit? It's a safe guard to make sure that someone doesn't download more work then they can possibly do.

What if someone downloaded 4000 work units then decided not to turn on his computer or if the computer had a melt down. Now all those work units are lost.

There is also a time limit as to when the work must be returned. Some computer are much slower than others. So sending them too many work units would result in work not completed in time.

Zalster

Edit..

If I remember correctly, this new system came about accidentally.

In the past it did learn your system and adjusted to that time to completion to match the request for how many days worth of work.

But I think they were trying to fix something and it morphed into what we have now. Others can give you a more detailed explanation.
ID: 1696637 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1696644 - Posted: 29 Jun 2015, 3:51:38 UTC - in response to Message 1696637.  
Last modified: 29 Jun 2015, 3:57:06 UTC

There is a limit of 100 CPU task for 1 cpu chip despite however many cores it has

There is a 100 work unit limit for each GPU.

So for your system with 2 GPU and the 1 chip, the max is 300 work units

Why have the limit? It's a safe guard to make sure that someone doesn't download more work then they can possibly do.

What if someone downloaded 4000 work units then decided not to turn on his computer or if the computer had a melt down. Now all those work units are lost.

There is also a time limit as to when the work must be returned. Some computer are much slower than others. So sending them too many work units would result in work not completed in time.

Zalster

Edit..

If I remember correctly, this new system came about accidentally.

In the past it did learn your system and adjusted to that time to completion to match the request for how many days worth of work.

But I think they were trying to fix something and it morphed into what we have now. Others can give you a more detailed explanation.

The CPU limit is a flat limit of 100 CPU tasks regardless of the number of cores or CPU socks. I wish my dual socket servers were granted a limit of 100 tasks per socket like systems with multiple GPUs get.

The limits were imposed to prevent the db server dying. The limits have been enabled at two different time. With the 2nd time being a more or less permanent decision. I don't recall if it was before the first or before the 2nd time. There were ~11,000,000 results out in the field. Which at that point the db server comes to a grinding halt and everything crashes on a regular basis.

The first time the limits were put in place the limit of 50 tasks per CPU core & 400 total for GPU was used back in 2010-2011.

Many other projects also implement limits.
Moo! 250 tasks
PrimeGrid 100 tasks
Collatz 100 tasks
Milkyway has a more complex limit limit system. CPU cores * 3 up to a maximum of 48.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1696644 · Report as offensive
Profile Cactus Bob
Avatar

Send message
Joined: 19 May 99
Posts: 209
Credit: 10,924,287
RAC: 29
Canada
Message 1696648 - Posted: 29 Jun 2015, 4:15:59 UTC

It seems to me that the limit on GPU tasks could be doubled without too much happening that would cause any problems. I know my GPU crunches at way more than twice the CPU rate. When a shutdown goes longer than expected a good portion of crunchers run out of GPU task way before the CPU tasks.

Is there someone who monitors this board that could implement a change on this. Assuming it would not be too difficult to code the change. I think we could triple it and not be problem but doubling to 200 would be a great improvement.

The only downside I see is the users that produces lot of invalids may increase the amount they are doing but I suspect it isn't a huge factor.

Just a thought.
Bob
Sometimes I wonder, what happened to all the people I gave directions to?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SETI@home classic workunits 4,321
SETI@home classic CPU time 22,169 hours
ID: 1696648 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1696660 - Posted: 29 Jun 2015, 4:58:12 UTC - in response to Message 1696648.  

It seems to me that the limit on GPU tasks could be doubled without too much happening that would cause any problems. I know my GPU crunches at way more than twice the CPU rate. When a shutdown goes longer than expected a good portion of crunchers run out of GPU task way before the CPU tasks.

Is there someone who monitors this board that could implement a change on this. Assuming it would not be too difficult to code the change. I think we could triple it and not be problem but doubling to 200 would be a great improvement.

The only downside I see is the users that produces lot of invalids may increase the amount they are doing but I suspect it isn't a huge factor.

Just a thought.
Bob


People have been asking for the limits to increase since they were implemented. So you can guess how effective that is.
The change to the BOINC code was made to prevent systems with mixed GPU vendors from going into GPU starvation. That change lead to the limits change from 100 GPU tasks to 100 * GPUs per vender. Matt or Eric noted the change & decided to let it stand rather than clamp it back down. Then watch for any negative effects.
So now a system with 2 ATI, 2 Nvidia, & an Intel GPU could have 500 GPU tasks in total. If work was requested for all 3 types. When the change was first implemented it was a little broken. In that the server added all GPUs together & set the limit on tasks that way. So you could have those same 5 GPUs & only request work for Nvidia. Which would result in a limit of 500 instead of 200 tasks.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1696660 · Report as offensive
Profile Cactus Bob
Avatar

Send message
Joined: 19 May 99
Posts: 209
Credit: 10,924,287
RAC: 29
Canada
Message 1696664 - Posted: 29 Jun 2015, 5:07:23 UTC

ok, then maybe a stupid question. Is there a way to fake more than 1 GPU? Can I make seti think I have 2 of the same GPU's.

I know you can setup a couple virtual boxes and double fake your computer so to speak but that is more work for me than worth.
99% of the time 100 task per works just fine. Anyways. it always is interesting to see how to circumvent the system.

Bob
Sometimes I wonder, what happened to all the people I gave directions to?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SETI@home classic workunits 4,321
SETI@home classic CPU time 22,169 hours
ID: 1696664 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1696671 - Posted: 29 Jun 2015, 5:55:46 UTC - in response to Message 1696664.  
Last modified: 29 Jun 2015, 5:57:19 UTC

ok, then maybe a stupid question. Is there a way to fake more than 1 GPU? Can I make seti think I have 2 of the same GPU's.

I know you can setup a couple virtual boxes and double fake your computer so to speak but that is more work for me than worth.
99% of the time 100 task per works just fine. Anyways. it always is interesting to see how to circumvent the system.

Bob

The easiest way would be to modify the number of GPUs that that BOINC client reports to the servers. Which would require a code change. The source can be downloaded from the repository.
If you made BOINC think it had twice as many GPUs. This might get weird. An entry in an app_config.xml with something like <gpu_usage>2.0</gpu_usage> may need to be used. However BOINC may have other issues trying to address a nonexistent device.
Normally people just put in small cheap low end GPUs to bump their GPU count & then their main GPU runs through most of the queue. Mixing high and low end GPUs has its own set of problems as well.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1696671 · Report as offensive
OTS
Volunteer tester

Send message
Joined: 6 Jan 08
Posts: 369
Credit: 20,533,537
RAC: 0
United States
Message 1696754 - Posted: 29 Jun 2015, 15:55:14 UTC - in response to Message 1696644.  


The limits were imposed to prevent the db server dying. The limits have been enabled at two different time. With the 2nd time being a more or less permanent decision. I don't recall if it was before the first or before the 2nd time. There were ~11,000,000 results out in the field. Which at that point the db server comes to a grinding halt and everything crashes on a regular basis.



Perhaps it would not be such an issue if the deadline dates more closely matched the maximum number of days of work you can download. 56 days in some cases to do a work unit when you can only download 10 days plus and additional 10 days seems like overkill and adds substantially to the number of work units in the field.

Another possibility would be to limit the number of days of work you can download based on past performance instead of the number of work units based on absolute numbers. Three days total might cover most downtime events and chances are those that download the most work units for a three day period are the least likely to be the ones that download and run.
ID: 1696754 · Report as offensive
Profile HAL9000
Volunteer tester
Avatar

Send message
Joined: 11 Sep 99
Posts: 6534
Credit: 196,805,888
RAC: 57
United States
Message 1696772 - Posted: 29 Jun 2015, 16:32:04 UTC - in response to Message 1696754.  


The limits were imposed to prevent the db server dying. The limits have been enabled at two different time. With the 2nd time being a more or less permanent decision. I don't recall if it was before the first or before the 2nd time. There were ~11,000,000 results out in the field. Which at that point the db server comes to a grinding halt and everything crashes on a regular basis.



Perhaps it would not be such an issue if the deadline dates more closely matched the maximum number of days of work you can download. 56 days in some cases to do a work unit when you can only download 10 days plus and additional 10 days seems like overkill and adds substantially to the number of work units in the field.

Another possibility would be to limit the number of days of work you can download based on past performance instead of the number of work units based on absolute numbers. Three days total might cover most downtime events and chances are those that download the most work units for a three day period are the least likely to be the ones that download and run.

The long deadlines allow a broader range of machines to participate. So I don't see that changing soon. A dynamic deadline based on the performance & history of a machine could be interesting. That would require the BOINC devs to do some work that I imagine they don't see a need to do at this point.

The 2nd solution you mentioned is the basic idea of how the servers are suppose to work. That is where the "Max tasks per day" & "Consecutive valid tasks" in your host details are meant to come into play. In practice that system needs some work. As it ramps up really high & then reduces very little. One of my hosts has a "Max tasks per day" of nearly 10,000 & it does 90-120 tasks a day. If it glitched & started trashing work it wouldn't matter if it had a queue of 100 or 5000. It would still trash around 10,000 tasks. However that isn't the problem the queue limits are meant to resolve.
SETI@home classic workunits: 93,865 CPU time: 863,447 hours
Join the [url=http://tinyurl.com/8y46zvu]BP6/VP6 User Group[
ID: 1696772 · Report as offensive
Profile Rowe Family and Friends

Send message
Joined: 25 Dec 00
Posts: 17
Credit: 38,395,231
RAC: 67
New Zealand
Message 1696794 - Posted: 29 Jun 2015, 18:11:54 UTC

Thanks for the replies

I just sucks that im wanting 10 days of work but am unable to fully get there.

Maybe some of this information could be more clearly displayed somewhere in the bonic client?
ID: 1696794 · Report as offensive

Message boards : Number crunching : "this computer has reached a limit on tasks in progress"


 
©2024 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.