Questions and Answers :
Macintosh :
Increase the number of concurrent tasks beyond 8
Message board moderation
Author | Message |
---|---|
Charles Long Send message Joined: 31 Jul 13 Posts: 3 Credit: 1,856,372,616 RAC: 0 |
I'm having trouble getting the Seti client to launch more than 8 concurrent tasks at a time even when there are twice that number of available processors. This problem only exists with the OS X Seti client; the Linux clients that I am using run approximately one task per CPU. I am running the stand alone version. 1) This exists between versions of OS X, having tested 10.9.2 and 10.7.4 2) Both versions of Boinc 7.2.42 and 7.3.19 3) With and without GPU processing enabled 4) Global settings set to 0 or 100 of multiprocessors and 0 or 100% of multiprocessors 5) Setting app config to: <app> <name>setiathome_v7</name> <max_concurrent>16</max_concurrent> <gpu_versions> <gpu_usage>.5</gpu_usage> <cpu_usage>.5</cpu_usage> </gpu_versions> </app> Or <app_version> <app_name>setiathome_v7</app_name> <avg_ncpus>.5</avg_ncpus> </app_version> I've seen a couple threads complaining about this same issue and no resolution. I'm beginning to believe 8 concurrent threads is either hard coded or all options to change this are ignored. I have tested other projects, such as asteroidsathome with all of the same global settings (and none of the app settings) and I get a full 16 concurrent tasks on 16 core systems. As that this is meant as a positive way to load test systems, only running on half the cores is counter to that goal. If there is no fix, I'll have to switch away from implementing Seti in our datacenters across the board. |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
What are your preferences set to for On multiprocessors, use at most N of the processors and On multiprocessors, use at most X% of the processors? If need be, post the contents of your global_prefs.xml file and if it exists the global_prefs_override.xml file. These can be found in the data directory, default on OS X at /Library/Application Support/BOINC Data/ |
Charles Long Send message Joined: 31 Jul 13 Posts: 3 Credit: 1,856,372,616 RAC: 0 |
There are no settings as outlined in your reply (see below), anything similar is set to 0. Also, the globalprefs is the same for both projects (below), it's only setiathome that is stuck on running only 8. I am using the stand alone boinc client. global_prefs.xml <global_preferences> <source_project>http://setiathome.berkeley.edu/</source_project> <source_scheduler>http://setiboinc.ssl.berkeley.edu/sah_cgi/cgi</source_scheduler> <mod_time>1403420747</mod_time> <run_on_batteries>0</run_on_batteries> <run_if_user_active>1</run_if_user_active> <run_gpu_if_user_active>1</run_gpu_if_user_active> <idle_time_to_run>3</idle_time_to_run> <suspend_if_no_recent_input>0</suspend_if_no_recent_input> <suspend_cpu_usage>25</suspend_cpu_usage> <leave_apps_in_memory>0</leave_apps_in_memory> <cpu_scheduling_period_minutes>60</cpu_scheduling_period_minutes> <max_cpus>0</max_cpus> <max_ncpus_pct>0</max_ncpus_pct> <cpu_usage_limit>0</cpu_usage_limit> <disk_max_used_gb>0</disk_max_used_gb> <disk_min_free_gb>0.1</disk_min_free_gb> <disk_max_used_pct>90</disk_max_used_pct> <disk_interval>60</disk_interval> <vm_max_used_pct>75</vm_max_used_pct> <ram_max_used_busy_pct>50</ram_max_used_busy_pct> <ram_max_used_idle_pct>90</ram_max_used_idle_pct> <work_buf_min_days>0.1</work_buf_min_days> <work_buf_additional_days>0.5</work_buf_additional_days> <confirm_before_connecting>0</confirm_before_connecting> <hangup_if_dialed>0</hangup_if_dialed> <max_bytes_sec_down>0</max_bytes_sec_down> <max_bytes_sec_up>0</max_bytes_sec_up> <daily_xfer_limit_mb>0</daily_xfer_limit_mb> <daily_xfer_period_days>0</daily_xfer_period_days> <dont_verify_images>0</dont_verify_images> </global_preferences> |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
There are no settings as outlined in your reply (see below), anything similar is set to 0. Why? <max_cpus>0</max_cpus> <max_ncpus_pct>0</max_ncpus_pct> With this you only limit BOINC. Unless you do have a global_prefs_override.xml file (local preferences), you're now telling BOINC not to use any CPU cores. These settings are the first settings that BOINC will look at, any next settings in files as app_info.xml or app_config.xml isn't going to magically add CPU cores if you told BOINC already that essentially there are none for it to use. I don't know why it's using 8, but you should be glad that it does at least do that. As I posted elsewhere in another thread: As it's supposed to work, barring any bugs and errors possible in the various codes: |
Charles Long Send message Joined: 31 Jul 13 Posts: 3 Credit: 1,856,372,616 RAC: 0 |
First, allow me to correct myself:
This was answered in the original post:
As to the global preferences, according to the settings page, zero means 'no restriction' in all the cases. I have tested both 0 and 100% in multiple instances, as stated above and as it is the default, and there was no change in the number of tasks, no statement as to max available CPU in the stdoutdae.txt. In addition, my Linux hosts, which use the same global settings, do not have a 8 task limitation and use the all available cores; I have tested multiple systems with different numbers of cores and this is always the case. That and other projects on OS X, such as asteroidsathome, which use the same global settings, they also use all available cores. This limitation is exclusive to the setiathome OS X client. Oh, and to add, if I add another project on the same OS X host, it picks up the remaining cores. |
rob smith Send message Joined: 7 Mar 03 Posts: 22202 Credit: 416,307,556 RAC: 380 |
This looks like an error on the prefs page - 0 in this case does mean zero, not "unrestricted" - there are folks who don't want to use their CPUs for crunching. Set it to 100% and all should be happy. Bob Smith Member of Seti PIPPS (Pluto is a Planet Protest Society) Somewhere in the (un)known Universe? |
baron_iv Send message Joined: 4 Nov 02 Posts: 109 Credit: 104,905,241 RAC: 0 |
I'm actually having the same problem. I have a 6-core CPU (12 cores with hyper threading) and the computer runs on all cores in both linux and windows, but when I fire up mac, it only uses 8 cores. I double checked my website preferences and it's set to use 99 CPUs, so I don't think that's the limiting issue. It seems to be something that ONLY happens with SETI and ONLY with the mac version. Very odd, really. -baron_iv Proud member of: GPU Users Group |
Zalster Send message Joined: 27 May 99 Posts: 5517 Credit: 528,817,460 RAC: 242 |
Baron, You are using a Mac Tower, Mac Pro, Hackitosh or are you running Mac on Virtual computer? I run a MacBook Pro and it recognizes all 8 of my cores (4 actual, 4 Hyperthread) I wonder why that is. Definitely interesting question. Zalster |
Chris Adamek Send message Joined: 15 May 99 Posts: 251 Credit: 434,772,072 RAC: 236 |
Hmm, wonder if this can be worked around by using a second instance of boinc... Chris |
OzzFan Send message Joined: 9 Apr 02 Posts: 15691 Credit: 84,761,841 RAC: 28 |
Before people start resorting to such solutions, I think it would be best if we heard back from Charles Long on if he ever changed the preferences from 0% to 100% as was suggested. Nor was it made clear if other people were using the correct CPU usage setting as there are two of them, but only one of them is used on current versions of BOINC. |
Chris Adamek Send message Joined: 15 May 99 Posts: 251 Credit: 434,772,072 RAC: 236 |
Any updates on this? I've seen several folks running new Mac Pro's that would have at least 12 cores. Just curious if anyone else is seeing this problem... Mainly because I'm deciding which processor to get in a new Mac Pro lol Thanks, Chris |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
Questions from the developers: 1. Do other projects get > 8 concurrent threads? 2. Does the client recognize > 8 CPUs (i.e. what does event log say at BOINC start)? Question 1 is to determine if it's a Seti or a BOINC problem. If it only happens at Seti.... |
xcarobx Send message Joined: 24 Oct 14 Posts: 2 Credit: 366,443 RAC: 0 |
I'm having the same problem on my 6 core Xeon hackintosh. (HyperThreading fully functional) It just doesn't matter wether I set number of processors to 99 and cpu utilization to 0 or 100%. The Event Log shows that it recognizes 12 cores. No matter how I change the settings I can't get it to handle more than 8 tasks at a time. Eight tasks have the status "Running" and the rest has the status "Waiting for shared memory". Note that I have 24 GB of RAM and set the disk/memory usage like this: Disk: use at most 50 GB 0 means no restriction Disk: leave free at least 20 GB Values smaller than 0.001 are ignored Disk: use at most 90% of total Tasks checkpoint to disk at most every 60 seconds Swap space: use at most 75% of total Memory: when computer is in use, use at most 50% of total Memory: when computer is not in use, use at most 90% of total |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
...and the rest has the status "Waiting for shared memory". RAM is something different than shared memory. From Configuring Shared Memory on Mac OS X : Shared memory is a form of Inter-Process Communication (IPC). Separate programs running on the same computer are able to communicate with each other by reading from and writing to a commonly allocated memory block. The amount of shared memory available on a Mac is configured at boot time. Once the shared memory system has been initiallized it is not possible to change the shared memory configuration. At present the same amount of shared memory is configured on any Mac (about 4MB), regardless of the number of processors or the amount of total memory available. You can view the shared memory settings on your Mac by opening the Terminal application (it is in the Applications -> Utilities folder) and giving the command sysctl -A | grep shm which should produce something like: kern.sysv.shmmax: 4194304 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 8 kern.sysv.shmall: 1024 As of Mac OS X 10.3.9 a relatively simple mechanism has existed for configuring shared memory at boot time. If the file /etc/sysctl.conf exists then the settings in this file are applied at boot time, before the default shared memory settings. To change the shared memory settings you need to be comfortable with creating and editing files using a text editor such as Emacs, vi, or pico. If you are not familiar with a Unix text editor then you can learn to use Emacs from this "Introduction to the Emacs Editor" (PDF format), which includes simple instructions for taking the Emacs tutorial. To change the shared memory settings you also need to be able to give commands as the "root" user on your Mac. If you like, you can follow these instructions on "How to Enable the root Account on Mac OS X". However, you can also simply log in as the "Admin" user, open up the Terminal application (it is in the Applications -> Utilities folder) and giving the command: sudo emacs /etc/sysctl.conf It is likely this file doesn't exist on your system, in which case an empty file will be created. Edit this file so that it contains the lines: kern.sysv.shmmax=16777216 kern.sysv.shmmin=1 kern.sysv.shmmni=128 kern.sysv.shmseg=32 kern.sysv.shmall=4096 These settings increase the amount of shared memory to four (4) times the usual default. These shared memory settings will be applied the next time the computer boots. You can verify the settings after the reboot using the "sysctl -A" command demonstrated above. Here are what the particular shared memory kernel settings mean: shmmax Maximum size of a shared memory segment shmmin Minimum size of a shared memory segment shmmni Maximum number of separate shared memory id's shmseg Maximum number of shared memory segments per user shmall Maximum amount of shared memory (measured in pages). This is generally shmmax divided by 4096. Allowing an arbitrary process to share memory with any other process on the machine would create a serious security problem. Restrictions can be imposed on which processes can use the shared memory segment, based on the userid and groupid of the processes, much as is done for Unix file permissions. See the shmget(1) man page for details |
xcarobx Send message Joined: 24 Oct 14 Posts: 2 Credit: 366,443 RAC: 0 |
SUCCESS ! It seems the problem is that SETI @ HOME uses shared memory. (To a larger extent than other Boinc projects?) With the changes suggested by Ageless I immediately got 12 threads working fully. Maybe this should be added to some FAQ? Maybe the developers should fix this? (Other apps/projects seem to be working just fine:) |
Jord Send message Joined: 9 Jun 99 Posts: 15184 Credit: 4,362,181 RAC: 3 |
I have been in contact over this with the BOINC developers. They say that the present BOINC versions (7.0 and above) do no longer use the shared memory segment, unless the project uses applications that have a fall back to the 6.0 API. It would seem that the science applications for the Mac do exactly that. I still have to contact the project & Lunatics people to find out who built the Mac apps and if they used a fall back to the 6.0 API when they built it. |
©2024 University of California
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.