Optimized Boinc CC v5.3.6 + modifications

Message boards : Number crunching : Optimized Boinc CC v5.3.6 + modifications
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · 4 . . . 5 · Next

AuthorMessage
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 216823 - Posted: 17 Dec 2005, 21:52:47 UTC
Last modified: 17 Dec 2005, 21:53:56 UTC

I built a new optimized BOINC core client from the latest developer source code v5.3.2 (yesterday's night tarball).

Besides my original changes - the configurable RPC port for accessing the client through NAT routers and firewalls, ad the option return_results_immediately, I added two new modifications:

  1. optional CPU affinity (for multiprocessor machines) - the official client can start up to as many processes (WU's) as many processors are available, but (at least under Windows) they are all shared among all of the CPU's and it is the OS that then assigns the processes to the individual CPU's as needed and often frequently re-assigns them in real-time. Depending on the system and the configuration, it can lead to a dramatic performance loss, due to the moving of a task among processors and because of the consequent L1/L2 cache misses. With this new option, you can assure that each WU is being processed with the same CPU (and only with it) from the beginning to the start.
  2. configurable process priority - normally, all project processes are run with the lowest priority 'IDLE'. With the help of the new parameter, you can change the priority to 'NORMAL', 'HIGH', or 'REALTIME'



The influence of the new options depends very much on the configuration of each machine and will not always help you gaining any performance boost. For example, already with S@H Classic, on most my machines S@H calculated in fact faster with the default 'IDLE' priority than with any higher ones. Using the priority 'REALTIME' will almost certainly practically freeze the system - it won't do much more than just processing the WU, but paradoxically it may not have any interesting effect on the calculation performance. Hence - feel free to experiment with new features, but be aware of possible complications, and do not expect too much from it.

As for the speed of the benchmarks - this part was not modified and hence they should be as fast as with my v5.3.1.

More details can be found on my website:
http://boinc.truxoft.com


trux
BOINC software
Freediving Team
Czech Republic
ID: 216823 · Report as offensive
Profile Geek@Play
Volunteer tester
Avatar

Send message
Joined: 31 Jul 01
Posts: 2467
Credit: 86,146,931
RAC: 0
United States
Message 216840 - Posted: 17 Dec 2005, 22:14:09 UTC

Thanks Trux. These are fine programs. Question....does the cpu affinity have any value on say a P4 with hyperthreading? Or do only real dual processors benefit.

Thanks again.


Boinc....Boinc....Boinc....Boinc....
ID: 216840 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 216844 - Posted: 17 Dec 2005, 22:18:15 UTC - in response to Message 216840.  

Question....does the cpu affinity have any value on say a P4 with hyperthreading?

Theoretically, it could have some very minor impact, but I suspect that you won't really notice it. Personally I did not test it on any HT machine, so cannot tell for sure. If you find any difference, please let us know.

trux
BOINC software
Freediving Team
Czech Republic
ID: 216844 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 216847 - Posted: 17 Dec 2005, 22:21:51 UTC

BTW, although apparent, I better mention it too - the changes have impact on all projects run on the BOINC client, not only the S@H application.
trux
BOINC software
Freediving Team
Czech Republic
ID: 216847 · Report as offensive
Hans Dorn
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 2262
Credit: 26,448,570
RAC: 0
Germany
Message 216869 - Posted: 17 Dec 2005, 22:39:54 UTC

Hi,

I'd advise against crunching with REALTIME priority :o)

Regards Hans
ID: 216869 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 216882 - Posted: 17 Dec 2005, 22:50:48 UTC - in response to Message 216869.  

I'd advise against crunching with REALTIME priority :o)

Yeah, that's what I am telling too, but if you have a specially built machine that should do nothing else than BOINC, you may be tempted to try if it does not help. Though, as I wrote, paradoxically it may be contraproductive, but I cannot tell for sure if it is always so. Therefore I kept it as an option and let it on the user if he wants to test it or not :)

trux
BOINC software
Freediving Team
Czech Republic
ID: 216882 · Report as offensive
Profile Tern
Volunteer tester
Avatar

Send message
Joined: 4 Dec 03
Posts: 1122
Credit: 13,376,822
RAC: 44
United States
Message 217128 - Posted: 18 Dec 2005, 4:11:03 UTC - in response to Message 216847.  

BTW, although apparent, I better mention it too - the changes have impact on all projects run on the BOINC client, not only the S@H application.


Trux, thank you for including this statement, and thank you for the explanation on the website. I have used your client myself on a PC that was doing primarily SETI, with great results. I removed it when that PC was changed over to doing Rosetta, as I and many others feel that using an optimized client there is at least bordering on cheating, due to the different method of issuing credits they use.

Until we get a replacement for the fatally-flawed benchmarking process currently in use, these optimized clients are sometimes a desirable thing to have. I would hope that even if the benchmarks later become "unnecessary", you continue with the rest of the features and improvements you put into your clients.
ID: 217128 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 217175 - Posted: 18 Dec 2005, 6:05:23 UTC

Since you have CPU affinity working, any thought of providing the change code to UCB so it could be added to the baseline?
ID: 217175 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 217215 - Posted: 18 Dec 2005, 9:16:34 UTC

Sure, whenever they want, they can have it. I am currently trying to implement another bigger change and as soon as I manage, I'll put the entire source code, or the diffs on my website too. The previous changes were mostly quite trivial, so I did not bother about publishing the code, but the processor affinity part is slightly more complicated. However, as I told, whoever wants it, can have it, or I can post it also here, if demanded.
trux
BOINC software
Freediving Team
Czech Republic
ID: 217215 · Report as offensive
Profile Tern
Volunteer tester
Avatar

Send message
Joined: 4 Dec 03
Posts: 1122
Credit: 13,376,822
RAC: 44
United States
Message 217313 - Posted: 18 Dec 2005, 14:46:45 UTC

Trux, another question - we have a lot of people who are having "connected to localhost" problems. If I understand this correctly, the configurable RPC port could solve that for _some_ of these people, where it is known that 1043 is being "stolen" by another process. (I'm thinking the British Telecom Satellite DSL issue more than Microsoft Outlook, but it might help in either case.)

Am I correct in assuming that if they do NOT want the optimized benchmarks, that if they have a P4 with SSE2, etc., they could just run the "P III no SSE" version, and the benchmarks would be pretty close to the "standard" client for them? Would this "deliberate mismatch" cause any other problems that you know of?
ID: 217313 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 217338 - Posted: 18 Dec 2005, 16:18:27 UTC - in response to Message 217215.  

Sure, whenever they want, they can have it. I am currently trying to implement another bigger change and as soon as I manage, I'll put the entire source code, or the diffs on my website too. The previous changes were mostly quite trivial, so I did not bother about publishing the code, but the processor affinity part is slightly more complicated. However, as I told, whoever wants it, can have it, or I can post it also here, if demanded.

Trux,

If you post the diffs on the developers mailing list they can be included by Dr. Anderson.

If you don't want to do that, you could send them to me and I will ... p.d.buck@comcast.net

The two changes that would be the most friendly are the processor affinity and the port setting ...


ID: 217338 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 217369 - Posted: 18 Dec 2005, 17:37:00 UTC
Last modified: 18 Dec 2005, 17:40:48 UTC

Paul: I am not on the developer list, but will try to create the diffs later today or tomorrow, and post them on my server so you can pick them up and send to the developers.

Bill: I think the PIII version will still return considerably faster benchmarks than the official client. However, I plan to release another version that will use modified benchmarks only for projects with app_info.xml at those it is relatively safe to assume that they are optimized. I do not know though when I manage to finish it - I have really just very few free time to spend on it, so it is quite well possible it won't be before January.
trux
BOINC software
Freediving Team
Czech Republic
ID: 217369 · Report as offensive
Hans Dorn
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 3 Apr 99
Posts: 2262
Credit: 26,448,570
RAC: 0
Germany
Message 217377 - Posted: 18 Dec 2005, 17:49:18 UTC - in response to Message 217369.  


Bill: I think the PIII version will still return considerably faster benchmarks than the official client. However, I plan to release another version that will use modified benchmarks only for projects with app_info.xml at those it is relatively safe to assume that they are optimized. I do not know though when I manage to finish it - I have really just very few free time to spend on it, so it is quite well possible it won't be before January.


Go for it!
I'd like to add some backup projects, but didn't dare because I would be flamed quite a lot because of claimed credit.

Regards Hans

ID: 217377 · Report as offensive
Profile Angus
Volunteer tester

Send message
Joined: 26 May 99
Posts: 459
Credit: 91,013
RAC: 0
Pitcairn Islands
Message 217388 - Posted: 18 Dec 2005, 18:09:43 UTC

I'm noticing that the return_results_immediately 'feature' of the optimized CC doesn't seem to work with SZTAKI.

Anyone else noticing this?
ID: 217388 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 217430 - Posted: 18 Dec 2005, 19:02:46 UTC
Last modified: 18 Dec 2005, 19:17:06 UTC

Paul: I've put the complete source code of the files I modified, on my server. The zip is linked in the download table at http://boinc.truxoft.com/

There are only five files, and most of the changes are very simple and short. All my changes are marked with a comment starting with // TX
trux
BOINC software
Freediving Team
Czech Republic
ID: 217430 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 217458 - Posted: 18 Dec 2005, 20:26:07 UTC
Last modified: 18 Dec 2005, 20:48:38 UTC

Btw, Paul: I've put my additional parameters into the remote_hosts.rpc file just as a simple and temporary hack (for several reasons that are too long to explain them here) - I do not think it should be implemented in this way in the official client, and it was not intended for that purpose. So, of course, if the dev team decides to use some of the modifications, they'll need to ignore this part and do it in their way.
trux
BOINC software
Freediving Team
Czech Republic
ID: 217458 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 217462 - Posted: 18 Dec 2005, 20:33:17 UTC

Return results immediately was removed from the BOINC code as it is hard on the BOINC server side scheduler. I don't think that it will be added again.

Processor affinity was discussed, and put on the to do list, but not very high.

Process priority needs finer granularity - it may be used to try to get BOINC to run with a non-BOINC DC project, and these run at very slightly different priority levels.


BOINC WIKI
ID: 217462 · Report as offensive
Profile Paul D. Buck
Volunteer tester

Send message
Joined: 19 Jul 00
Posts: 3898
Credit: 1,158,042
RAC: 0
United States
Message 217871 - Posted: 19 Dec 2005, 7:06:20 UTC

Trux,

Thanks ...

Not sure this will be take up, but it is/was too good an opportunity to pass up. So, I sent the zip file off to Dr. Anderson (don't want to post a zip to the mailing list.
ID: 217871 · Report as offensive
Pepperammi

Send message
Joined: 3 Apr 99
Posts: 200
Credit: 737,775
RAC: 0
United Kingdom
Message 217932 - Posted: 19 Dec 2005, 10:52:11 UTC
Last modified: 19 Dec 2005, 11:32:19 UTC

Can anyone please give me a hand setting up the cpu affinity? I do have a true duel core proccessor, Pentium D 830.
I've been throught the whole of Trux's site but i'm still a little unsure (also would be great if you could tell me if theres anyway to check the changes have been put into place). I read on the site about putting it into the general pref but I think thats refuring to setting it to use x number of max CPU's, mines set to 2, but then it goes on to say putting it into the remote_hosts.cfg file which I cant find. Reading further up the site said make one but I got the impression this was in relation to using BoincView which I dont use.
I did try making that file (in the boinc directory) and adding the set affinity code as stated. I then checked in task manager each WU affinity and still showed spread accross both so as another check i shut it down again and stuck the set priority code in there at 2 and then run boinc and checked its priority but still normal.
Any help would be grateful. Do I need to wait for the next set of WU to start? nothing is said in boinc manager messages.

By the way thanks for the great work Trux, been using your 5.3.1 client for ages and still gives me a more accurate benchmark than crunch3rs client on my system and these new additions are things that so many can take advantage of. Some even really needed them so I hope these mods will make there way into the official client. Thanks Again

+++++++++++++++++++++++++++++++++++++++EDIT
Never mind I think I got it now. Went throught it again and the same checks as I did earler say its all working.
ID: 217932 · Report as offensive
Profile trux
Volunteer tester
Avatar

Send message
Joined: 6 Feb 01
Posts: 344
Credit: 1,127,051
RAC: 0
Czech Republic
Message 217948 - Posted: 19 Dec 2005, 11:51:49 UTC

Never mind I think I got it now.

C Betteridge - maybe you could tell us what you did wrong not to see the effect immediately, so that others can learn from it, or that I can put better instructions on my website.
trux
BOINC software
Freediving Team
Czech Republic
ID: 217948 · Report as offensive
1 · 2 · 3 · 4 . . . 5 · Next

Message boards : Number crunching : Optimized Boinc CC v5.3.6 + modifications


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.