Using multiple CPU cores to crunch = higher RAM requirement?

Message boards : Number crunching : Using multiple CPU cores to crunch = higher RAM requirement?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Mahoujin Tsukai
Volunteer tester
Avatar

Send message
Joined: 21 Jul 07
Posts: 147
Credit: 2,204,402
RAC: 0
Singapore
Message 642082 - Posted: 16 Sep 2007, 5:11:10 UTC

If I'm not wrong, SETI@Home needs 64MB of RAM to run.

But when multiple CPU cores are used, does the RAM requirement get multiplied by the no. of CPU cores being utilized for SETI@home?

eg)
single core being used = 64MB RAM needed
2 cores being used = 128MB RAM needed
4 cores being used = 256MB RAM needed

Am I correct here?
ID: 642082 · Report as offensive
Profile Dirk Sadowski
Volunteer tester

Send message
Joined: 6 Apr 07
Posts: 7105
Credit: 147,663,825
RAC: 5
Germany
Message 642089 - Posted: 16 Sep 2007, 5:22:45 UTC
Last modified: 16 Sep 2007, 6:20:15 UTC



My QX6700-2.66 @ 3.14 GHz use per Core ~ 29.000 KB for SETI@home (opt. app)..
Completely he take around 300 to 320 MB..


EDIT:
So for WinXP it must be maybe at least 2 x 512 MB (Dual-Channel)..
WinVista maybe more.. it need more RAM, I heard..
..for your complete 'only crunch'- system..


ID: 642089 · Report as offensive
Profile AlphaLaser
Volunteer tester

Send message
Joined: 6 Jul 03
Posts: 262
Credit: 4,430,487
RAC: 0
United States
Message 642108 - Posted: 16 Sep 2007, 5:59:29 UTC

Yes, that's correct, as each CPU core runs a separate and independent instance of the science application.
ID: 642108 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 642326 - Posted: 16 Sep 2007, 14:02:16 UTC - in response to Message 642082.  

If I'm not wrong, SETI@Home needs 64MB of RAM to run.

But when multiple CPU cores are used, does the RAM requirement get multiplied by the no. of CPU cores being utilized for SETI@home?

eg)
single core being used = 64MB RAM needed
2 cores being used = 128MB RAM needed
4 cores being used = 256MB RAM needed

Am I correct here?

Now each instance of SETI@home needs only about 32MB of RAM on most systems while running. This applies to stock 5.27 as well as optimized apps, the chirping routine which used the additional 32MB is unlikely to be chosen as newer optimized routines which don't need large arrays are also faster on almost any system. Each instance temporarily uses up to about 96MB during the period just after startup when it tests which functions are optimal for the host, but that doesn't last long.

You are correct that BOINC runs separate instances for each WU being crunched, it just adds up to less.
                                                                Joe
ID: 642326 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21514
Credit: 7,508,002
RAC: 20
United Kingdom
Message 642357 - Posted: 16 Sep 2007, 14:42:12 UTC - in response to Message 642326.  

... when multiple CPU cores are used, does the RAM requirement get multiplied by the no. of CPU cores being utilized for SETI@home?

eg)
single core being used = 64MB RAM needed
2 cores being used = 128MB RAM needed
4 cores being used = 256MB RAM needed ...

Now each instance of SETI@home needs only about 32MB of RAM on most systems while running. This applies to stock 5.27 as well as optimized apps, the chirping routine which used the additional 32MB is unlikely to be chosen as newer optimized routines which don't need large arrays are also faster on almost any system. Each instance temporarily uses up to about 96MB during the period just after startup when it tests which functions are optimal for the host, but that doesn't last long.

You are correct that BOINC runs separate instances for each WU being crunched, it just adds up to less...

Are the multiple instances spawned in such a way that the code part of each instance is reused? (That is, only one copy of the code is kept in memory and just new data segments are allocated for each new instance?...)

Regards,
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 642357 · Report as offensive
Josef W. Segur
Volunteer developer
Volunteer tester

Send message
Joined: 30 Oct 99
Posts: 4504
Credit: 1,414,761
RAC: 0
United States
Message 642411 - Posted: 16 Sep 2007, 16:42:42 UTC - in response to Message 642357.  

...BOINC runs separate instances for each WU being crunched, it just adds up to less...

Are the multiple instances spawned in such a way that the code part of each instance is reused? (That is, only one copy of the code is kept in memory and just new data segments are allocated for each new instance?...)

Regards,
Martin

I don't know, but apps are launched with a CreateProcess() call on Windows, so it certainly could recognize when that's another instance of something already running. Perhaps someone with a Multi-CPU system could check to see if instances 2 through n actually use a little less? The stock app FFTW implementation is in a DLL, that should definitely be shared.
                                                                Joe
ID: 642411 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 642424 - Posted: 16 Sep 2007, 16:59:04 UTC - in response to Message 642357.  
Last modified: 16 Sep 2007, 17:23:49 UTC


Are the multiple instances spawned in such a way that the code part of each instance is reused? (That is, only one copy of the code is kept in memory and just new data segments are allocated for each new instance?...)

Regards,
Martin


Hopefully ---

I haven't looked at that particular code, but If you are really asking if there are duplicate programs in memory I would say generally no. I would 'guess' that there are registered classes involved... meaning OS will have some generic parts, and there'll be some unique 'per instance' parts.

In a general sense even old fashioned DLL's are shared and only one instance in memory at a time ...old school style. But they cannot look like multiple separate instances which is what we want. We're hopefully talking classes which are 'new school'.

The active class code of say a particular class of thread 'KWSN_BlahBlah" is in memory and may have many instances (4 on a quad). The many (4 on a quad) instances [each] have their own data. The thread (code) and instance (particular instantiation's data) are defined separately and so you should be able to have as many instantiations and all will use the same original class definition....(single copy of executable code )

So that's one copy of the [thread] Code... and 4 copies of the thread data .. on a quad.
... Which when combined (1 Class x 4 instantiations) make 4 instantiations of a process...


That's the theory... the actual implementation is OS ( Windows :S), Language (C++) and programmer (dudes) dependant. will vary in effectiveness , efficiency and performance by brand version (of dudes :P).

@Joe -- Cross Post !!!


"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 642424 · Report as offensive
John McLeod VII
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 15 Jul 99
Posts: 24806
Credit: 790,712
RAC: 0
United States
Message 642460 - Posted: 16 Sep 2007, 17:27:04 UTC

Sharing code for different data running on identical applications is entirely an OS function. I believe that all relatively modern OSs do code sharing. Even to the extent that if it is in the L2 or L1 cache the code is shared.


BOINC WIKI
ID: 642460 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 642464 - Posted: 16 Sep 2007, 17:29:41 UTC - in response to Message 642460.  
Last modified: 16 Sep 2007, 17:35:01 UTC

Sharing code for different data running on identical applications is entirely an OS function. I believe that all relatively modern OSs do code sharing. Even to the extent that if it is in the L2 or L1 cache the code is shared.

Yes, though it is an OS function selected by the application [I the application hereby instantiate this class], not selected by the OS intelligently. The fact that cache becomes shared is a [desirable] side effect. [ a side effect of needing the same code that's already in the cache :D]



"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 642464 · Report as offensive
Profile dnolan
Avatar

Send message
Joined: 30 Aug 01
Posts: 1228
Credit: 47,779,411
RAC: 32
United States
Message 642522 - Posted: 16 Sep 2007, 18:42:10 UTC - in response to Message 642411.  
Last modified: 16 Sep 2007, 18:43:35 UTC


Perhaps someone with a Multi-CPU system could check to see if instances 2 through n actually use a little less? The stock app FFTW implementation is in a DLL, that should definitely be shared.


Looking at my stock Q6600, running 2.4V of the SSSE optimized app, it's showing instances 1 - 4 fluctuating between high 27,000K to mid 28,000K range of mem usage in process manager.

-Dave

[Edit} and no core seems to be using notably less or more than any of the others...
ID: 642522 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 642528 - Posted: 16 Sep 2007, 18:47:06 UTC - in response to Message 642522.  
Last modified: 16 Sep 2007, 18:48:17 UTC


Looking at my stock Q6600, running 2.4V of the SSSE optimized app, it's showing instances 1 - 4 fluctuating between high 27,000K to mid 28,000K range of mem usage in process manager.

-Dave

[Edit} and no core seems to be using notably less or more than any of the others...


Yes, They're probably all [each] data plus a class identifier ...

"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 642528 · Report as offensive
Profile ML1
Volunteer moderator
Volunteer tester

Send message
Joined: 25 Nov 01
Posts: 21514
Credit: 7,508,002
RAC: 20
United Kingdom
Message 642534 - Posted: 16 Sep 2007, 18:51:27 UTC - in response to Message 642464.  
Last modified: 16 Sep 2007, 18:52:07 UTC

Sharing code for different data running on identical applications is entirely an OS function. I believe that all relatively modern OSs do code sharing. Even to the extent that if it is in the L2 or L1 cache the code is shared.

Yes, though it is an OS function selected by the application [I the application hereby instantiate this class], not selected by the OS intelligently. The fact that cache becomes shared is a [desirable] side effect. [ a side effect of needing the same code that's already in the cache :D]

Exactly so, hence the question as to whether that is actually being done. This should be of greater interest now that multi-core CPUs are quickly becoming more commonplace...

Anyone know the Boinc code well enough to comment?

Happy crunchin',
Martin

See new freedom: Mageia Linux
Take a look for yourself: Linux Format
The Future is what We all make IT (GPLv3)
ID: 642534 · Report as offensive
Profile jason_gee
Volunteer developer
Volunteer tester
Avatar

Send message
Joined: 24 Nov 06
Posts: 7489
Credit: 91,093,184
RAC: 0
Australia
Message 642541 - Posted: 16 Sep 2007, 19:04:12 UTC - in response to Message 642534.  
Last modified: 16 Sep 2007, 19:15:13 UTC


Anyone know the Boinc code well enough to comment?

Happy crunchin',
Martin


Martin, I suggest you take a look at the stock code. You would quickly see how OS dependant it is. it is relatively easy to follow. Just think of it as ordinary C++ code on crack. the Winmain() starts good, then passes to main() then that's it! really simple... All the wierd #ifdefs just decide which version platform you are on (what facilities are available)

[What I am Trying to say is that the stock code tries to do things in the 'Proper' way for each defined platform that it encounters.)
"Living by the wisdom of computer science doesn't sound so bad after all. And unlike most advice, it's backed up by proofs." -- Algorithms to live by: The computer science of human decisions.
ID: 642541 · Report as offensive
Profile sterling0466
Avatar

Send message
Joined: 5 Oct 00
Posts: 204
Credit: 742,621
RAC: 0
United States
Message 642559 - Posted: 16 Sep 2007, 19:27:28 UTC
Last modified: 16 Sep 2007, 19:28:09 UTC

With my QX6800 Core2Extreme I am running on an Intel D975XBX2 Mobo and 4 Gigs. of DDR2 Dual Channel Kingston Memory. According to my Task Manager in MS Windows Vista Ultimate, here is how the memory usage is broken down:

Seti@Home (Core 1) 42,075k
Seti@Home (Core 2) 42,840k
Seti@Home (Core 3) 42,024k
Seti@Home (Core 4) 42,580k
Boinc.Exe 18,264k
BoincMgr 4.744k

Total for all Seti Apps: 192,527k


ID: 642559 · Report as offensive
Profile Carlos
Volunteer tester
Avatar

Send message
Joined: 9 Jun 99
Posts: 30935
Credit: 57,275,487
RAC: 157
United States
Message 642623 - Posted: 16 Sep 2007, 21:35:10 UTC - in response to Message 642559.  
Last modified: 16 Sep 2007, 21:36:05 UTC

With my QX6800 Core2Extreme I am running on an Intel D975XBX2 Mobo and 4 Gigs. of DDR2 Dual Channel Kingston Memory. According to my Task Manager in MS Windows Vista Ultimate, here is how the memory usage is broken down:

Seti@Home (Core 1) 42,075k
Seti@Home (Core 2) 42,840k
Seti@Home (Core 3) 42,024k
Seti@Home (Core 4) 42,580k
Boinc.Exe 18,264k
BoincMgr 4.744k

Total for all Seti Apps: 192,527k




Good info. Did the same check on my system to see what I am using. My processor is the QX6700 on an ASUS striker extreme MB with 2 Gig of DDR2 1066 Corsair XMS2. Running XP pro.
Here's the memory usage from task manager

Seti@Home (KWSN, core 1) 29,500
Seti@Home (KWSN, core 2) 29,800
Rosetta Beta (Core 3) 9,500
Rosetta Beta (Core 4) 9,900
Boinc.exe 19,000
BoincMgr 9,500

Total for all Seti Apps: 107,200
ID: 642623 · Report as offensive

Message boards : Number crunching : Using multiple CPU cores to crunch = higher RAM requirement?


 
©2025 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.