A little memory usage advice for Linux/Fedora 29, enhanced I/O usage

Message boards : Number crunching : A little memory usage advice for Linux/Fedora 29, enhanced I/O usage
Message board moderation

To post messages, you must log in.

AuthorMessage
The_Matrix
Volunteer tester

Send message
Joined: 17 Nov 03
Posts: 405
Credit: 5,792,829
RAC: 131
Germany
Message 1987985 - Posted: 30 Mar 2019, 14:50:25 UTC
Last modified: 30 Mar 2019, 14:58:13 UTC

Hey, weathers going every day better here, but still crunching. :)

Several machines here run only with 4 gigabytes of ram, an there is
a many I/O usage on every machine here.

Sometimes the machines got stuck, first i tried with "preload" not so
good effects on this.

Next i'll tried to cut down "/tmp" folder usage and edited "/etc/fastab" to
use only 1GB of ram. not so bad, BUT...even better was to install
the tool "memcached" an reboot the machines.

Try this. Only an advice NO rule ! U won't be dissapointed.

Greetings , have a nice day !
ID: 1987985 · Report as offensive
Profile Tom M
Volunteer tester

Send message
Joined: 28 Nov 02
Posts: 3371
Credit: 173,310,837
RAC: 780,062
United States
Message 1988050 - Posted: 30 Mar 2019, 22:06:34 UTC - in response to Message 1987985.  

Hey, weathers going every day better here, but still crunching. :)

Several machines here run only with 4 gigabytes of ram, an there is
a many I/O usage on every machine here.

Sometimes the machines got stuck, first i tried with "preload" not so
good effects on this.

Next i'll tried to cut down "/tmp" folder usage and edited "/etc/fastab" to
use only 1GB of ram. not so bad, BUT...even better was to install
the tool "memcached" an reboot the machines.

Try this. Only an advice NO rule ! U won't be dissapointed.

Greetings , have a nice day !


Ok, so basically you have a Linux/Fedora machine with 4 GB of ram that you wanted to dedicate to Boinc crunching but not spend much money for the ram. So you have come up with some nice tuning/changes so it will reduce paging out to the HD when it is running?
The goal being to get it to run reliably in that small amount of ram.

Congratulations. And thank you for publishing that.

Tom
I will stop procrastinating tomorrow.
\\// Live Long & Prosper (starting tomorrow ;)
ID: 1988050 · Report as offensive
Cruncher-American Crowdfunding Project Donor*Special Project $75 donorSpecial Project $250 donor

Send message
Joined: 25 Mar 02
Posts: 1497
Credit: 328,114,970
RAC: 167,710
United States
Message 1988255 - Posted: 1 Apr 2019, 3:57:34 UTC

Mem usage depends on the number of threads you run (not much of a surprise, that).
My 40 thread (dual i5-2680v2) machines with dual GTX 1080s running 3 WUs each in // (total of 38 threads running SETI in WIN 7) use about 5GB of RAM according to Task Manager. So your 4gb should be sufficient for all but the largest crunchers.
ID: 1988255 · Report as offensive
The_Matrix
Volunteer tester

Send message
Joined: 17 Nov 03
Posts: 405
Credit: 5,792,829
RAC: 131
Germany
Message 1988277 - Posted: 1 Apr 2019, 7:58:28 UTC
Last modified: 1 Apr 2019, 8:07:21 UTC

Additional GOOD news.

Even I cut down this two "drives"

/dev/shm
/tmp

to 512M (512 megabytes) no negative issues are encountered.
Truly using "memcached" at this time TOO AND the SWAP drive is enabled.

Especialy I discovered some "tools" I have removed:

sudo dnf remove tracker* (about 1gigabytes memory traffic). Nautilus won't be avaiiable no more.
ID: 1988277 · Report as offensive
The_Matrix
Volunteer tester

Send message
Joined: 17 Nov 03
Posts: 405
Credit: 5,792,829
RAC: 131
Germany
Message 1988279 - Posted: 1 Apr 2019, 8:01:42 UTC - in response to Message 1988277.  
Last modified: 1 Apr 2019, 8:02:17 UTC

sorry
ID: 1988279 · Report as offensive
Juha
Volunteer tester

Send message
Joined: 7 Mar 04
Posts: 388
Credit: 1,823,371
RAC: 159
Finland
Message 1988324 - Posted: 1 Apr 2019, 16:35:40 UTC

if by "memcached" you are referring to this memcached then I don't understand how that would help you.

The purpose of memcached is to keep frequently used data in RAM instead of fetching it from e.g. database. So it's consuming RAM and if I understand correctly you've been trying to do the exact opposite of that.

FWIW, BOINC client side parts don't use memcached. Server parts could use it for caching frequently shown webpages. (If memcached isn't used then the server uses on-disk caching system.)

As for shrinking filesystems. For tmpfs type filesystems you don't set its size but how large the filesystem can grow. tmpfs uses only as much RAM as the files in the filesystem need, not more.
ID: 1988324 · Report as offensive
The_Matrix
Volunteer tester

Send message
Joined: 17 Nov 03
Posts: 405
Credit: 5,792,829
RAC: 131
Germany
Message 1988629 - Posted: 3 Apr 2019, 21:40:16 UTC
Last modified: 3 Apr 2019, 22:04:34 UTC

ok, my drives here are very slow. That must be the point in not using
drive cache functions instead of RAM caching in any point.

Do you know Thoshiba 320GB sata-II 2,5" 5400rpm ?

btw. Boinc still runs on computers using ram,cpu and data storage.
ID: 1988629 · Report as offensive

Message boards : Number crunching : A little memory usage advice for Linux/Fedora 29, enhanced I/O usage


 
©2019 University of California
 
SETI@home and Astropulse are funded by grants from the National Science Foundation, NASA, and donations from SETI@home volunteers. AstroPulse is funded in part by the NSF through grant AST-0307956.