"free" memory and buffers/cache dropping question.

kokoko3k · 2009-11-06 16:04:29

Hi all, from what i understood about "free",
a command output like this:

Gozer bin # free
             total       used       free     shared    buffers     cached
Mem:       2073556    1893500     180056          0     164368    1115292
-/+ buffers/cache:     613840    1459716
Swap:      4104556          0    4104556

Means:
I have: 2073556KB of total ram
I'm using 1893500KB of memory,
from which: 164368KB are used for buffers
and 1115292KB for the system cache.
So the "mandatory" ram i'm using should be: 1893500KB-164368KB-1115292KB=613840KB (which "free" correctly report on the second column / second row).

Now, if i empty/free buffers and caches by using:

echo 3 > /proc/sys/vm/drop_caches

I would expect that my used memory will be == 613840KB (or more, if some buffer/cache is not freed)

But what i get is:

Gozer bin # free
             total       used       free     shared    buffers     cached
Mem:       2073556     589204    1484352          0        300      86064
-/+ buffers/cache:     502840    1570716
Swap:      4104556          0    4104556

It seems that not only i freed buffers and caches, but something more.

Now "free" tells me that
Used memory - buffers - caches = 502840KB
while before dropping caches it said
Used memory - buffers - caches = 613840KB
..and i didn't closed any programs or freed anything else by myself.

So, for what those 613840B-502840B=111000KB (about 100MB!) were used?
Why drop_caches freed that?

Thanks for reading

Last edited by kokoko3k (2009-11-06 19:53:12)

djgera · 2009-11-06 16:48:43

Hi

bad calc: 2073556KB - 164368KB - 1115292KB != 613840KB

correct calc: 1893500KB - 164368KB - 1115292KB = 613840KB

Look at /proc/meminfo, to see where is the memory don't use 3 in drop_caches first use 1 then 2, or in reverse, to observe in details

kokoko3k · 2009-11-06 19:54:09

I edited the post to correct bad calculation, it was just a typo, but the sense of the topic remains.
Anyway i'll look to /proc/meminfo

Last edited by kokoko3k (2009-11-06 19:54:48)

kokoko3k · 2009-11-20 10:03:46

I managed to find the "Culprit"

#sync
#cat /proc/meminfo >/tmp/meminfo.0.synced
#echo 3 > /proc/sys/vm/drop_caches
#cat /proc/meminfo >/tmp/meminfo.3

#cat /tmp/meminfo.0.synced
MemTotal:        2073556 kB
MemFree:           58084 kB
Buffers:            7700 kB
Cached:           934628 kB
SwapCached:        17728 kB
Active:           671748 kB
Inactive:         566980 kB
Active(anon):     203500 kB
Inactive(anon):    92996 kB
Active(file):     468248 kB
Inactive(file):   473984 kB
Unevictable:          68 kB
Mlocked:              68 kB
HighTotal:       1187336 kB
HighFree:           4464 kB
LowTotal:         886220 kB
LowFree:           53620 kB
SwapTotal:       4622940 kB
SwapFree:        4465948 kB
Dirty:                 8 kB
Writeback:             0 kB
AnonPages:        291528 kB
Mapped:            69924 kB
Slab:             525808 kB
SReclaimable:     504400 kB
SUnreclaim:        21408 kB
PageTables:         3800 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     5659716 kB
Committed_AS:    1007028 kB
VmallocTotal:     122880 kB
VmallocUsed:       60216 kB
VmallocChunk:      47604 kB
DirectMap4k:      876536 kB
DirectMap4M:       32768 kB


#cat /tmp/meminfo.3
MemTotal:        2073556 kB
MemFree:         1400944 kB
Buffers:             780 kB
Cached:            63772 kB
SwapCached:        17728 kB
Active:           248044 kB
Inactive:         113032 kB
Active(anon):     203648 kB
Inactive(anon):    92996 kB
Active(file):      44396 kB
Inactive(file):    20036 kB
Unevictable:          68 kB
Mlocked:              68 kB
HighTotal:       1187336 kB
HighFree:         700104 kB
LowTotal:         886220 kB
LowFree:          700840 kB
SwapTotal:       4622940 kB
SwapFree:        4465948 kB
Dirty:                88 kB
Writeback:             0 kB
AnonPages:        291676 kB
Mapped:            69688 kB
Slab:              60808 kB
SReclaimable:      46720 kB
SUnreclaim:        14088 kB
PageTables:         3800 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     5659716 kB
Committed_AS:    1006920 kB
VmallocTotal:     122880 kB
VmallocUsed:       60216 kB
VmallocChunk:      47604 kB
DirectMap4k:      876536 kB
DirectMap4M:       32768 kB

TotalMemory: 2073556
(Before dropping buffers/cache)
free1: 58084
buff1:7700
cache1:934628
mandatory1: tot-free1-buff1-cache1=1073144

(dropped buffers/cache)
free2: 1400944
buff2: 63772
cache2: 780
mandatory2: tot-free2-buff2-cache2=608060

Now the "missing memory"
mandatory1-mandatory2=465084 <-- ?

...which is Slab:
Slab1:525808
Slab2:60808
And Slab1-Slab2=465000
(There are just 84kB of dirty data, i think).

Don't watch to numbers, as buffer and cache are already dropped, but "Slabtop" says the biggest hits are:

#slabtop
  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
 83460  83442  99%    0.13K   2782       30     11128K dentry
 68937  68937 100%    0.59K   2554       27     40864K ext4_inode_cache

Is that normal to have 450MB+ of slab cache allocated?

djgera · 2009-11-20 16:22:00

Yes. Is a type of cache. If you need to know where is used in details, look at slabtop

kokoko3k · 2009-11-20 18:37:06

yep, i already used.
dentry and ext4_inode_cache are on top.
also, echoing 2 to drop_caches free that and it seems that performances are not worse, at least in starting (again) heavy applications like firefox, i think because disk buffers are still in memory.
So I ended up in freeing it every hour via cron, but i really don't know if it is a good idea or not.

tavianator · 2009-11-20 18:41:20

You shouldn't need to manually drop your caches. Linux will free them itself if something needs more memory, and they can't possibly be hurting your performance.

djgera · 2009-11-20 18:46:48

free for what? This is managed by kernel in an intelligent way. If some application needs memory, most old caches are freed automatically.

If your system have much memory is better to use it, rather than having unused!

kokoko3k · 2009-11-21 20:25:30

Yes of course, but i doubt the kernel knows what i'm about to do with my pc, nor does it learn from my daily usage (i think).
For example, buffer memory grows when i'm watching a movie with mplayer; yes, copying that file when i'm done will be an extremly fast operation.. but i never did it; probably i will delete it.
If that file is very big, probably all the system will be slow to react from that moment on, like it was just powered on.
I know all that has nothing to do with slab, but my point is that even if kernel tries to use all the memory the best as it can (being somehow 'blind'), it's behaviour may be not perfect from my subjective point of view.
Infact, in /proc/sys/vm/ there's some knobs and sliders which helps tuning the way linux manages memory to the user needs, but i didn't find any that is directly related to slab (although i have still to study/undesrtand well what exactly it is used for and how)

By now, i noticed that clearing slab periodically helps the system to stay reactive when launching heavy applications like openoffice, firefox or even konqueror again and again; basically because they remains buffered.

That said, probably i will face the drawback and i really wonder what it will be.

Arch Linux

#1 2009-11-06 16:04:29

"free" memory and buffers/cache dropping question.

#2 2009-11-06 16:48:43

Re: "free" memory and buffers/cache dropping question.

#3 2009-11-06 19:54:09

Re: "free" memory and buffers/cache dropping question.

#4 2009-11-20 10:03:46

Re: "free" memory and buffers/cache dropping question.

#5 2009-11-20 16:22:00

Re: "free" memory and buffers/cache dropping question.

#6 2009-11-20 18:37:06

Re: "free" memory and buffers/cache dropping question.

#7 2009-11-20 18:41:20

Re: "free" memory and buffers/cache dropping question.

#8 2009-11-20 18:46:48

Re: "free" memory and buffers/cache dropping question.

#9 2009-11-21 20:25:30

Re: "free" memory and buffers/cache dropping question.

Board footer