Arch Linux

jsgt

Member

Registered: 2009-03-16

Posts: 29

System memory leak

For the past few weeks (I don't know exactly when it started but probably after a pacman upgrade) my system has been consuming more memory that never gets freed even after stopping all programs including X. The memory used indicated by free or top/htop does not equal the memory of the running programs.

~ free -h
              total        used        free      shared  buff/cache   available
Mem:           7.5G        3.7G        2.8G         54M        970M        3.5G
Swap:          2.0G        136M        1.9G

And here is a partial screenshot of htop sorted by reserved memory:

After reboot:

~ free -h
              total        used        free      shared  buff/cache   available
Mem:           7.5G        1.3G        5.0G        261M        1.2G        5.7G
Swap:          2.0G          0B        2.0G

What could be causing this?

jsgt

Member

Registered: 2009-03-16

Posts: 29

Re: System memory leak

Bump

onslow77

Member

Registered: 2014-09-21

Posts: 279

Re: System memory leak

Hello!

Please do not "Bump", it is against the forum rules, see here.

Here is a good place to start learning how "ram memory" works in Linux: http://www.linuxatemyram.com/

Regards
Martin

Last edited by onslow77 (2016-09-15 15:00:42)

---

It is advised to follow the How to post guide when posting on the Arch forum. If one consciously jumps over these elementary steps like reading the wiki and providing necessary information about the problem, one can be regarded as a Help Vampire.

jsgt

Member

Registered: 2009-03-16

Posts: 29

Re: System memory leak

Thanks but this is not an issue of Linux using unused memory for caching. Only 970 MB was being used according to "free", 3.7 GB used which is too high after shutting down all programs. The buffers and cache are even higher after the reboot which indicates caching isn't the problem. If I leave my computer on for long enough it starts thrashing. Something is using my memory for non-cache purposes and it isn't released even after I close/stop the top memory consumers--Firefox, MySQL, Apache, etc.

loqs

Member

Registered: 2014-03-06

Posts: 17,333

Re: System memory leak

https://bbs.archlinux.org/viewtopic.php?id=216716?

jsgt

Member

Registered: 2009-03-16

Posts: 29

Re: System memory leak

That issue seems to be the opposite of what I'm experiencing--I get no OOM errors but do run out of memory.

~ free -h
              total        used        free      shared  buff/cache   available
Mem:           7.5G        5.3G        1.3G        216M        986M        1.8G
Swap:          2.0G        818M        1.2G

so the programs should be using about 5.3 GB - 986 MB = 4.3 GB, but

~ sudo ps_mem                                                                   
 Private  +   Shared  =  RAM used   Program                                     
                                                                                
...                                                                             
 58.8 MiB +  10.3 MiB =  69.2 MiB   Xorg                                        
 67.5 MiB +   7.9 MiB =  75.4 MiB   urxvt (26)                                  
631.9 MiB +  12.6 MiB = 644.5 MiB   firefox                                     
---------------------------------                                               
                          1.1 GiB                                               
=================================  

How do I find out what is allocating the memory?

shawntan

Member

Registered: 2010-04-13

Posts: 12

Re: System memory leak

jsgt, I'm having similar problems. Did you manage to find an explanation?

seth

Member

Registered: 2012-09-03

Posts: 51,072

Re: System memory leak

cat /proc/meminfo

free != available, the latter can be larger.

echo 3 | sudo tee /proc/sys/vm/drop_caches

This will drop all page, dentry & inode caches. ("1" drops pages and "2" drops dentry/inodes)

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

jsgt

Member

Registered: 2009-03-16

Posts: 29

Re: System memory leak

jsgt, I'm having similar problems. Did you manage to find an explanation?

I never found any explanation but I did start using linux-lts (currently 4.4.23) a few days ago and the problem is gone.

cat /proc/meminfo

free != available, the latter can be larger.

echo 3 | sudo tee /proc/sys/vm/drop_caches

This will drop all page, dentry & inode caches. ("1" drops pages and "2" drops dentry/inodes)

The field "available" in "free" was shrinking with system usage despite closing programs. From the man page,

       available
              Estimation  of how much memory is available for starting new applications, without swapping. Unlike the
              data provided by the cache or free fields, this field takes into account page cache and also  that  not
              all  reclaimable  memory  slabs will be reclaimed due to items being in use (MemAvailable in /proc/mem‐
              info, available on kernels 3.14, emulated on kernels 2.6.27+, otherwise the same as free)

i.e. it accounts for cache which proves it wasn't a cache issue.

I have been an Arch Linux user since around 2010 and this is the first time I've had to restart my computer on more than a weekly basis, and my usage patterns haven't changed. This, along with the fact that the problem disappeared when I downgraded the kernel, convinces me it's a kernel problem

seth

Member

Registered: 2012-09-03

Posts: 51,072

Re: System memory leak

Ok, sorry.
Could you spot anything suspicious (growing or unexpectedly not shrinking number) in /proc/meminfo then? Like an enourmous slab?

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

R00KIE

Forum Fellow

From: Between a computer and a chair

Registered: 2008-09-14

Posts: 4,734

Re: System memory leak

Shot in the dark: are you using VLC or anything that might use VLC as a backend?

---

R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

jsgt

Member

Registered: 2009-03-16

Posts: 29

Re: System memory leak

Thanks for the suggestions. The problem disappeared when I started using linux-lts but I accidentally booted into "linux" the other day and haven't had time to reboot so now I am experiencing (temporarily, until I reboot into linux-lts) the problem again.

~ cat /proc/meminfo
MemTotal:        7856580 kB
MemFree:         1180688 kB
MemAvailable:    2283972 kB
Buffers:           91096 kB
Cached:          1473584 kB
SwapCached:        77868 kB
Active:          2003036 kB
Inactive:        2085604 kB
Active(anon):    1612740 kB
Inactive(anon):  1194160 kB
Active(file):     390296 kB
Inactive(file):   891444 kB
Unevictable:           4 kB
Mlocked:               4 kB
SwapTotal:       2097148 kB
SwapFree:        1169308 kB
Dirty:                12 kB
Writeback:             0 kB
AnonPages:       2302620 kB
Mapped:           176204 kB
Shmem:            282940 kB
Slab:            2483012 kB
SReclaimable:      81780 kB
SUnreclaim:      2401232 kB
KernelStack:        5904 kB
PageTables:        29732 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     6025436 kB
Committed_AS:    5198544 kB

I am not using anything related to VLC.

seth

Member

Registered: 2012-09-03

Posts: 51,072

Re: System memory leak

2.5GB slab, most of it unclaimable.

The kernel or some module *is* pretty hungry.
(By comparism, slab is 100MB here and 90% are reclaimable)

You may check whether unloading/blacklisting modules (start with the exotic ones) free some memory there.

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc