Arch Linux

LandonDeam

Member

Registered: 2022-12-17

Posts: 30

[SOLVED] Very high memory usage, but seemingly nothing using it

I've been experiencing this problem for a while now, but I am still unable to locate the cause. On boot, I only use a few gigabytes worth of memory, and I would expect to be using around 20-25GB at most at idle with how I have everything set up. My issue is that I idle at 35+ GB of used RAM, but that I cannot find what is actually using it all, let alone why so much is being used. The following is the output I get from free -m and cat /proc/meminfo :

               total        used        free      shared  buff/cache   available
Mem:           48097       37091        7866         408        4122       11005
Swap:           8191        3794        4397

MemTotal:       49251520 kB
MemFree:         7972808 kB
MemAvailable:   11186448 kB
Buffers:          356364 kB
Cached:          3694720 kB
SwapCached:        14460 kB
Active:          5469548 kB
Inactive:        2102800 kB
Active(anon):    2879612 kB
Inactive(anon):  1110456 kB
Active(file):    2589936 kB
Inactive(file):   992344 kB
Unevictable:        2296 kB
Mlocked:            2296 kB
SwapTotal:       8388604 kB
SwapFree:        4502636 kB
Zswap:           1147396 kB
Zswapped:        3405876 kB
Dirty:                16 kB
Writeback:             0 kB
AnonPages:       3452556 kB
Mapped:          1671220 kB
Shmem:            468804 kB
KReclaimable:     220288 kB
Slab:             958688 kB
SReclaimable:     220288 kB
SUnreclaim:       738400 kB
KernelStack:       34704 kB
PageTables:        83416 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    33014364 kB
Committed_AS:   21295048 kB
VmallocTotal:   34359738367 kB
VmallocUsed:     1415644 kB
VmallocChunk:          0 kB
Percpu:            33664 kB
HardwareCorrupted:     0 kB
AnonHugePages:    720896 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:    118784 kB
FilePmdMapped:    104448 kB
CmaTotal:              0 kB
CmaFree:               0 kB
Unaccepted:            0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:     7092308 kB
DirectMap2M:    43165696 kB
DirectMap1G:           0 kB

I've tried clearing page caches and such, but that typically only clears a few GB's out at most. I'm not entirely sure due to a lack of complete understanding, but I also don't think the numbers for the memory really add up here. It would seem that more memory is being use than is accounted for in these numbers.

Last edited by LandonDeam (2024-11-04 15:35:31)

seth

Member

Registered: 2012-09-03

Posts: 59,012

Re: [SOLVED] Very high memory usage, but seemingly nothing using it

AMD GPU?
https://bbs.archlinux.org/viewtopic.php?id=285160

GART/GTT is very generous to itself, you can try to re-claim RAM by allocating it.
https://unix.stackexchange.com/question … ree-memory explains how to do that using only "head" or "tail" - you want to claim more than "available", but less than more than total-(active + some padding for slab and stuff)

If that doesn't work

systemctl isolate rescue.target # this will kill your graphical session!
# echo 3 > /proc/sys/vm/drop_caches

https://unix.stackexchange.com/question … lear-cache
Then check the memory

Also try to disable "zswap.enabled=0", https://wiki.archlinux.org/title/Kernel_parameters

---
Edit, postponed - test and report on the above first: https://bbs.archlinux.org/viewtopic.php … 7#p2110357

Last edited by seth (2024-01-06 09:14:12)

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

LandonDeam

Member

Registered: 2022-12-17

Posts: 30

Re: [SOLVED] Very high memory usage, but seemingly nothing using it

AMD GPU?
https://bbs.archlinux.org/viewtopic.php?id=285160

Yeah, looking at it, I don't think this is very normal

Jan 03 21:16:31 desktop kernel: [drm] amdgpu: 24048M of GTT memory ready.

Edit 1:

I haven't messed with the kernel parameters yet, because I think something was messed up with AMDGPU/Radeon the last time I built their kernel modules, but I am able to reclaim the memory using head/tail method (at least 20GB so far)

Working on messing with GTT and THP now, will report back with results.

Edit 2:

Okay, GTT and THP don't seem to be the issue at hand, still getting 35GB+ usage without any particular process using that amount of RAM. I am using a RAID array through ZFS, and the RAM usage seemed to spike after reading/writing large amounts of data, so my thinking is that it might be ZFS trying to cache files en masse.

Last edited by LandonDeam (2024-01-08 03:12:51)

TheSgtBilko

Member

Registered: 2013-08-13

Posts: 88

Re: [SOLVED] Very high memory usage, but seemingly nothing using it

ZFS and ram usage: https://wiki.archlinux.org/title/ZFS#ZF … o_much_RAM

LandonDeam

Member

Registered: 2022-12-17

Posts: 30

Re: [SOLVED] Very high memory usage, but seemingly nothing using it

Marking as solved for now, because I can reclaim the memory, and the most likely reason has been found (and in my case, it's not breaking much of anything).

LandonDeam

Member

Registered: 2022-12-17

Posts: 30

Re: [SOLVED] Very high memory usage, but seemingly nothing using it

Seems to be an issue with GTT/Gart, or one of the last couple of kernel releases fixed it. I'm using

 amdgpu.gttsize=4096 amdgpu.gartsize=512 

as my kernel params, and it seems to be working fine.

My guess is gart is at fault, because I seem to have much less than 4GB of cached memory from Vulkan shader compiling, which has been really driving most of the memory leak here.

Last edited by LandonDeam (2024-11-04 15:39:32)