RAM appears to fill up rapidly after GNOME 47

thingfish57 · 2024-09-19 18:41:30

Fresh install of Arch. Judging by gnome-system-monitor, system RAM is rapidly reaching maximum capacity (16 GB). Starts this immediately upon boot to GNOME desktop, you can watch the RAM usage tick up pretty fast. Top reports gnome-shell as the main memory and CPU user. I'm wondering if maybe gnome-system-monitor isn't the culprit: closing it seems to "reset" the RAM usage issue and when you reopen it, RAM usage is nothing out of the ordinary.

It does indeed seem to be that gnome-system-monitor is at fault. The system only seems to chew up RAM when it's open. Otherwise, RAM usage stays well within the expected range.

Last edited by thingfish57 (2024-09-19 19:53:47)

midian · 2024-09-20 14:18:57

Just checked and I can confirm the same thing happens on my system. Closing gnome-system-monitor just stops the ram usage increasing, but does not reset it to the original value.

Last edited by midian (2024-09-20 14:19:34)

gfxmojo · 2024-09-21 01:02:55

Found that switching to Wayland instead of Xorg is a possible workaround. This may not be applicable in all cases, depending of course on your graphics card.

seth · 2024-09-21 12:01:10

gnome-shell as the main memory and CPU user

switching to Wayland instead of Xorg is a possible workaround

Everybody please post your Xorg log, https://wiki.archlinux.org/title/Xorg#General - might be the GPU driver allocating VRAM.
Do you actually run OOM or does the memory ultimately get surrendered when required by other processes?

quetzyg · 2024-09-21 17:25:39

I can confirm that by not running GNOME System Monitor, the issue is gone.

Pook · 2024-09-21 20:36:25

My system runs OOM with gnome-system-monitor running under Xorg. It does not run OOM with Wayland.

Full Xorg log is here after running gnome-system-monitor for a few seconds.
https://nextcloud.yonderserver.com/inde … kfcmMdiAgL

Here's the tail of the log with timestamps.

Sep 21 15:33:09 [    23.460] (WW) EDID timing clock 595.50 exceeds claimed max 595MHz, fixing
Sep 21 15:33:09 [    23.873] (EE) AMDGPU(0): drmmode_do_crtc_dpms cannot get last vblank counter
Sep 21 15:33:09 [    23.875] (II) AMDGPU(0): Allocate new frame buffer 2560x1440
Sep 21 15:33:09 [    23.876] (II) AMDGPU(0):  => pitch 10240 bytes
Sep 21 15:33:09 [    24.177] (EE) event23 - BT5.1 Mouse: client bug: event processing lagging behind by 263ms, your system is too slow
Sep 21 15:33:09 [    24.232] (II) AMDGPU(0): EDID vendor "AUO", prod id 56204
Sep 21 15:33:09 [    24.232] (II) AMDGPU(0): Printing DDC gathered Modelines:
Sep 21 15:33:09 [    24.232] (II) AMDGPU(0): Modeline "1920x1080"x0.0  142.60  1920 1978 2020 2080  1080 1083 1088 1142 -hsync -vsync (68.6 kHz eP)
Sep 21 15:33:09 [    24.232] (II) AMDGPU(0): Modeline "1920x1080"x0.0   95.07  1920 1978 2020 2080  1080 1083 1088 1142 -hsync -vsync (45.7 kHz e)
Sep 21 15:33:09 [    24.239] (WW) EDID timing clock 595.50 exceeds claimed max 595MHz, fixing

I was unable to find anything conclusive in the logs.

seth · 2024-09-21 20:39:00

Try to remove xf86-video-amdgpu.

Pook · 2024-09-21 20:47:03

Removed xf86-video-amdgpu and rebooted. I have not noticed any change.
https://nextcloud.yonderserver.com/inde … kfcmMdiAgL
The file is noAmdgpuXorg.0.log
Omitted the tail because it's just edid stuff.

seth · 2024-09-21 20:51:56

[    21.227] (II) RADEON: Driver for ATI/AMD Radeon chipsets:

You can remove all xf86-video-* packages though I doubt that's the cause and this will leak either in mutter or mesa.
Can you reproduce this w/ only one output?

Pook · 2024-09-21 21:06:48

No change.
Uninstalled everything relevant to xf86-video*.
Rebooted with only the laptop's built in monitor.
https://nextcloud.yonderserver.com/inde … kfcmMdiAgL
New log is noXorgVidXorg.0.log.

eomanis · 2024-09-22 10:52:40

GNOME Shell 47 on Wayland here, an application was just OOM-killed right after I opened it.
19 of 32 GiB RAM in use after a system update and reboot, but I cannot figure out what the hell is using it, the visible processes do not add up to that by a long shot.

seth · 2024-09-22 13:31:51

Does anybody here NOT use an AMD GPU?
https://gitlab.gnome.org/GNOME/gnome-sh … ssues/5729

Does "LIBGL_ALWAYS_SOFTWARE=true" or using zink avoid it?
https://wiki.archlinux.org/title/OpenGL#Mesa
https://wiki.archlinux.org/title/OpenGL … kan_(Zink)

gfxmojo · 2024-09-22 14:57:05

AMD GPU is not an option for me due to the RTX capabilities required to perform my 3D work. So no dice.

I can still see RAM usage growing every few seconds with LIBGL_ALWAYS_SOFTWARE=true.
Zink, on the other hand, generates a libEGL warning: DRI3: Screen seems not DRI3 capable.

seth · 2024-09-22 15:04:02

AMD GPU is not an option for me due to the RTX capabilities required to perform my 3D work. So no dice.

More importantly that means you get the leak on nvidia? No hybrid graphics?
(I asked to check whether the leak might have been in the amdgpu specific dri implementation)

The entire mesa stuff there will not work w/ the binary nvidia driver.

https://gitlab.gnome.org/GNOME/gnome-sy … issues/321
But that's not even new?
https://bbs.archlinux.org/viewtopic.php … 0#p2163550

Edit:
https://bugs.launchpad.net/ubuntu/+sour … ug/2079898

Last edited by seth (2024-09-22 15:06:10)

gfxmojo · 2024-09-22 15:29:40

I use nvidia exclusively. Multiple monitors, no hybrid graphics. I also reverted to the Gnome 46 system monitor and the memory issue still occurs, albeit slower.

Pook · 2024-09-22 17:08:02

Here is a valgrind output of gnome-system-monitor.
The leak report can be downloaded here, the file is valgrind-gsm-leak.txt
https://nextcloud.yonderserver.com/inde … kfcmMdiAgL
It's a rather large text file, so be aware of that when you open it.
I am not able to pinpoint how this relates back to xorg, I am not familiar with that codebase or GTK.
Perhaps those with more knowledge can provide some insight.

seth · 2024-09-22 21:09:13

==3623== LEAK SUMMARY:
==3623==    definitely lost: 78,024 bytes in 251 blocks
==3623==    indirectly lost: 276,290 bytes in 1,913 blocks
==3623==      possibly lost: 15,718,299 bytes in 68,945 blocks
==3623==    still reachable: 34,902,315 bytes in 193,230 blocks
==3623==                       of which reachable via heuristic:
==3623==                         newarray           : 589,632 bytes in 8 blocks
==3623==                         multipleinheritance: 103,864 bytes in 405 blocks
==3623==         suppressed: 48 bytes in 1 blocks

That's roughly 15MB - is that even close to the number you'd expect from that run?
The losses seems to stem from https://gitlab.gnome.org/GNOME/gtk/-/tree/main/gsk/gpu so the strategy would be to downgrade gtk and see whether that fixes it - except:
https://bbs.archlinux.org/viewtopic.php … 2#p2196562

peshov · 2024-09-22 21:39:12

From what I understand, GTK has a new OpenGL driver. I have been experiencing memory leaks with it. What fixed it for me what changing $GSK_RENDERER to gl, which was the default OpenGL renderer from Gnome 46. The default value now is ngl. There is also a vulkan implementation that has the same issue.

I am using AMD Ryzen™ 7 8845HS w/ Radeon™ 780M Graphics.

The issue I think with system monitor is, that it somehow uses more vram and ram to render perhaps the graphs. This is speculation on my part, though.

The other issue changing GSK_RENDERER fixed was that when i log out and log back in, GDM/Shell freezes and I need to force reboot. I can't even ctrl+alt+f3 from there.

Update: I upgraded opencl-amd from the testing repos and it seems to somewhat remedy the problem. Changing the renderer for now is the definite fix for me.

Last edited by peshov (2024-09-22 21:45:56)

gfxmojo · 2024-09-23 01:20:22

I can confirm that GSK_RENDERER=gl works. System appears stable at 77MB RAM usage. YMMV.

Update:
After a couple of hours, memory usage has grown to ~100MB. So I guess the leak occurs here too, though not as fast.

Last edited by gfxmojo (2024-09-23 02:25:24)

Pook · 2024-09-23 02:36:07

Setting env GSK_RENDERER=gl has mitigated the issue on my system as well.

$ GSK_RENDERER=gl gnome-system-monitor

Thanks for taking the time to look through that valgrind output, Seth!
I would have expected a leak on the order of gigabytes.

seth · 2024-09-23 06:59:27

From what I understand, GTK has a new OpenGL driver. I have been experiencing memory leaks with it. What fixed it for me what changing $GSK_RENDERER to gl, which was the default OpenGL renderer from Gnome 46. The default value now is ngl. There is also a vulkan implementation that has the same issue.

Afaiu the default is vulkan, but ngl is also affected?
https://gitlab.gnome.org/GNOME/gnome-sy … issues/321 might be two issues, the steady leak and ("4GB/m") soemthing related to the GSK renderer which, if mesa and pure nvidia systems are affected and also vulkan and ngl is rather not a driver bug, but either a bug in the GSK implementations or a poor GSK design that allows clients (like the system monitor) to make assumptions about the implementation details which can be horribly wrong - what seems to ring a bell with some other GTK code…

Someone should probably record that upstream (and that someone is certainly not gonna be me)

Pook · 2024-09-23 16:14:52

I have mentioned the issue in what appears to be a relevant issue request on the gnome gitlab and linked my comment back to this thread.
Gnome Gitlab: Potential memory leak due to excessive mmap() calls

tuxayo · 2024-09-24 03:20:59

The leak can also happen on an Intel HD 4000 iGPU (on xorg)

leo z · 2024-09-24 03:57:27

I think this commit https://gitlab.gnome.org/GNOME/gtk/-/co … 8ec2ec980b fixed this issue. You can test it with gtk4-git.

peshov · 2024-09-24 09:10:01

@leo z, thank you for noting that! There is also the https://aur.chaotic.cx/ repo with this package prebuild as of less than an hour ago. A tag 4.16.2 was made which includes this fix.

I am testing this now. At first glance, on my machine, it doesn't seem to be much difference. With just a web browser, and slack, my machine hangs currently at 9.5/16 GB and ever so slowly rises.
I also tried downgrading to 4.14.5, but the windows have no background then.

I tried rebuilding gtk4-git from the AUR but ran out of memory very rapidly. I'm unsure why this is.

Edit:
The change log states that this is actually for ngl, whereas I was trying it with gl. https://gitlab.gnome.org/GNOME/gtk/-/co … 9677fb2ea1

I am trying this now with $GSK_RENDERER=ngl. It seems that 5.7/16 GB is better than before. Let's see how it fares when I have some meetings and do some coding

Kudos to the Gnome devs for the quick fix!! Hopefully it rolls out to package repos soon. It feels like this needs to be fixed in more places, though.

Last edited by peshov (2024-09-24 09:26:25)

Arch Linux

#1 2024-09-19 18:41:30

RAM appears to fill up rapidly after GNOME 47

#2 2024-09-20 14:18:57

Re: RAM appears to fill up rapidly after GNOME 47

#3 2024-09-21 01:02:55

Re: RAM appears to fill up rapidly after GNOME 47

#4 2024-09-21 12:01:10

Re: RAM appears to fill up rapidly after GNOME 47

#5 2024-09-21 17:25:39

Re: RAM appears to fill up rapidly after GNOME 47

#6 2024-09-21 20:36:25

Re: RAM appears to fill up rapidly after GNOME 47

#7 2024-09-21 20:39:00

Re: RAM appears to fill up rapidly after GNOME 47

#8 2024-09-21 20:47:03

Re: RAM appears to fill up rapidly after GNOME 47

#9 2024-09-21 20:51:56

Re: RAM appears to fill up rapidly after GNOME 47

#10 2024-09-21 21:06:48

Re: RAM appears to fill up rapidly after GNOME 47

#11 2024-09-22 10:52:40

Re: RAM appears to fill up rapidly after GNOME 47

#12 2024-09-22 13:31:51

Re: RAM appears to fill up rapidly after GNOME 47

#13 2024-09-22 14:57:05

Re: RAM appears to fill up rapidly after GNOME 47

#14 2024-09-22 15:04:02

Re: RAM appears to fill up rapidly after GNOME 47

#15 2024-09-22 15:29:40

Re: RAM appears to fill up rapidly after GNOME 47

#16 2024-09-22 17:08:02

Re: RAM appears to fill up rapidly after GNOME 47

#17 2024-09-22 21:09:13

Re: RAM appears to fill up rapidly after GNOME 47

#18 2024-09-22 21:39:12

Re: RAM appears to fill up rapidly after GNOME 47

#19 2024-09-23 01:20:22

Re: RAM appears to fill up rapidly after GNOME 47

#20 2024-09-23 02:36:07

Re: RAM appears to fill up rapidly after GNOME 47

#21 2024-09-23 06:59:27

Re: RAM appears to fill up rapidly after GNOME 47

#22 2024-09-23 16:14:52

Re: RAM appears to fill up rapidly after GNOME 47

#23 2024-09-24 03:20:59

Re: RAM appears to fill up rapidly after GNOME 47

#24 2024-09-24 03:57:27

Re: RAM appears to fill up rapidly after GNOME 47

#25 2024-09-24 09:10:01

Re: RAM appears to fill up rapidly after GNOME 47

Board footer