GNOME + Wayland + Nvidia: Desktop corruption, application crashes

twistedpair · 2024-05-04 23:06:56

Hi! I'm running GNOME on Wayland. After using the machine for a while, launching more than a few apps, opening more than a few windows or doing anything GPU intensive gets the system into a state where I start to experience crashes and/or graphical corruption.

I've experienced:

Invisible windows (e.g., GNOME Files shows as open in the Dash, yet the "window" that's open is invisible)
Resizing windows past a certain size causes them to flicker or the app crashes (eg, maximise fails)
GNOME Videos complains that it can't Initialize OpenGL.

I'm thinking this is a VRAM related as the problems start to appear when nvidia-smi shows the VRAM usage approaching 100% used.
That also correlates to issues with lots of windows or larger windows (bigger surfaces == more memory usage).
Closing apps that are listed by nvidia-smi as using lots of VRAM sometimes alleviates the issues temporarily. Often a restart is required to fully fix everything.

My machine has 32GB of RAM and a Quadro P620 which has 2GB of VRAM.
Running kernel 6.8.8 and nvidia 550.78-1, and everything else is up-to-date (ran pacman -Syu before posting this).

For example, opening Blender and one of their example scenes in full textured mode has Blender use upwards of 1.6GB of VRAM. Add gnome-shell and firefox and it maxes out the 2GB on my card.
At this point trying to open new apps/windows or resize existing ones causes lots of breakage.

Kernel journal occasionally shows things like:

[drm:nv_drm_gem_alloc_nvkms_memory_ioctl [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00002b00] Failed to allocate NVKMS memory for GEM object

One thing worth noting that I thought of while posting this: I don't have swap enabled (machine runs root-on-zfs and never bothered to work out how to use a zvol as swap). Is it just that simple? My main memory usage is no where near close to full when these issues happen. I assumed that VRAM would be "swapped" to main memory when under pressure. Nothing I can find on Google seems to imply that swap is necessary for low VRAM situations.

V1del · 2024-05-04 23:23:38

Almost certain this doesn't work. A cursory google suggests something like this might have been added something to that effect on Windows drivers but likely not available on linux and I'm also not aware of any dedicated graphics card having a general mechanism for this.

This is one of those cases were opting for some remotely more up to date HW might be a good investment -- of your time and sanity.

twistedpair · 2024-05-05 09:05:19

V1del wrote:

Almost certain this doesn't work. A cursory google suggests something like this might have been added something to that effect on Windows drivers but likely not available on linux and I'm also not aware of any dedicated graphics card having a general mechanism for this.
This is one of those cases were opting for some remotely more up to date HW might be a good investment -- of your time and sanity.

That can't be right. There are plenty of still usable 2nd-hand machines or even brand-new machines that have low-end GPUs or iGPUs with 2-4GB of VRAM...

Nvidia launched the GeForce GT 1010 in 2021 with 2GB of VRAM.
AsRock currently sells an Intel ARC GPU with 4GB of VRAM.
A local retailer sells a $300 laptop with and Intel N4500 CPU with iGPU with 4GB of system RAM shared with the iGPU.
HP still sells on their website a HP Pavilion Laptop 15, with a NVIDIA GeForce MX550 with 2GB of dedicated VRAM -- compatible with Windows 11

And while 4GB of VRAM is double what I have, that would, by extrapolation, mean I could only open twice the number of windows before the exact same problem occurred. Which in the case when Blender is open would be say... all of 6 windows!
Surely crashing and corruption is not the expected behavior?

The Quadro P620 I have was released in 2018 -- 6 years ago is getting old, but I've had this machine for nearly 2 years now and have put up with this issue since it was built. So it was only 4 years old when I bought it!

I found a thread over on the freedesktop mailing list: https://lists.freedesktop.org/archives/ … 07332.html
It's for the AMD gpu driver, but seems to be more general and even mentions the Intel driver. It's about buffer eviction to system RAM, and that it currently isn't the best implementation and causes performance issues in low VRAM situations. But that seems to imply that, at least for AMD GPUS, in low VRAM situations things get "swapped" to system RAM rather than apps just crashing.

Or is this basically a symptom of the close-source Nvidia driver being shit?

Last edited by twistedpair (2024-05-05 09:23:38)

V1del · 2024-05-05 11:52:53

Nice to see that this "might work" on amdgpu (seeing as this is literally like a week ago that this patchset was proposed and it not being in the kernel yet, chances are this isn't going to work as smoothly there either), but yes I'm saying unlikely to implemented on nvidia.

You generally can't compare iGPUs to dGPUs... iGPUs are inherently designed to do that and have much tighter integration with the system memory by physical positioning.

You can still "use this" even with that low VRAM but then you need to temper your expectations and which software and how many windows you are going to run at the same time. Blender is a heavy tool and the current versions have already by their OpenGL baseline evicted quite a few older GPUs just by what the version baseline is. And you're unlikely to find people that do actual work in blender's current version with that little VRAM. Also Wayland by definition is going to be more taxing on your GPU/VRAM usage, have you tested a xorg session? Have you tested nouveau? Some of these mechanisms might not be really trivially usable by nvidia if it would mandate calling GPL code in their non-GPL driver, so it might also be a legal limitation rather than a technical one. You also in general can't think of swapping to side memory as a magic bullet, assets that are actively needed need to be swapped in to VRAM anyway (whether that swap is system memory is... irrelevant) so if you're running something that outright exceeds your 2GB you are going to crash anyway -- even on amdgpu.

2GB and a 620 level GPU 6 years ago is absolutely entry level intended for office stuff. Or maybe a HTPC with a single application open.

Last edited by V1del (2024-05-05 12:01:23)

seth · 2024-05-05 12:47:34

blender running out of VRAM is a frequent issue.
GL drivers can allocate and move stuff into RAM but if the data is *required* in VRAM for the GPU to operate on it you *need* to have that amount of VRAM available.

2GB VRAM is the bare minimum for blender, otherwise it won't even start.
If you want to do bigger stuff you need more VRAM - realistically you want 8GB+ to be actually any productive (besides maybe entry-level hello-world-i-learn-blender stuff) and for bigger projects 24+GB VRAM are a good idea.

twistedpair · 2024-05-06 04:50:52

I should not have mentioned Blender. I don't really use Blender.

I used Blender as an easy way to gobble up lots of VRAM to easily reproduce the various failure cases that I have been encountering.
Blender is an example of something that easily causes the whole desktop environment to become unstable -- application crashes, invisible and flickering windows, etc.

2GB and a 620 level GPU 6 years ago is absolutely entry level intended for office stuff.

Day-to-day I use this machine for web browsing, email and programming -- Firefox, Thunderbird, a terminal window or two, and VS Code.

Surely I don't need a 4080ti to do that??

Maybe its just GNOME? Apparently setting a desktop wallpaper can use upwards of 200MB of VRAM usage! https://gitlab.gnome.org/GNOME/mutter/-/issues/1271

twistedpair · 2024-05-06 04:59:00

I also found this old-ish thread on the Nvidia developers forum: https://forums.developer.nvidia.com/t/s … ux/41466/2

A reply from a Nvidia employee mentions that the driver (at least used to) handle VRAM pressure by "swapping" to system RAM:

Once the VRAM limit is reached, the driver behavior will be a mix of evicting video memory not currently in use and spilling to system memory.

seth · 2024-05-06 07:19:50

Maybe its just GNOME?

Are you positive that the VRAM usage is actually the (solitary) cause of your symptoms?
Do you get similar problems w/ an equally obese DE like plasma?

Do you run gnome on wayland or X11? Does it matter?
Do you enable https://wiki.archlinux.org/title/NVIDIA … de_setting - use the "nvidia_drm.modeset=1" kernel parameter (modprobe.conf won't do!)?
Do you get the same issues w/ https://aur.archlinux.org/packages/nvidia-535xx-dkms ?(cause there're certainly known w/ the 550xx versions)

If so, what processes actually are the little VRAM hogs?
(nvidia-smi should report that for you?)

Also

My main memory usage is no where near close to full when these issues happen.

You might want to quantify that - "oh, I still have totally 64MB RAM free when this happens" is maybe not the safe condition you might think it is

Arch Linux

#1 2024-05-04 23:06:56

GNOME + Wayland + Nvidia: Desktop corruption, application crashes

#2 2024-05-04 23:23:38

Re: GNOME + Wayland + Nvidia: Desktop corruption, application crashes

#3 2024-05-05 09:05:19

Re: GNOME + Wayland + Nvidia: Desktop corruption, application crashes

#4 2024-05-05 11:52:53

Re: GNOME + Wayland + Nvidia: Desktop corruption, application crashes

#5 2024-05-05 12:47:34

Re: GNOME + Wayland + Nvidia: Desktop corruption, application crashes

#6 2024-05-06 04:50:52

Re: GNOME + Wayland + Nvidia: Desktop corruption, application crashes

#7 2024-05-06 04:59:00

Re: GNOME + Wayland + Nvidia: Desktop corruption, application crashes

#8 2024-05-06 07:19:50

Re: GNOME + Wayland + Nvidia: Desktop corruption, application crashes

Board footer