Arch Linux

Askannz

Member

Registered: 2016-10-22

Posts: 36

Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

Hi,

There was an interesting discussion on Reddit the other day. One user reported very poor CPU performance on his system equipped with an Nvidia GPU and a Kaby Lake CPU. No-one could figure out what the problem was until another user suggested to load the Nvidia driver with the option NVreg_UsePageAttributeTable=1.

This miraculously fixed the performance issue (as proven by CPU benchmark numbers). According to this second user, the Nvidia drivers make the very conservative assumption that Page Attribute Tables (PAT) may not be available on the system, and thus "do not setup MTRR correctly" and "messes up CPU caching" (in his words). I had never heard about this issue before, and it's nowhere to be found on the Arch Wiki.

I do not have sufficient knowledge to back up his affirmations (although I tend to believe him), so I wanted to hear your opinion on this matter. Should we add a section to the NVIDIA/Troubleshooting page about this ? Something like "if you experience poor CPU performance with the Nvidia driver loaded, set this parameter" ?

afader

Member

Registered: 2013-09-12

Posts: 162

Re: Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

I added the following to my /etc/modprobe.d/nvidia.conf

options nvidia-drm modeset=1 
options nvidia NVreg_UsePageAttributeTable=1 NVreg_EnablePCIeGen3=1

Definitely seems to have helped.

I have added to https://wiki.archlinux.org/index.php/NV … parameters

Last edited by afader (2018-11-17 21:49:08)

Askannz

Member

Registered: 2016-10-22

Posts: 36

Re: Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

Well, holy shit, thanks to you I've just found out my GPU supports PCIe 3.0 and it's not enabled by default in the Nvidia driver.

Thanks for editing the Wiki. Maybe there should also be a warning message about not using the PCIe options on unsupported hardware ? The Gentoo wiki has that scary line : "Some have even reported damage to hardware enabling this when it is not properly supported."

EDIT : Actually, I've discussed with the person who initially reported the NVreg_UsePageAttributeTable fix, and he says it's possible the Gentoo Wiki is outdated and the nvidia driver uses PCIe 3.x now. Any way I could check that on my setup ? There is no "parameters" folder in /sys/module/nvidia/ and modinfo only reports the list of parameters, not their values.

Last edited by Askannz (2018-11-18 06:12:10)

Ropid

Member

Registered: 2015-03-09

Posts: 1,069

Re: Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

You can check the PCIe link speed in the nvidia-settings tool. It is on the same tab where it shows clock speeds of the different power states and where you can overclock. The link speed can change dynamically, you will need to put some stress on the card to see its max speed (try just moving a window around). For my GTX 680, it did report PCIe 3.0 values without having to use any special parameters. The speeds for 3.0 are "8 GT/s".

I currently don't have my nvidia card installed so I can't experiment myself. What I'd like to know is if this NVreg_UsePageAttributeTable changes that terrible Xorg process CPU usage when moving windows around. If there were certain programs running at the same time, for example watching a Twitch stream in Chromium, then the CPU usage of the Xorg process would hit 100% and moving windows would stutter terribly.

afader

Member

Registered: 2013-09-12

Posts: 162

Re: Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

My nvidia-settings lists on the GPU 0  Graphics Card Information page

Bus Type: PCI Express x16 Gen
Maximum PCIe Link Speed: 8.0 GT/s

cat /proc/driver/nvidia/params
Mobile: 4294967295
ResmanDebugLevel: 4294967295
RmLogonRC: 1
ModifyDeviceFiles: 1
DeviceFileUID: 0
DeviceFileGID: 0
DeviceFileMode: 438
UpdateMemoryTypes: 4294967295
InitializeSystemMemoryAllocations: 1
UsePageAttributeTable: 4294967295
EnableMSI: 1
MapRegistersEarly: 0
RegisterForACPIEvents: 1
CheckPCIConfigSpace: 1
EnablePCIeGen3: 0
MemoryPoolSize: 0
KMallocHeapMaxSize: 0
VMallocHeapMaxSize: 0
IgnoreMMIOCheck: 0
TCEBypassMode: 0
UseThreadedInterrupts: 1
EnableStreamMemOPs: 0
EnableBacklightHandler: 0
EnableUserNUMAManagement: 1
RegistryDwords: ""
RegistryDwordsPerDevice: ""
RmMsg: ""
AssignGpus: ""
GpuBlacklist: ""

I also have the following

cat /etc/profile.d/kwin.sh                                                   
export KWIN_TRIPLE_BUFFER=1
export __GL_YIELD='usleep'

Askannz

Member

Registered: 2016-10-22

Posts: 36

Re: Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

EnablePCIeGen3: 0

Same thing on my system. So PCIe 3 is not enabled by default. Interesting. I wonder if it can make any difference in performances (probably not).

Ropid

Member

Registered: 2015-03-09

Posts: 1,069

Re: Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

EnablePCIeGen3: 0

Same thing on my system. So PCIe 3 is not enabled by default. Interesting. I wonder if it can make any difference in performances (probably not).

Even though that parameter is not set to "1", it's still running at PCIe 3.0 speeds. You can see that in that "8 GT/s" link speed being mentioned in the nvidia-settings tool. For PCIe 2.0 it would be "5 GT/s".

afader

Member

Registered: 2013-09-12

Posts: 162

Re: Nvidia, poor CPU performance and NVreg_UsePageAttributeTable

I have found that this bit only flips to 1 if I put "options nvidia_415 NVreg_EnablePCIeGen3=1" in the modprobe conf (with the version number). Then I see "PCI Express x16 Gen3" in the nvidia-settings. You can also see the LnkSta in lspci -vv