Arch Linux

CrazyPython

Member

Registered: 2022-01-18

Posts: 3

NVIDIA GPU has LnkSta that suggests 8 PCIe Gen 1.0 lanes are used

I have an RTX 3090 plugged into 16x PCIe 5.0 slot. nvtop says I'm using 8x PCIe Gen 1.0 lanes.

I run lspci:

➜  ~ sudo lspci -vv | grep -A30 3090                
lspci: Unable to load libkmod resources: error -2
01:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3090] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: eVga.com. Corp. Device 3982
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0
        Interrupt: pin A routed to IRQ 200
        Region 0: Memory at 84000000 (32-bit, non-prefetchable) [size=16M]
        Region 1: Memory at 6000000000 (64-bit, prefetchable) [size=256M]
        Region 3: Memory at 6010000000 (64-bit, prefetchable) [size=32M]
        Region 5: I/O ports at 6000 [size=128]
        Expansion ROM at 85000000 [virtual] [disabled] [size=512K]
        Capabilities: [60] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee00c58  Data: 0000
        Capabilities: [78] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
                DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                        ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s (downgraded), Width x8 (downgraded)
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR+
                         10BitTagComp+ 10BitTagReq+ OBFF Via message, ExtFmt- EETLPPrefix-

My LnkCap (link capacity):

                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <16us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+

My LnkSta (???):

                LnkSta: Speed 2.5GT/s (downgraded), Width x8 (downgraded)
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

I read that LnkSta means what's actually being used. This would imply I'm running on degraded bandwidth and fewer lanes used.

My GUI uses integrated graphics. (I use my GPU for machine learning.) I have my DisplayPort cable plugged into my main motherboard for that reason.

How can I make my 3090 use 16 PCIe 4 lanes?

CrazyPython

Member

Registered: 2022-01-18

Posts: 3

Re: NVIDIA GPU has LnkSta that suggests 8 PCIe Gen 1.0 lanes are used

I loaded a machine learning model (GPT-J) and the connection upgraded to PCIe 4x8. It still isn't using all 16 lanes.

➜  ~ sudo lspci -vv | grep -A30 3090 | grep Speed
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
                LnkSta: Speed 16GT/s (ok), Width x8 (downgraded)

PCIe Gen 4x8