You are not logged in.

Pages: 1

#1 2015-04-09 21:55:46

Omar007
Member
Registered: 2015-04-09
Posts: 368

NVidia SLI X.org conf

Hi all,

I'm currently trying to set up an xorg configuration file to enable Mosaic SLI on my system.
I had created the file 10-monitors.conf under /etc/X11/xorg.conf.d/ with the following content:

Section "Device"
        Identifier      "Card A"
        Driver          "nvidia"
        BusID           "PCI:1:00:0"
EndSection

Section "Device"
        Identifier      "Card B"
        Driver          "nvidia"
        BusID           "PCI:2:00:0"
EndSection

Section "Monitor"
        Identifier      "Left Monitor"
EndSection

Section "Monitor"
        Identifier      "Center Monitor"
        Option          "Primary" "true"
EndSection

Section "Monitor"
        Identifier      "Right Monitor"
EndSection

Section "Screen"
        Identifier      "Left Screen"
        Device          "Card B"
        Monitor         "Left Monitor"
        DefaultDepth    24
        Option          "SLI" "Mosaic"
        Option          "Stereo" "0"
        Option          "BaseMosaic" "True"
        Option          "MetaModes" "GPU-1.DFP-4.8: 1920x1200+0+0, GPU-0.DFP-4.8: 1920x1200+1920+0, GPU-1.DFP-6.9: 1920x1080+3840+0"
        Option          "nvidiaXineramaInfo" "FALSE"
        SubSection      "Display"
                Depth           24
        EndSubSection
EndSection

Section "Screen"
        Identifier      "Center Screen"
        Device          "Card A"
        Monitor         "Center Monitor"
        DefaultDepth    24
        Option          "SLI" "Mosaic"
        Option          "Stereo" "0"
        Option          "BaseMosaic" "True"
        Option          "MetaModes" "GPU-1.DFP-4.8: 1920x1200+0+0, GPU-0.DFP-4.8: 1920x1200+1920+0, GPU-1.DFP-6.9: 1920x1080+3840+0"
        Option          "nvidiaXineramaInfo" "FALSE"
        SubSection      "Display"
                Depth           24
        EndSubSection
EndSection

Section "Screen"
        Identifier      "Right Screen"
        Device          "Card B"
        Monitor         "Right Monitor"
        DefaultDepth    24
        Option          "SLI" "Mosaic"
        Option          "Stereo" "0"
        Option          "BaseMosaic" "True"
        Option          "MetaModes" "GPU-1.DFP-4.8: 1920x1200+0+0, GPU-0.DFP-4.8: 1920x1200+1920+0, GPU-1.DFP-6.9: 1920x1080+3840+0"
        Option          "nvidiaXineramaInfo" "FALSE"
        SubSection      "Display"
                Depth           24
        EndSubSection
EndSection

Section "ServerLayout"
        Identifier      "Default"
        Screen 0        "Left Screen" 0 0
        Screen 1        "Center Screen" RightOf "Left Screen"
        Screen 2        "Right Screen" RightOf "Center Screen"
        Option          "Xinerama" "0"
EndSection

However, as soon as I start GDM, the system freezes and only shows an underscore in the top left corner of my screen. I can't do anything locally but I can SSH into the system so it's not the whole system that freezes/crashes.

I also checked the wiki for setting up SLI Mosaic, and I found a config file that was similar in its core structure to what I had set up here. Because it differed a bit I tried to clone that exact file as well though, but it did not work either.

Some things I was able to find out while browsing the logs:
Linux always sets the GPU on PCI 2 as GPU-0/primary GPU as soon as there is a monitor connected to it. (as such, I tried flipping the numbers under MetaModes but that did not help)

Starting GDM with the above xorg configuration causes the following to appear in my logs:

apr 09 23:10:56 Omar-PC kernel: dmar: DRHD: handling fault status reg 2
apr 09 23:10:56 Omar-PC kernel: dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fffcf000
                                DMAR:[fault reason 06] PTE Read access is not set
apr 09 23:10:56 Omar-PC kernel: dmar: DRHD: handling fault status reg 102
apr 09 23:10:56 Omar-PC kernel: dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fffcf000
                                DMAR:[fault reason 06] PTE Read access is not set
apr 09 23:10:56 Omar-PC kernel: NVRM: GPU at PCI:0000:01:00: GPU-42b36c49-d1d1-aae8-f809-5b9a590e97c6
apr 09 23:10:56 Omar-PC kernel: NVRM: Xid (PCI:0000:01:00): 32, Channel ID 00000000 intr 00008000
apr 09 23:10:56 Omar-PC kernel: NVRM: Xid (PCI:0000:01:00): 32, Channel ID 00000000 intr 00008000

....

apr 09 23:10:56 Omar-PC kernel: dmar: DRHD: handling fault status reg 202
apr 09 23:10:56 Omar-PC kernel: dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr fffcd000
                                DMAR:[fault reason 06] PTE Read access is not set
apr 09 23:10:56 Omar-PC kernel: NVRM: Xid (PCI:0000:01:00): 56, CMDre 00000000 00000000 00000000 00000001 00000001
apr 09 23:11:08 Omar-PC kernel: NVRM: GPU at PCI:0000:02:00: GPU-fea6a08c-4a8b-8f9c-eb06-2d3b8a0cf594
apr 09 23:11:08 Omar-PC kernel: NVRM: Xid (PCI:0000:02:00): 8, Channel 00000000
apr 09 23:11:08 Omar-PC kernel: NVRM: os_schedule: Attempted to yield the CPU while in atomic or interrupt context
apr 09 23:11:08 Omar-PC kernel: sched: RT throttling activated

And later when I had attempted a restart of GDM:

apr 09 23:41:50 Omar-PC kernel: dmar: DRHD: handling fault status reg 102
apr 09 23:41:50 Omar-PC kernel: dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr ff
                                DMAR:[fault reason 06] PTE Read access is not set
apr 09 23:41:50 Omar-PC kernel: dmar: DRHD: handling fault status reg 202
apr 09 23:41:50 Omar-PC kernel: dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr ff
                                DMAR:[fault reason 06] PTE Read access is not set
apr 09 23:41:50 Omar-PC kernel: NVRM: Xid (PCI:0000:01:00): 32, Channel ID 00000000 intr 000
apr 09 23:41:50 Omar-PC kernel: NVRM: Xid (PCI:0000:01:00): 32, Channel ID 00000000 intr 000

....

apr 09 23:40:00 Omar-PC kernel: NVRM: Xid (PCI:0000:02:00): 8, Channel 00000000
apr 09 23:40:00 Omar-PC kernel: NVRM: os_schedule: Attempted to yield the CPU while in atomi

If I do not use any config files, GDM works fine but obviously in that case SLI will not be running.

Does anyone happen to know how to correctly configure the system for SLI??

Offline

#2 2015-04-10 17:03:19

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: NVidia SLI X.org conf

Ok I did some more digging around and I'm getting the impression that SLI Mosaic just doesn't work on GeForce cards atm.
Is this true?

Last edited by Omar007 (2015-04-10 17:03:30)

Offline

#3 2015-05-03 14:07:45

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: NVidia SLI X.org conf

Ok so it turns out my config file actually DOES work!

The problem turns out to be related to IOMMU. As soon as I put in any type of Xorg.conf or xorg.conf.d/*.conf file and I have intel_iommu=on supplied as kernel parameter, Xorg will not boot at all and always throws the above DMAR [DMA Read] error for the GPUs.
It seems it does not matter what I put in the config files; as soon as one is present, xorg just no longer boots.

Only when I remove the intel_iommu=on kernel parameter, it allows me to use xorg config files. If I do not remove it, I can not use xorg config files.

Is there a workaround that allows me to use both xorg config files as well as IOMMU??


My current config file:

# GPU @ PCI:1:00:0 = 42b36c49-d1d1-aae8-f809-5b9a590e97c6
# GPU @ PCI:2:00:0 = fea6a08c-4a8b-8f9c-eb06-2d3b8a0cf594

Section "Device"
	Identifier	"Card0"
	Driver		"nvidia"
	VendorName	"NVIDIA Corporation"
	BoardName	"GeForce GTX 970"
	BusID		"PCI:1:00:0"
EndSection

Section "Device"
	Identifier	"Card1"
	Driver		"nvidia"
	VendorName	"NVIDIA Corporation"
	BoardName	"GeForce GTX 970"
	BusID		"PCI:2:00:0"
EndSection

Section "Monitor"
	Identifier	"LeftMonitor"
	VendorName	"Unknown"
	ModelName	"DELL U2415"
	Option		"DPMS"
EndSection

Section "Monitor"
	Identifier	"CenterMonitor"
	VendorName	"Unknown"
	ModelName	"DELL U2415"
	Option		"DPMS"
	Option		"Primary" "true"
EndSection

Section "Monitor"
	Identifier	"RightMonitor"
	VendorName	"Unknown"
	ModelName	"DELL U2415"
	Option		"DPMS"
EndSection

Section "Screen"
	Identifier	"LeftScreen"
	Device		"Card1"
	Monitor		"LeftMonitor"
	DefaultDepth	24
	Option		"SLI" "Mosaic"
	Option		"Stereo" "0"
	Option		"BaseMosaic" "on"
	Option		"MetaModes" "GPU-fea6a08c-4a8b-8f9c-eb06-2d3b8a0cf594.DP-2.8: 1920x1200+0+0, GPU-42b36c49-d1d1-aae8-f809-5b9a590e97c6.DP-2.8: 1920x1200+1920+0, GPU-42b36c49-d1d1-aae8-f809-5b9a590e97c6.DP-4.8: 1920x1080+3840+0"
	Option		"nvidiaXineramaInfo" "FALSE"
	SubSection	"Display"
		Depth		24
	EndSubSection
EndSection

Section "Screen"
	Identifier	"CenterScreen"
	Device		"Card0"
	Monitor		"CenterMonitor"
	DefaultDepth	24
	Option		"SLI" "Mosaic"
	Option		"Stereo" "0"
	Option		"BaseMosaic" "on"
	Option		"MetaModes" "GPU-fea6a08c-4a8b-8f9c-eb06-2d3b8a0cf594.DP-2.8: 1920x1200+0+0, GPU-42b36c49-d1d1-aae8-f809-5b9a590e97c6.DP-2.8: 1920x1200+1920+0, GPU-42b36c49-d1d1-aae8-f809-5b9a590e97c6.DP-4.8: 1920x1080+3840+0"
	Option		"nvidiaXineramaInfo" "FALSE"
	SubSection	"Display"
		Depth		24
	EndSubSection
EndSection

Section "Screen"
	Identifier	"RightScreen"
	Device		"Card0"
	Monitor		"RightMonitor"
	DefaultDepth	24
	Option		"SLI" "Mosaic"
	Option		"Stereo" "0"
	Option		"BaseMosaic" "on"
	Option		"MetaModes" "GPU-fea6a08c-4a8b-8f9c-eb06-2d3b8a0cf594.DP-2.8: 1920x1200+0+0, GPU-42b36c49-d1d1-aae8-f809-5b9a590e97c6.DP-2.8: 1920x1200+1920+0, GPU-42b36c49-d1d1-aae8-f809-5b9a590e97c6.DP-4.8: 1920x1080+3840+0"
	Option		"nvidiaXineramaInfo" "FALSE"
	SubSection	"Display"
		Depth		24
	EndSubSection
EndSection

Section "ServerLayout"
	Identifier	"Default"
	Screen 0	"LeftScreen" 0 0
	Screen 1	"CenterScreen" RightOf "LeftScreen"
	Screen 2	"RightScreen" RightOf "CenterScreen"
	Option		"Xinerama" "0"
EndSection

If you see room for improvements to this config file, I'll gladly hear them wink

Last edited by Omar007 (2015-05-03 14:21:38)

Offline

#4 2015-05-03 15:45:13

Buddlespit
Member
From: Chesapeake, Va.
Registered: 2014-02-07
Posts: 501

Re: NVidia SLI X.org conf

Have you tried any of the other iommu options?

Hw-Probe

Offline

#5 2015-05-03 16:54:55

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: NVidia SLI X.org conf

Other IOMMU options?

To be able to use IOMMU on an Intel CPU, you need intel_iommu=on. I never heard of any other option to enable IOMMU on an Intel CPU.

Last edited by Omar007 (2015-05-03 16:55:51)

Offline

#6 2015-05-03 17:17:36

Buddlespit
Member
From: Chesapeake, Va.
Registered: 2014-02-07
Posts: 501

Re: NVidia SLI X.org conf

Try replacing "intel_iommu=on" with "iommu=soft". Please note the lack of 'intel_'. From what I can Google, this appears to be a valid choice.

Also note: I have yet to get SLI and multiple monitors to work. I've only heard of one guy that says he can do it (on Ubuntu) but his method doesn't work for me. Nvidia has SLI on multiple monitors disabled. I can do SLI OR multiple monitors, but not both in linux. The current (acceptable) method is to connect each monitor to a different graphics card.

Good luck!!

Hw-Probe

Offline

#7 2015-05-03 19:55:52

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: NVidia SLI X.org conf

Afaik that would prevent the usage of hardware IOMMU. I guess I could be wrong though. I'll google around for a bit smile

EDIT: Turns out I was almost right. I prevents hardware IOMMU as "iommu=soft" makes it use a software based IOMMU system. Not exactly an ideal solution I'd say. I'll give it a try later on anyway though to see if it would work at all.
But I'd probably be better of continuously reconnecting wires when I switch OS'es than using xorg configs to handle monitor setup and losing hardware IOMMU. sad

PS. Would you say that error is an Xorg problem or an IOMMU problem? Afaik this is probably caused by a bug in either one or the other right?

Last edited by Omar007 (2015-05-03 20:03:01)

Offline

Pages: 1

Board footer

Powered by FluxBB