You are not logged in.

#1 2016-10-10 02:49:13

robc
Member
Registered: 2016-10-06
Posts: 9

Can't use framebuffer for boot console with non UEFI gpu passthrough

I've been messing with primary GPU passthrough all week.  I can boot up, start x, use the GPU, turn off X, unbind all the drivers, bind them to vfio, startup windows 7 with seabios and use the GPU, shut it off, boot up windows 10 with ovmf, no problem.  Everything is working great.  The only problem is I had to use the kernel option 

video=efifb:off

I would rather not have to remote in to get things started, or have x startup by itself.

Is there any frame buffer that can be used?  I would like any console at all, no matter how ugly.

I have also tried vesafb and uvesafb.    uvesafb actually allows windows 7 to eventually show a mangled startup page and randomly reboots, obviously not working.

Edit:  To clarify windows 10 always works even with the framebuffer in place and OVMF.  Only seabios based passthrough requires the disabled framebuffer.

Edit2:  I'm new to this, looking around it sounds like it should be possible with vga only, but i've had zero luck trying all kernel combinations of:

nofb nomodeset modeset=0/1 vga=normal vga=[decimal and hex codes] nvidia-drm.modeset=0/1

I suspect what I want is not possible, mainly because I am doing true primary passthrough with windows 7 which isn't true UEFI or something.  I'm on AMD and it seems like most Intel builds come with integrated graphics so it is much easier to pass in your real GPU.  I get better performance on win7 somehow... and hate windows 10, so unless anyone actually has an answer that gives me a console on boot, I'm probably going to have it kick off my primary desktop VM with gpu passthrough on boot, and setup the session manager to also select for windows 7 or 10, which will auto drop out of x, setup the drivers, and start the selected vm with gpu again, catching it on exit and restarting my primary desktop vm with session manager again.  I guess I don't really mind having to launch a 'default' vm on boot to get a screen.

Last edited by robc (2016-10-10 05:58:18)

Offline

#2 2016-10-10 13:00:27

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,866

Re: Can't use framebuffer for boot console with non UEFI gpu passthrough

For clarity : do you have 2 videocards in your system or one ?

lspci -k output would be helpful .


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#3 2016-10-10 16:32:20

robc
Member
Registered: 2016-10-06
Posts: 9

Re: Can't use framebuffer for boot console with non UEFI gpu passthrough

One video card, AMD based, no IGP.  I'm pretty sure it isn't the driver.  With or without the framebuffer I can drop/resassign drivers and use them natively, GPU passthrough in linux, or GPU passthrough in windows 10.  Only windows 7 requires the framebuffer to be disabled or it won't boot in seabios mode, and a full uefi install of windows 7 hangs just the same.

I want to say when I first started this journey I found a page that listed 7+ threads and patches to look at, or said to use the latest Arch with OVMF.  I went the OVMF route, and was frankly surprised I was able to get Win7 working period, but disabling the frame buffer did the trick.  The 'kernel drivers in use' section is either vfio when bound, missing when unbound, and nvidia when loaded natively.  Again I can reconfigure this all on the spot and launch VM's just fine, I'm pretty sure it is framebuffer related and not drivers.

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP106 [GeForce GTX 1060] [10de:1c03] (rev a1)
	Kernel driver in use: vfio-pci
	Kernel modules: nvidia_drm, nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f1] (rev a1)
	Subsystem: Gigabyte Technology Co., Ltd Device [1458:3720]
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

Edit:  Basically the entire issue is, when the framebuffer is enabled I get a good looking console, but no seabios passthrough, only OVMF, and I'm getting the feeling I'm lucky win7 gpu passthrough works at all with 1 GPU in seabios.

Edit2:  When I do attempt to load windows 7 with a framebuffer in place, I still get the seabios startup screen, so everything is connected correctly.  I believe it is windows 7 that can't handle a UEFI handoff and hangs, which somehow gets forced when you load up a framebuffer.  My system is a full UEFI install with UEFI only on the motherboard.  If someone knows that it is possible, maybe I'll go try a non UEFI mobo setting/install and see what happens.

Last edited by robc (2016-10-10 17:10:53)

Offline

#4 2016-10-10 18:09:02

robc
Member
Registered: 2016-10-06
Posts: 9

Re: Can't use framebuffer for boot console with non UEFI gpu passthrough

I'm recompiling the kernel without efifb (can't be a module) and will see what happens, then I'll compile with no framebuffer support and try that.

Offline

#5 2016-10-10 21:09:10

robc
Member
Registered: 2016-10-06
Posts: 9

Re: Can't use framebuffer for boot console with non UEFI gpu passthrough

No luck, I also tried compiling in the old nvidiafb driver but didn't get anything.  I also tried linux-vfio and linux-vfio-lts.

I think this ends my quest for the answer unless someone comes up with a good lead.  I'm happy enough to continue getting the rest of my system setup.

Offline

#6 2016-10-10 21:34:28

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,866

Re: Can't use framebuffer for boot console with non UEFI gpu passthrough

Robc,frankly i'm impressed with the results you got with binding / unbinding drivers.

It doesn't feel long ago (15 months ?) that vfio.blogspot.com expected dynamically binding/unbinding video drivers would crash systems for years to come.


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#7 2016-10-10 22:26:08

robc
Member
Registered: 2016-10-06
Posts: 9

Re: Can't use framebuffer for boot console with non UEFI gpu passthrough

After everything I read in the forums it sounds like something got fixed in the kernel recently, I don't even have to use linux-vfio, but then again I'm not on Intel that might need a patch.  I'm on a Gigabyte 990FX UD3, single GTX1060, 8 core 3.5GHz, 32GB ram.

I use pci stubs in the beginning, after that nvidia can take them, or vfio can take them, nvidia loads automatically if there is a framebuffer.  Nvidia has to be unloaded one at a time, rmmod nvidia_drm nvidia_uvm nvidia_modeset then the actual nvidia driver.   I get crashes if i try to unbind and forget to unload the kernel modules first.

All the ovmf passthrough was automatic through virt-manager after i pulled the latest ovmf binaries.  Seabios passthrough requires a 'tainted' qemu config with the command line options to pass in a few extra parameters, I believe x-vga=on -nographic -secondary-display vga or something.

Oh yeah and I'm passing in the GPU rom file, hiding the kvm signature, and changing the hv_vendor signature.  I think that covers just about everything... I really think I got lucky with my timing with the latest kernel based on all the forum stuff.

Last edited by robc (2016-10-10 22:33:03)

Offline

#8 2016-10-10 22:40:02

robc
Member
Registered: 2016-10-06
Posts: 9

Re: Can't use framebuffer for boot console with non UEFI gpu passthrough

To clarify, on startup my driver is pci-stub, i can bind and unbind to vfio effortlessly.  To go to nvidia I unbind vfio and do modprobe nvidia.  After that I have to do the rmmod sequence, then the same command to unbind vfio works to unbind from nvidia... or maybe it is actually when the nvidia driver unloads it drops the device, but I do the unbind anyway, then bind again to vfio, or even do another 'modprobe nvidia' and I'm off and running again.  vgaarb kicks of automatically no matter what I"m doing, maybe that is newish.

Edit:  Interestingly, the driver reassignment was never my goal, I just did it to test, my final base system will be pure vfio and do pci passthrough to my VMs.

Last edited by robc (2016-10-10 23:38:49)

Offline

Board footer

Powered by FluxBB