Arch Linux

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

As far as i understand it - the CSM actually provides VGA, aw told us a while ago about the fact that we explicitly do not want it.

yep, i already thought about it being vga fallback as well. it won't do if it really is...
still, i think that win 7 setup running is a good sign - it's just that some kind of trick is needed to make it going. the hard part is uncovering this trick (and for now, i don't even have a clue on where to look for to get more info that may prove useful - i hope installer have some kind of logging) )

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

As far as i understand it - the CSM actually provides VGA, aw told us a while ago about the fact that we explicitly do not want it.

yep, i already thought about it being vga fallback as well. it won't do if it really is...
still, i think that win 7 setup running is a good sign - it's just that some kind of trick is needed to make it going. the hard part is uncovering this trick (and for now, i don't even have a clue on where to look for to get more info that may prove useful - i hope installer have some kind of logging) )

I have install running right now.

qemu-system-x86_64 \
-boot menu=on \
-enable-kvm \
-monitor stdio \
-M q35 \
-m 1024 \
-cpu host \
-net none \
-rtc base=localtime \
-smp 1,sockets=1,cores=1,threads=1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
-drive if=pflash,format=raw,file=/mnt/hdd/qemu/OVMF_VARS.fd \
-drive file='/mnt/hdd/qemu/qemu-uefi-win7.img',id=disk,format=raw,aio=native,cache=none,if=none \
-drive file='/mnt/hdd/qemu/virtio.iso',id=cdrom,format=raw,readonly=on,if=none \
-drive file='/mnt/hdd/qemu/windows7.iso',id=cdrom2,format=raw,readonly=on,if=none \
-drive file='/dev/null',id=dummy \
-device virtio-scsi-pci \
-device virtio-blk-pci,drive=disk \
-device ide-cd,drive=cdrom \
-device scsi-cd,drive=cdrom2 \
-vga std

Well, i could've forced windowns instadamnation to use GPT, but... i just pressed "next" on the disk select menu, and it actually installs.

It MIGHT not detect the disks from the first time, saying something like "driver was installed, but no new devices appeared" - you just have to tell it "try again, silly".

Wonderful fact: OVMF doesn't recognize IDE-CD and shows virtio-blk-pci as EFI Misc. device(seems like we'll need to use SCSI anyway, to force OVMF to at least try booting from it), BUT windows installation system recognizes IDE-CD and loads drivers from it just fine.

I doubt the use of q35, maybe with -M pc it will work too.

Isn't windows 7 is using legacy MBR-based bootloader? We MAY have to force the use of GPT to make it choose the right, EFI-compatible bootloader.

EDIT: Whoa, hasty update! It finished copying installation files on the disk, and hung. But CTRL-C thrown at qemu, and starting it over again actually made it booting the installer. It means we're booted from virtio-blk-pci. But it hung again.

Booting from the device(virtio-blk-pci,that is), after "starting windows" animation, results in a black screen, some disk activity(on the host!) and zero cpu consumption. I've pushed the reset, windows said "ow, that did hurt. safe mode?" - booting in safe mode results in a message that "windows installation won't continue in safe-mode, restart the computer normally".
That means:
1.IT BOOTS.Kind of.
2.Maybe resetting everything wasn't a good idea.
3.Try again!

Oh, i get it. Guest reboots just don't work.

It actually uses GPT. So we don't have to force it. The install is possible, you just need to do it right. Surprisingly enough, even a zero-ed out OVMF-based VM fails to reset. Maybe that's OVMF's problem.

Last edited by Duelist (2014-12-11 11:29:55)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

As far as i understand it - the CSM actually provides VGA, aw told us a while ago about the fact that we explicitly do not want it.

yep, i already thought about it being vga fallback as well. it won't do if it really is...
still, i think that win 7 setup running is a good sign - it's just that some kind of trick is needed to make it going. the hard part is uncovering this trick (and for now, i don't even have a clue on where to look for to get more info that may prove useful - i hope installer have some kind of logging) )

I have install running right now.

qemu-system-x86_64 \
-boot menu=on \
-enable-kvm \
-monitor stdio \
-M q35 \
-m 1024 \
-cpu host \
-net none \
-rtc base=localtime \
-smp 1,sockets=1,cores=1,threads=1 \
-drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
-drive if=pflash,format=raw,file=/mnt/hdd/qemu/OVMF_VARS.fd \
-drive file='/mnt/hdd/qemu/qemu-uefi-win7.img',id=disk,format=raw,aio=native,cache=none,if=none \
-drive file='/mnt/hdd/qemu/virtio.iso',id=cdrom,format=raw,readonly=on,if=none \
-drive file='/mnt/hdd/qemu/windows7.iso',id=cdrom2,format=raw,readonly=on,if=none \
-drive file='/dev/null',id=dummy \
-device virtio-scsi-pci \
-device virtio-blk-pci,drive=disk \
-device ide-cd,drive=cdrom \
-device scsi-cd,drive=cdrom2 \
-vga std

Well, i could've forced windowns instadamnation to use GPT, but... i just pressed "next" on the disk select menu, and it actually installs.

It MIGHT not detect the disks from the first time, saying something like "driver was installed, but no new devices appeared" - you just have to tell it "try again, silly".

Wonderful fact: OVMF doesn't recognize IDE-CD and shows virtio-blk-pci as EFI Misc. device(seems like we'll need to use SCSI anyway, to force OVMF to at least try booting from it), BUT windows installation system recognizes IDE-CD and loads drivers from it just fine.

I doubt the use of q35, maybe with -M pc it will work too.

Isn't windows 7 is using legacy MBR-based bootloader? We MAY have to force the use of GPT to make it choose the right, EFI-compatible bootloader.

EDIT: Whoa, hasty update! It finished copying installation files on the disk, and hung. But CTRL-C thrown at qemu, and starting it over again actually made it booting the installer. It means we're booted from virtio-blk-pci. But it hung again.

Booting from the device(virtio-blk-pci,that is), after "starting windows" animation, results in a black screen, some disk activity(on the host!) and zero cpu consumption. I've pushed the reset, windows said "ow, that did hurt. safe mode?" - booting in safe mode results in a message that "windows installation won't continue in safe-mode, restart the computer normally".
That means:
1.IT BOOTS.Kind of.
2.Maybe resetting everything wasn't a good idea.
3.Try again!

Oh, i get it. Guest reboots just don't work.

https://i.imgur.com/a5MviRd.png
It actually uses GPT. So we don't have to force it. The install is possible, you just need to do it right. Surprisingly enough, even a zero-ed out OVMF-based VM fails to reset. Maybe that's OVMF's problem.

Have you tried integrating your amd/nvidia drivers on the install disk to get past the black screen?

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

OK i got windows 7 working using OVMF, so what i did was integrate redhat virtio drivers and qxl drivers into the windows installation disk using win-toolkit then i created a 4GB GPT fat32 formatted disk and copied the installation files there, then i launched QEMU with -vga qxl, a raw disk image, the windows installation image, also using -device virtio-scsi-pci ...

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

OK i got windows 7 working using OVMF, so what i did was integrate redhat virtio drivers and qxl drivers into the windows installation disk using win-toolkit then i created a 4GB GPT fat32 formatted disk and copied the installation files there, then i launched QEMU with -vga qxl, a raw disk image, the windows installation image, also using -device virtio-scsi-pci ...

So you're running with -vga qxl and it still needs drivers? Strange. I was running with -vga std, and it needed drivers too! That's weird. But it's nice to see that i was right, and windows7+pure EFI+GPT is really possible. Guess i'll have to include -vga std drivers myself. Actually, maybe someone who made my(YARR) iso threw these drivers out.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

OK i got windows 7 working using OVMF, so what i did was integrate redhat virtio drivers and qxl drivers into the windows installation disk using win-toolkit then i created a 4GB GPT fat32 formatted disk and copied the installation files there, then i launched QEMU with -vga qxl, a raw disk image, the windows installation image, also using -device virtio-scsi-pci ...

So you're running with -vga qxl and it still needs drivers? Strange. I was running with -vga std, and it needed drivers too! That's weird. But it's nice to see that i was right, and windows7+pure EFI+GPT is really possible. Guess i'll have to include -vga std drivers myself. Actually, maybe someone who made my(YARR) iso threw these drivers out.

There are no -vga std drivers AFAIK, windows will use its standar vga driver which i believe is causing the problems, plus the only way you can install windows using EFI is with a GPT partitioned disk

Last edited by nbhs (2014-12-11 16:49:51)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

OK i got windows 7 working using OVMF, so what i did was integrate redhat virtio drivers and qxl drivers into the windows installation disk using win-toolkit then i created a 4GB GPT fat32 formatted disk and copied the installation files there, then i launched QEMU with -vga qxl, a raw disk image, the windows installation image, also using -device virtio-scsi-pci ...

So you're running with -vga qxl and it still needs drivers? Strange. I was running with -vga std, and it needed drivers too! That's weird. But it's nice to see that i was right, and windows7+pure EFI+GPT is really possible. Guess i'll have to include -vga std drivers myself. Actually, maybe someone who made my(YARR) iso threw these drivers out.

There are no -vga std drivers AFAIK, windows will use its standar vga driver which i believe is causing the problems, plus the only way you can install windows using EFI is with a GPT partitioned disk

Isn't CSM helping when using MBR+UEFI?

Well, windows' default VGA driver works in safe mode and during installation, that's a little bit borked.
Seems like it actually loads something different and it's broken. If i'll trick it to use the fall-back-default one, which is rendering in safemode and during boot-up animation..

Last edited by Duelist (2014-12-11 16:54:58)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So you're running with -vga qxl and it still needs drivers? Strange. I was running with -vga std, and it needed drivers too! That's weird. But it's nice to see that i was right, and windows7+pure EFI+GPT is really possible. Guess i'll have to include -vga std drivers myself. Actually, maybe someone who made my(YARR) iso threw these drivers out.

There are no -vga std drivers AFAIK, windows will use its standar vga driver which i believe is causing the problems, plus the only way you can install windows using EFI is with a GPT partitioned disk

Isn't CSM helping when using MBR+UEFI?

Well, windows' default VGA driver works in safe mode and during installation, that's a little bit borked.
Seems like it actually loads something different and it's broken. If i'll trick it to use the fall-back-default one, which is rendering in safemode and during boot-up animation..

Actually maybe it doesnt load anything at all

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

There are no -vga std drivers AFAIK, windows will use its standar vga driver which i believe is causing the problems, plus the only way you can install windows using EFI is with a GPT partitioned disk

Isn't CSM helping when using MBR+UEFI?

Well, windows' default VGA driver works in safe mode and during installation, that's a little bit borked.
Seems like it actually loads something different and it's broken. If i'll trick it to use the fall-back-default one, which is rendering in safemode and during boot-up animation..

Actually maybe it doesnt load anything at all

That is very strange, because safe mode works. I'll try to grab a bootlog from a safe-mode boot to see which driver actually gets loaded in it.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Isn't CSM helping when using MBR+UEFI?

Well, windows' default VGA driver works in safe mode and during installation, that's a little bit borked.
Seems like it actually loads something different and it's broken. If i'll trick it to use the fall-back-default one, which is rendering in safemode and during boot-up animation..

Actually maybe it doesnt load anything at all

That is very strange, because safe mode works. I'll try to grab a bootlog from a safe-mode boot to see which driver actually gets loaded in it.

Yup like i said, its because it not loading the vga driver, go into safe mode and disable the the the display adapter

sinny

Member

Registered: 2014-05-24

Posts: 75

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

ok, nice to see people got it working - now i only need to somehow catch up.

for now i'm getting weird issues - i definitely can not uefi-boot from any of 3 isos i got (all 3 are not recognized as FS's, onnly as BLK devices)
and installing from self-assembled image failed requiring drivers. still need more testing using all the data you assembled

i also tried using virt-manager plenty of time, but it seemed less manageable (due to lack of fine-grained control)

anyways, thanks for all the work - it's really valuable, considering the mess win8 is

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Actually maybe it doesnt load anything at all

That is very strange, because safe mode works. I'll try to grab a bootlog from a safe-mode boot to see which driver actually gets loaded in it.

Yup like i said, its because it not loading the vga driver, go into safe mode and disable the the the display adapter

The problem is - i'll have to reinvent the bicycle: safe mode isn't functional. So it's either you're pushing all the needed drivers in the ISO at the very beginning, or you're booting a livecd system to fix the not-yet-installed system. Thankfully, that livecd may be MSDART, which is included in my installation ISO, so i can even fire up the explorer to have more-or-less GUI representation of what i'll need to do to revive the system. Or i could just shuffle through all available display devices and find one that works nice.

BTW, are you using -M pc or -M q35(like i am)?

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

That is very strange, because safe mode works. I'll try to grab a bootlog from a safe-mode boot to see which driver actually gets loaded in it.

Yup like i said, its because it not loading the vga driver, go into safe mode and disable the the the display adapter

The problem is - i'll have to reinvent the bicycle: safe mode isn't functional. So it's either you're pushing all the needed drivers in the ISO at the very beginning, or you're booting a livecd system to fix the not-yet-installed system. Thankfully, that livecd may be MSDART, which is included in my installation ISO, so i can even fire up the explorer to have more-or-less GUI representation of what i'll need to do to revive the system. Or i could just shuffle through all available display devices and find one that works nice.

BTW, are you using -M pc or -M q35(like i am)?

Both seem to work just fine, i suggest you try the method i mentioned before, it will save you a lot of time, perhaps its possible to load the qxl driver during the installation, idk and i havent tried, i also mentioned booting with -vga std ( this is of course post install ) and disabling it in safe mode also seems to allow windows to boot, as of trying another vga adapter, i dont think it will work since windows always try load its standard vga driver, another thing i tested is passing though my GPU ( R9 280x ) and it will work but the drivers have to be installed first ( booting with -vga qxl then installing the catalyst driver )

Last edited by nbhs (2014-12-11 18:19:54)

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Yup like i said, its because it not loading the vga driver, go into safe mode and disable the the the display adapter

The problem is - i'll have to reinvent the bicycle: safe mode isn't functional. So it's either you're pushing all the needed drivers in the ISO at the very beginning, or you're booting a livecd system to fix the not-yet-installed system. Thankfully, that livecd may be MSDART, which is included in my installation ISO, so i can even fire up the explorer to have more-or-less GUI representation of what i'll need to do to revive the system. Or i could just shuffle through all available display devices and find one that works nice.

BTW, are you using -M pc or -M q35(like i am)?

Both seem to work just fine, i suggest you try the method i mentioned before, it will save you a lot of time, perhaps its possible to load the qxl driver during the installation, idk and i havent tried, i also mentioned booting with -vga std ( this is of course post install ) and disabling it in safe mode also seems to allow windows to boot, as of trying another vga adapter, i dont think it will work since windows always loads its standard vga driver, another thing i tested is passing though my GPU ( R9 280x ) and it will work but the drivers have to be installed first ( booting with -vga qxl then installing the catalyst driver )

But i've installed the whole system using only -vga std and no other stuff! Tried changing it on that undead system to something else(-vga qxl for example) - and no, the install won't finish because there is no graphics.

BTW, great to know that even VFIO works with that tricky setup.

Also a thing i've remembered: do you have guest reboots working? Because my test-system hangs in the OVMF after a reset, showing an underscore somewhere in the middle and nothing else. Only ctrl-c to qemu saves the day.

Last edited by Duelist (2014-12-11 20:16:33)

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Also a think i've remembered: do you have guest reboots working? Because my test-system hangs in the OVMF after a reset, showing an underscore somewhere in the middle and nothing else. Only ctrl-c to qemu saves the day.

Looks like an issue with OVMF + q35

nbhs

Member

From: Montevideo, Uruguay

Registered: 2013-05-02

Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

But i've installed the whole system using only -vga std and no other stuff! Tried changing it on that undead system to something else(-vga qxl for example) - and no, the install won't finish because there is no graphics.

Then i believe the only solution is integrating either amd/nvidia drivers or the qxl drivers, or perhaps replacing windows 7 vga driver with windows 8's one (if its possible at all)

Last edited by nbhs (2014-12-11 18:27:59)

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, I got my new hardware - including a new 1 TB SSD from Samsung. So I have one SSD for archlinux (240 GB) and the new Samsung SSD for virtualized Windows 8.1.

NOW: How do I add the SSD as a direct block device as AW suggested? I have done it like this:

<devices>
    <emulator>/usr/sbin/qemu-system-x86_64</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/sdb'/>
      <target dev='vda' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </disk>

It did install Windows like that. But I am not sure it's correct. I really want as much performance as possible in a VM.

Thank you guys for your help!

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi, I got my new hardware - including a new 1 TB SSD from Samsung. So I have one SSD for archlinux (240 GB) and the new Samsung SSD for virtualized Windows 8.1.

Offtopic question:
1TB SSD?! Entirely for windows?! Why would you need linux in that case?
I'm fine with my 60+10(SSD) setup, as it hosts some games on steam and i'm using linux as a main system, not the secondary. I'd understand if you would use all the info from something like redhat's VM security guide or fiddled with iptables so hard, that even skype wouldn't work, just for security reasons, but setting up vfio entirely for gaming..

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

4kGamer

Member

Registered: 2014-10-29

Posts: 88

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I need linux as a backend for Live TV. I know, there are easier and more efficient ways like building a second minimal server for it, but I like the idea of having one system that can do it all.
And regarding the 1TB SSD: there's been a great offer last week. And having all my games on one SSD isn't too bad right?

btw: I did some testing and I am sure the disk performance isn't right. Please help guys.

Last edited by 4kGamer (2014-12-13 13:36:08)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I don't have a spare ssd large enough for my VM, instead I use a small one in front of a spinning disk with bcache.  I'm currently using multi-queue virtio-scsi for the guest and it seems pretty quick.

    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source file='/mnt/store/vm/Steam.raw'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <driver queues='4'/>
      <alias name='scsi0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Duelist

Member

Registered: 2014-09-22

Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

multi-queue virtio-scsi

Can you hint on how to do that via command-line?
BTW, it seems somewhat strange that virtio-blk is slower than virtio-scsi with scsi disk attached.

---

The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

multi-queue virtio-scsi

Can you hint on how to do that via command-line?
BTW, it seems somewhat strange that virtio-blk is slower than virtio-scsi with scsi disk attached.

I think these are the relevant lines from the libvirt log:

-device virtio-scsi-pci,id=scsi0,num_queues=4,bus=pci.0,addr=0x5 \
-drive file=/mnt/store/vm/Steam.raw,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none,aio=native \
-device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1

I don't know which is faster between virtio-blk and virtio-scsi, try it and see.  I just picked up multi-queue virtio-scsi based on the virt tuning and optimization guide.  There's also x-data-plane=on support in virtio-blk-pci which you can google about.

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

oegat

Member

Registered: 2014-12-14

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I am trying to pass through an MSI geforce GTX 970 as a secondary adapter, to a Windows 8.1 vm. I get the Code 43 error in device manager, even though I seem to have followed aw:s instructions on how to work around nvidia's "bugs".  Can anyone see an explanation in what follows?

The card is a drop-in replacement for a radeon 7790, which was working with vfio passthrough just before. So my problem is arguably device-specific. Note that I'm trying secondary passthrough, so no extra vga logic involved (unless I misunderstood things and this misunderstanding is the reason for my problems).

System: Opteron magny-cours (two 6134 cpu:s), on Supermicro H8DG6
Numa turned off in cmos while testing.
Host vga: amd firepro v4900 (radeon driver)

ubuntu 14.10 (upgraded from 14.04 during testing this, in order to get support for kvm=off)
qemu 2.1/2.2 (same result)
libvirt 1.2.8
virt-manager 1.0.1
Nvidia driver: 244.11 / 244.75 (same result with both, cannot try older since the gpu is too new)

Passed thru adapters:
Geforce GTX 970 (gpu+audio): 42:00.{01} (giving code 43 error)
TI USB 73x0: 41:00.0 (usb controller for kb+mouse, works fine)

I run libvirt for help with CPU-pinning, using 'virsh edit' for settings not understood by virt-manager. The qemu command line it gives rise to is this one:

qemu-system-x86_64 
-enable-kvm 
-name Win8 
-S 
-machine pc-i440fx-2.2,accel=kvm,usb=off,mem-merge=off 
-cpu host,kvm=off 
-m 4096 
-mem-prealloc 
-mem-path /hugepages/libvirt/qemu 
-realtime mlock=on 
-smp 1,sockets=1,cores=1,threads=1 
-uuid xxx 
-no-user-config 
-nodefaults 
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/Win8.monitor,server,nowait 
-mon chardev=charmonitor,id=monitor,mode=control 
-rtc base=utc 
-no-shutdown 
-boot order=c,menu=on,strict=on 
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 
-device ahci,id=ahci0,bus=pci.0,addr=0x6 
-drive file=/dev/sdb,if=none,id=drive-ide0-0-0,format=raw 
-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 
-drive file=/dev/zvol/data/win8d,if=none,id=drive-ide0-0-1,format=raw 
-device ide-hd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 
-drive file=/opt/images/en-gb_windows_8_n_x64_dvd_915799.iso,if=none,id=drive-ide0-1-0,readonly=on,format=raw 
-device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 
-netdev tap,fd=24,id=hostnet0 
-device e1000,netdev=hostnet0,id=net0,mac=52:54:00:46:9f:a9,bus=pci.0,addr=0x3 
-netdev tap,fd=25,id=hostnet1 
-device e1000,netdev=hostnet1,id=net1,mac=52:54:00:25:10:ce,bus=pci.0,addr=0x9 
-chardev pty,id=charserial0 
-device isa-serial,chardev=charserial0,id=serial0 -device usb-tablet,id=input0 -vnc 127.0.0.1:0 
-device VGA,id=video0,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 
-device vfio-pci,host=42:00.0,id=hostdev0,bus=pci.0,addr=0x8,rombar=1 
-device vfio-pci,host=42:00.1,id=hostdev1,bus=pci.0,addr=0xa,rombar=1 
-device vfio-pci,host=41:00.0,id=hostdev2,bus=pci.0,addr=0x7 
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -msg timestamp=on

I replaced the real UUID above with xxx, and when running qemu 2.1 the machinetype was 'pc-i440fx-trusty'. Otherwise was everything the same. So the question is, am I missing something, or could it be some other previously undocumented quirk in my setup which is detected by the nvidia driver?

Unless I am mistaken that a Geforce 970 can be passed through as secondary this easily. I have assumed that options such as x-vga=on and/or custom firmware won't be needed when doing it this way. I also tried turing MSI on and off in windows, but without luck.

(As you see I am running ubuntu as host, not arch. I'm posting here anyway because I think it is a good thing to keep the collected research on vfio passthrough in one place at this stage, and because the level of abstraction of the techniques discussed here is anyway above the workings of specific package managers and service daemons. I hope I am not breaking any forum rule or such.)

Edit: broke up the qemu command line for readability.

Last edited by oegat (2014-12-14 16:22:25)

aw

Member

Registered: 2013-10-04

Posts: 921

Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Unless I am mistaken that a Geforce 970 can be passed through as secondary this easily.

You are mistaken

---

http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

oegat

Member

Registered: 2014-12-14

Posts: 3

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You are mistaken

Thanks, then I must have taken the wrong path somewhere. Explains a lot.