You are not logged in.

#5726 2015-07-24 15:49:40

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@SXX

Yes, you can make new qcow2 images with your existing as a backing store.  I think it even works with raw images, so your disk might also work as well.  Also, don't come crying to me when -snapshot either uses all your ram or fills your tmpfs and starts giving write errors, I warned you.


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5727 2015-07-24 16:17:55

SXX
Member
Registered: 2014-03-31
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@aw
Thanks for information! I'll try to setup once I have time it and report back.
Don't think that write errors anything like an issue as long as it's only affect running VM and not actual disk image. :-)

PS: BTW to change where is snapshot going "TMPDIR=/home/tmp" can be used, by default it's write into /var/tmp.

Last edited by SXX (2015-07-24 16:35:02)

Offline

#5728 2015-07-24 22:50:19

SXX
Member
Registered: 2014-03-31
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So you were right @aw! It's working with raw drive partitions just fine and IO performance with QCOW2 image is better than with "-snapshot" (e.g Google Chrome on guest start to freeze with that option when downloading file). I'd put my current configuration on gist.

Sadly VM with Nvidia have inconsistent startup success rate (e.g sometimes only load after it's started once with Linux VM) and looks like it's may cause host freeze on restart or 2nd startup. Though it's may be easily hardware failure due to overheating or who know what else (GPU is 7 years old). Anyway I successfully run Unigine Heaven on host and both VMs simultaneously.

Thanks again! Now I just need to find out how to do CPU pinning without using libvirt.

Offline

#5729 2015-07-25 07:44:36

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

SXX wrote:

Sadly VM with Nvidia have inconsistent startup success rate (e.g sometimes only load after it's started once with Linux VM) and looks like it's may cause host freeze on restart or 2nd startup. Though it's may be easily hardware failure due to overheating or who know what else (GPU is 7 years old). Anyway I successfully run Unigine Heaven on host and both VMs simultaneously.

It's too early to roar with success.

I've had that sort of problem when i ran SeaBIOS on an AMD platform - it worked just sometimes. Try disabling hugepages maybe?

Last edited by Duelist (2015-07-25 09:07:09)


The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5730 2015-07-25 12:31:23

SXX
Member
Registered: 2014-03-31
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist wrote:

Try disabling hugepages maybe?

I don't use huge pages because those never improved VM performance a lot for my use cases. BTW I only applied ACS because never needed any other patches for one GPU passthrough, can this be reason of my issues with Nvidia?

UPD: Found a way to have consistent startup success:

  1. Run Kubuntu livecd with nouveau till display activated.

  2. Now shutdown it or just kill QEMU.
    Display remain active with some image freezed on it.

  3. Boot into Windows with QXL.

  4. Go to Control Panel -> Device Manager.

  5. Disable and enable device.
    At that point cursor on freeze image goes into left top corner of screen and monitor lose signal.
    Though Windows still don't see monitor, GPU not working.

  6. Now disable and enable it once again.
    Monitor connected to Nvidia now active! It's working!

Though there is still weird issue with fan speed control. E.g if Nouveaus started with high speed fan then even after driver become working Windows VM don't decrease fan speed. Though if started with low speed than VM can control speed just fine.

UDP2: Looks like there issue with clocks too, sadly. Though one more disable / enable cycle fixes it.

Last edited by SXX (2015-07-25 15:59:51)

Offline

#5731 2015-07-26 10:01:56

impulse_255
Member
Registered: 2015-06-29
Posts: 22

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The latest OVMF dated today works for me with Windows 8.1 again.

Edit: not after all. My existing install booted up but as I was doing some experiments with a new test install, the installer still crashed when it started copying files.

Last edited by impulse_255 (2015-07-26 14:50:06)

Offline

#5732 2015-07-26 12:47:29

Denso
Member
Registered: 2014-08-30
Posts: 179

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@impulse_255 :

I approve that I can finally install Windows 10 . The code was compiled 30 minutes ago .

Offline

#5733 2015-07-26 17:54:26

kantras
Member
Registered: 2013-06-04
Posts: 12

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Using the ovmf-git AUR package, I was just able to install Windows 8 Pro earlier this morning - new install, compiled from about 6 hours ago

Last edited by kantras (2015-07-26 17:55:10)

Offline

#5734 2015-07-26 18:09:53

devianceluka
Member
Registered: 2014-05-19
Posts: 44

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

OVMF boot from physical SATA disk possible yet (SATA controller passthrough)?

Offline

#5735 2015-07-26 21:08:22

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

With the latest OVMF builds, i am unable to install windows 7.
The installer successfully does it's job, reboots the VM and the system freezes without the logo showing with 4 cores on 100%.
When force resetting the machine, next boot shows me a loading logo which freezes in the middle.
I am not passing through any devices, the only GPU is the emulated QXL...
Seems like it's time to sing along to that song, where "all men play on ten".

There's a message for us.
Anyone had same experience?

Last edited by Duelist (2015-07-26 21:43:45)


The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5736 2015-07-26 23:06:29

lersek
Member
Registered: 2015-03-15
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:

BTW, I reported the Win8.1/OVMF install issue to Laszlo who bisected and found the commit that introduced it.  There's already a patch on their list to fix it, but the ongoing sorceforge outage is delaying it being committed and new builds including it.  Stick with a pre-July build until things get moving again.

Right.

Here's some more background.

The bug breaks the UEFI variable runtime services when going from physical addressing to virtual addressing, at SetVirtualAddressMap(). The variable service drivers were rewritten recently (there had been two separate variable drivers, one for secure boot enabled platforms, and another for secure boot-less platforms -- those were unified now). During this, commits a6811666 (SVN 17758) and fa0737a8 (SVN 17765) introduced the bug. Date: 2015-Jul-01.

The rest of the timeline (mostly in UTC+02:00, ie. CEST):

  • July 16th (approx): sourceforge.net blows up. The most recent commit in our SVN repo at this point is 18032. The last SVN commit shoveled over into our git mirror corresponds to SVN rev 18030. (The mirroring daemon runs periodically.)

  • July 21st: Alex reports the installation regression to me at 1 AM approx. Patches for the bug are (independently) posted to the list at 11 AM. http://thread.gmane.org/gmane.comp.bios.edk2.devel/133 . At 2-3 PM I read Alex's  email, bisect the bug, and recall the patches from the list I saw just a few hours earlier. I test them, and they work. The patches cannot be committed because SVN is down.

  • July 23rd: SVN has been down for a week. We cannot exist without a repo any longer, not just due to the above bug(fixes), but also because other patches have been accumulating. Jordan sets up a temporary git repo (a fork of our well-known svn-to-git mirror) on github, so that we can queue up the ready-to-merge (reviewed) patches in the SVN interim. See http://thread.gmane.org/gmane.comp.bios.edk2.devel/227 . First of all, Jordan queues the two patches from the list that made it to SVN (as revs 18031 and 18032) but never to the git mirror.

  • July 23rd still: I round up the bulk of pending reviewed patches for Jordan in a personal branch, and Jordan merges them to the temporary git repo (fast forward merge), on top of 18032. These new patches include my work that ports the SMBIOS client code to ARM, and refactor the original x86 code a bit. They also include the variable driver bugfix noted above.

  • July 25th: Jordan goes on and picks up three more patches from the list that have become ready for merging

  • July 26th (today -- well, it's past midnight, so "yesterday"): the SVN repo comes back from the dead. Surprise: the most recent revision it knows about is 18014. So, in a hurry, Jordan commits patches from the git repo back to SVN (revisions 18015..18030), which thereby qualifies as restoration from backup, and commits 18031..18032 from the tempoary queue (which is still restoral from backup, theoretically).

  • July 26th still: because SVN is back, with new patches no less, SVN revisions 18031..18032 are mirrored to our semi-official git mirror. Then Gerd's Jenkins daemon performs a git pull (at the commit corresponding to SVN rev 18032: b59e5ad), and successfully builds build#1121. This is what you guys have now, and it contains the bug.

  • July 26th still: Jordan flushes the 31 new queued patches to SVN. The SVN revision number we thus reach is 18063. The temporary git queue is practically discontinued. The new SVN commits percolate to the git mirror as well. The bugfix you guys would need corresponds to SVN revs 18054 (git e43525ee) and 18055 (git f18b2162).

  • July 26th still: Jenkins build 1122 (which would correspond to SVN rev 18063 / git 88656abf1b) fails, beause my SMBIOS refactoring (SVN r18040 / git 164cf403) conflicts with some downstream-only patches we have in Gerd's build. I get an automatic email about the build failure, but I'm not surprised. Namely, I keep those same downstream patches in my local tree, so I have already adapted them. I send them to Gerd so he can resume the Jenkins build. Unfortunately, Gerd seems to have entered a phase of "not reading email for a while" already -- my email misses him and the build remains stuck at build#1121 / SVN rev 18032 / git b59e5ad, leaving you guys without the necessary fix.

Offline

#5737 2015-07-26 23:18:23

lersek
Member
Registered: 2015-03-15
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Until the Jenkins build gets sorted out, you guys can try this build: http://people.redhat.com/~lersek/ovmf-r … -4530eb75/ . It is based on 18063 and has a bunch of local patches (my usual env setup and some of my work in progress, but nothing that should interfere with your use case). No RPM, just raw binaries, but you all ought to know how to point the domain XML / libvirt's qemu.conf to them at this point. smile

Offline

#5738 2015-07-26 23:20:49

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

lersek wrote:

Here's some more background.

Whoa, what a mess you've had there.
So, I guess, the conclusion for the majority of us is to wait until Gerd Hoffman wakes up and resumes the auto builds, right?
That would be one or two days, I think, not a problem.
I'm kind of surprised that you had the patch ready in 11 hours, but couldn't deliver it for almost a week due to sourceforge storming.It's still storming, right?

UPD:

    <loader readonly='yes' type='pflash'>/mnt/hdd/qemu/OVMF_CODE.fd</loader>
    <nvram>/mnt/hdd/qemu/OVMF_VARS.fd</nvram>


I guess that would be the lines to use the files you've provided, but i'm having some troubles with permissions...

Last edited by Duelist (2015-07-26 23:50:25)


The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5739 2015-07-26 23:41:37

lersek
Member
Registered: 2015-03-15
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist wrote:

I'm kind of surprised that you had the patch ready in 11 hours, but couldn't deliver it for almost a week due to sourceforge storming.It's still storming, right?

For us at least, it's okay, I'd think. http://sourceforge.net/blog/

Duelist wrote:

UPD:

    <loader readonly='yes' type='pflash'>/mnt/hdd/qemu/OVMF_CODE.fd</loader>
    <nvram>/mnt/hdd/qemu/OVMF_VARS.fd</nvram>

I guess that would be the lines to use the files you've provided, but i'm having some troubles with permissions...

First, the above XML fragment is not entirely correct. The loader element looks okay, but the nvram element should not be updated -- you should continue using the virtual machine's private varstore. (The OVMF_CODE.fd binary that I provided should be compatible with it.) ... Actually, the loader element is the only one you have to care about, if you already have a prior VM. If you insist, the varstore *template* (which OVMF_VARS.fd is) can be specified in the "template" attribute of the nvram element.

Libvirt's qemu.conf should be modified, and libvirt restarted, if you intend to create new VMs with the files I provided. That config file is where you stick them both. So: loader is okay, and revert nvram to what it used to be earlier.

Then, regarding permissions -- qemu runs with unprivileged UID and GID, plus in an SELinux "sandbox", if you run Fedora / RHEL and launch qemu from libvirt. So for now the easiest would be probably if you just copied these new files under

/usr/share/edk2.git/ovmf-x64

(which is a directory that belongs to Gerd's edk2.git-ovmf-x64 package), and then ran:

cd /usr/share/edk2.git/ovmf-x64
chown root:root OVMF_*.fd
chmod 644 OVMF_*.fd
restorecon -FvvR .

Offline

#5740 2015-07-26 23:47:15

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

lersek wrote:
Duelist wrote:

I'm kind of surprised that you had the patch ready in 11 hours, but couldn't deliver it for almost a week due to sourceforge storming.It's still storming, right?

For us at least, it's okay, I'd think. http://sourceforge.net/blog/

Duelist wrote:

UPD:

    <loader readonly='yes' type='pflash'>/mnt/hdd/qemu/OVMF_CODE.fd</loader>
    <nvram>/mnt/hdd/qemu/OVMF_VARS.fd</nvram>

I guess that would be the lines to use the files you've provided, but i'm having some troubles with permissions...

First, the above XML fragment is not entirely correct. The loader element looks okay, but the nvram element should not be updated -- you should continue using the virtual machine's private varstore. (The OVMF_CODE.fd binary that I provided should be compatible with it.) ... Actually, the loader element is the only one you have to care about, if you already have a prior VM. If you insist, the varstore *template* (which OVMF_VARS.fd is) can be specified in the "template" attribute of the nvram element.

Libvirt's qemu.conf should be modified, and libvirt restarted, if you intend to create new VMs with the files I provided. That config file is where you stick them both. So: loader is okay, and revert nvram to what it used to be earlier.

Then, regarding permissions -- qemu runs with unprivileged UID and GID, plus in an SELinux "sandbox", if you run Fedora / RHEL and launch qemu from libvirt. So for now the easiest would be probably if you just copied these new files under

/usr/share/edk2.git/ovmf-x64

(which is a directory that belongs to Gerd's edk2.git-ovmf-x64 package), and then ran:

cd /usr/share/edk2.git/ovmf-x64
chown root:root OVMF_*.fd
chmod 644 OVMF_*.fd
restorecon -FvvR .

You are right in every sentence, i've concluded the same. VARS are not needed, it's easiest to just copy the files over, which i did.

However, that didn't resolve my windows7 installation freeze issue.(win 10 release is in two or three days, so i don't care much) I don't know for others. The time(and they) will tell.


BTW, it's awesome to read SF's blog with all those "system_xxx online" posts almost every day. That voice in my head keeps saying stuff like "Battlecruiser operational".

Last edited by Duelist (2015-07-26 23:52:35)


The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5741 2015-07-27 01:12:06

SXX
Member
Registered: 2014-03-31
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

So as I expected problem with Nvidia GTX 260 VM startup was caused by VGA arbitration and it's caused KVM error with x-vga=on too. It's still really weird because AMD GPU only had temporary gamma corruption issue, but looks like Nvidia Windows driver fail to take control over GPU when it's not initialized on boot properly.

Would be great if that eventually will be solved without EFI needed (GTX 260 doesn't support it), but for now it's will work for me as when I need two VMs I don't really care about 3D acceleration on host. I'll just reboot with different options when I need two VMs or just wouldn't worry about fan noise. Thanks to everyone who helped me!

Offline

#5742 2015-07-27 01:39:31

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@SXX

Maybe time for new hardware?  GTX 260 may have been pretty sweet 6 years ago... just sayin'


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#5743 2015-07-27 02:54:04

SXX
Member
Registered: 2014-03-31
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

No doubt about that. Though this is just what I got for free because I don't really want to buy any Nvidia products. Unfortunately still need it for humble open source game development and testing.

Later on if AMD finally going to drop Catalyst kernel blob I'll buy Fury X or something.

...

BTW is there anyone using Windows with access time update disabled? Any downside? I just think it's should decrease QCOW2 growth quite a lot. Oops, looks like windows don't enable it by default.

Last edited by SXX (2015-07-27 04:36:52)

Offline

#5744 2015-07-27 09:58:10

lersek
Member
Registered: 2015-03-15
Posts: 38

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Duelist wrote:

However, that didn't resolve my windows7 installation freeze issue.(win 10 release is in two or three days, so i don't care much) I don't know for others. The time(and they) will tell.

Can you link an earlier comment of yours about this issue, with more details? (I assume you described it earlier.)

Also, Gerd has incorporated my updated downstream-only smbios patches, and the Jenkins build is back to normal -- I just got an email about build # 1125 succeeding; I believe the RPMs too should be soon refreshed at the usual location.

Offline

#5745 2015-07-27 10:41:44

Duelist
Member
Registered: 2014-09-22
Posts: 358

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

lersek wrote:
Duelist wrote:

However, that didn't resolve my windows7 installation freeze issue.(win 10 release is in two or three days, so i don't care much) I don't know for others. The time(and they) will tell.

Can you link an earlier comment of yours about this issue, with more details? (I assume you described it earlier.)

Also, Gerd has incorporated my updated downstream-only smbios patches, and the Jenkins build is back to normal -- I just got an email about build # 1125 succeeding; I believe the RPMs too should be soon refreshed at the usual location.

Strange, it's still B1121...
11:50 UTC - B1125 is in place.

...
Ouch it hurts.
Is there a legitimate way of attaching an ISA debug console to a libvirt domain(VM)? I tried to pass the regular options as qemu:commandline, but it says that there's no such option...
Googling that problem leads me to my own messages about that debug feature.

Last edited by Duelist (2015-07-27 15:10:19)


The forum rules prohibit requesting support for distributions other than arch.
I gave up. It was too late.
What I was trying to do.
The reference about VFIO and KVM VGA passthrough.

Offline

#5746 2015-07-27 17:31:51

winie
Member
Registered: 2014-03-01
Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Does anyone know if it's possible to boot (with gpu passthrough of course) only using coreboot for linux guests and getting rid of both seabios and ovmf?
One of the reasons I'm asking is because ovmf loads really slow.

Offline

#5747 2015-07-27 17:45:41

The_Moves
Member
Registered: 2015-01-06
Posts: 59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

winie wrote:

Does anyone know if it's possible to boot (with gpu passthrough of course) only using coreboot for linux guests and getting rid of both seabios and ovmf?
One of the reasons I'm asking is because ovmf loads really slow.

http://www.coreboot.org/QEMU

Offline

#5748 2015-07-27 18:50:09

winie
Member
Registered: 2014-03-01
Posts: 17

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

The_Moves wrote:
winie wrote:

Does anyone know if it's possible to boot (with gpu passthrough of course) only using coreboot for linux guests and getting rid of both seabios and ovmf?
One of the reasons I'm asking is because ovmf loads really slow.

http://www.coreboot.org/QEMU

I mean how to get it working with vga passthrough

Offline

#5749 2015-07-27 23:22:25

The_Moves
Member
Registered: 2015-01-06
Posts: 59

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

You might try posting on their IRC channel: http://www.coreboot.org/IRC

Offline

#5750 2015-07-27 23:38:48

zir_blazer
Member
Registered: 2013-12-12
Posts: 35

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

winie wrote:
The_Moves wrote:
winie wrote:

Does anyone know if it's possible to boot (with gpu passthrough of course) only using coreboot for linux guests and getting rid of both seabios and ovmf?
One of the reasons I'm asking is because ovmf loads really slow.

http://www.coreboot.org/QEMU

I mean how to get it working with vga passthrough

You shouldn't even be able to do anything close to that, since Coreboot doesn't work in a standalone fashion. Its purpose is to initialize the Hardware components, THEN use SeaBIOS or OVMF for BIOS/UEFI support, which do the actual booting and presents the devices, ACPI, etc, to the OS. You can't really omit them.

A different matter is using Xen to make a paravirtualized VM. Since Linux is fully concious of the fact that it is a PV VM, these can skip having a Firmware. But that isn't reelevant to this Thread.

Last edited by zir_blazer (2015-07-27 23:39:30)

Offline

Board footer

Powered by FluxBB