You are not logged in.

#1 2016-08-16 19:19:38

brando56894
Member
From: NYC
Registered: 2008-08-03
Posts: 681

KVM Nested Virtualization

I have a FreeNAS 10 KVM setup via libvirt on my Arch server and I'd like to be able to test out the virtualization features in the nightly FreeNAS 10 builds but the problem is that I can't seem to get VT-x to correctly pass through to the guest. I have followed the Nested Virtualization section of the KVM wiki and I'm sure it's supported and enabled.

Here's proof

 [bran@nas ~]$ sudo systool -m kvm_intel -v | grep nested
    nested              = "Y"

 [bran@nas ~]$ lscpu|grep Virtualization
Virtualization:        VT-x

So the host/hardware isn't the problem, I believe the problem lies within libvirt.

RedHat says to use copy host CPU configuration or host-passthrough, with the latter being preferred. If I use the former I get the CMT not supported error, but if I type in host-passthrough which according to this post should work. When I set host-passthrough it allows the system to boot up, but when I try to start a guest FreeNAS gives me the error that VT-x instructions aren't available, I have no idea how to check them either since the proc pseudo-filesystem doesn't exist in BSD. I know for a fact that this isn't a problem with the FreeNAS builds because I've been testing them out for months on my Windows 10 desktop via VMware and nested virtualization works without issue, so it must be an issue with KVM/libvirt.

Can someone clue me in on what the issue is?

Here's the entire XML config for the FreeNAS VM

<!--
WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE
OVERWRITTEN AND LOST. Changes to this xml configuration should be made using:
  virsh edit FreeNAS_10
or other application using the libvirt API.
-->

<domain type='kvm'>
  <name>FreeNAS_10</name>
  <uuid>ea816b85-7685-495a-bc97-28a882f190d7</uuid>
  <title>FreeNAS v10</title>
  <description>Nightly Alpha Test Releases</description>
  <memory unit='KiB'>6340608</memory>
  <currentMemory unit='KiB'>6340608</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc-i440fx-2.6'>hvm</type>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <vmport state='off'/>
  </features>
  <cpu mode='host-passthrough'/>
  <clock offset='utc'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/sbin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/storage/vm-storage/FreeNAS_Disk1.img'/>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/storage/vm-storage/FreeNAS_Disk2.img'/>
      <target dev='vdc' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/mnt/storage/vm-storage/FreeNAS_Disk3.img'/>
      <target dev='vdd' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw' cache='directsync' io='native'/>
      <source file='/var/lib/libvirt/images/FreeNAS_10.img'/>
      <target dev='vde' bus='virtio'/>
      <boot order='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='hda' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='ich9-ehci1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x7'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci1'>
      <master startport='0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0' multifunction='on'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci2'>
      <master startport='2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x1'/>
    </controller>
    <controller type='usb' index='0' model='ich9-uhci3'>
      <master startport='4'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:24:5c:08'/>
      <source bridge='vmbridge'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target type='serial' port='0'/>
    </console>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>
    </graphics>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='1'/>
    </redirdev>
    <redirdev bus='usb' type='spicevmc'>
      <address type='usb' bus='0' port='2'/>
    </redirdev>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </memballoon>
  </devices>
</domain>

Offline

#2 2016-08-30 00:50:47

brando56894
Member
From: NYC
Registered: 2008-08-03
Posts: 681

Re: KVM Nested Virtualization

Looks like the problem is actually with Bhyve and not libvirt....

Q: Can I run multiple bhyve hosts under Linux KVM with nested VT-x EPT?

A: Maybe. You must enable it and note the known limitations. Please let us know if you are successful.

Last edited by brando56894 (2016-09-01 18:34:02)

Offline

Board footer

Powered by FluxBB