You are not logged in.

#1651 2014-05-05 11:02:01

rabcor
Banned
Registered: 2013-02-09
Posts: 500

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
Val532 wrote:
aw wrote:

This is likely because you're using the "downstream" option of the patch.  Regardless, we don't have confirmation from Intel that it's safe to do this for processor-based root ports or what configuration bits we can tweak to make it safe.  So for the time being, we've only been able to create a quirk for PCH-based root ports, which are the ones coming out of the top left box of the Z87 chipset box in the diagram above.  Unfortunately, that's often where lower speed devices are attached, but it's all we've got at the moment.  You'll need to continue to use the ACS override patch and take the risk that there could be non-IOMMU translated DMA between devices.

Ok so for the moment i have to use the old patch not the new because of probably not a safe patch.

Ok ^^.

Are your planning to make an adition to the new patch ? That add the processor-based root ports a not safe patch but just for people like me with multiple VGA passthrough ^^.

No, I proposed the ACS override patch upstream and it was rejected for fear that it once again places a dangerous tool in the arsenal that users are quick to jump on for functionality without fully understanding the implications (as evidenced by this forum wink).  The kernel will only accept quirks that the vendors identify as safe.  I expect that as a result of this, we'll see a resurgence of ACS support in Intel chipsets, but getting confirmation and code in place for existing devices is a tedious process.


So then, ACS support in intel chipsets is in? (GVT-d)

Last edited by rabcor (2014-05-05 11:05:10)

Offline

#1652 2014-05-05 12:20:31

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

rabcor wrote:
aw wrote:
Val532 wrote:

Ok so for the moment i have to use the old patch not the new because of probably not a safe patch.

Ok ^^.

Are your planning to make an adition to the new patch ? That add the processor-based root ports a not safe patch but just for people like me with multiple VGA passthrough ^^.

No, I proposed the ACS override patch upstream and it was rejected for fear that it once again places a dangerous tool in the arsenal that users are quick to jump on for functionality without fully understanding the implications (as evidenced by this forum wink).  The kernel will only accept quirks that the vendors identify as safe.  I expect that as a result of this, we'll see a resurgence of ACS support in Intel chipsets, but getting confirmation and code in place for existing devices is a tedious process.


So then, ACS support in intel chipsets is in? (GVT-d)

Unrelated


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1653 2014-05-05 16:40:52

maduraa
Member
Registered: 2014-05-01
Posts: 4

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I saw on an online forum that some motherboards have the PCI slots behind a switch and they do not support passthrough, is this true?

-[0000:00]-+-00.0
           +-01.0-[01]--+-00.0
           |            \-00.1
           +-02.0
           +-03.0
           +-14.0
           +-16.0
           +-1a.0
           +-1b.0
           +-1c.0-[02]--
           +-1c.3-[03]----00.0
           +-1c.7-[04]----00.0
           +-1d.0
           +-1f.0
           +-1f.2
           \-1f.3

Is there any thing wrong in the about lspci -t output that'd disqualify my board? 01:00.0, 01:00.1 are for VGA and VGA audio

Last edited by maduraa (2014-05-05 16:43:30)

Offline

#1654 2014-05-05 17:01:31

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

maduraa wrote:

I saw on an online forum that some motherboards have the PCI slots behind a switch and they do not support passthrough, is this true?

-[0000:00]-+-00.0
           +-01.0-[01]--+-00.0
           |            \-00.1
           +-02.0
           +-03.0
           +-14.0
           +-16.0
           +-1a.0
           +-1b.0
           +-1c.0-[02]--
           +-1c.3-[03]----00.0
           +-1c.7-[04]----00.0
           +-1d.0
           +-1f.0
           +-1f.2
           \-1f.3

Is there any thing wrong in the about lspci -t output that'd disqualify my board? 01:00.0, 01:00.1 are for VGA and VGA audio

a) PCIe switches don't prevent device assigned, b) you don't have one.  PCIe-to-PCI bridges mean that all the devices behind the bridge may need to be assigned together, which may include multiple slots... you don't have one of those either.


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1655 2014-05-06 00:48:14

Destroy
Member
Registered: 2014-05-05
Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Hi

I am not archlinux user but this post is so great, many thanks.

Unfortunately my virtual machine is not working as I expected.

My setup:
Slackware 14.1
Kernel 3.14.2 (with vfio as module and vfio-vga enabled)
qemu 1.6.2
seabios 0.6.1
Motherboard Gygabyte z77x-ud5h
Micro Intel i5 3330
1st Video: Builtin Intel
2nd Video: MSI Nvidia 7600

Using two patch for the kernel and one for the qemu.

Kernel:
1-Patch_with_quirk_for_incorrect_PCIe_requester_IDs-SATA_Marvell

@@ -, +, @@ 
---
  as 'ghost' functions.
--- a/drivers/iommu/amd_iommu_init.c	
+++ a/drivers/iommu/amd_iommu_init.c	
@@ -1023,6 +1023,62 @@ static void __init free_iommu_all(void)
 	}
 }
 
+static void quirk_map_multi_requester_ids(struct pci_dev *pdev)
+{
+	int fn;
+	u16 devid = PCI_DEVID(pdev->bus->number,
+			      pdev->devfn);
+	u8 fn_map = pci_multi_requesters(pdev);
+
+	/* this is the common case; a non-quirky device. */
+	if (!fn_map)
+		return;
+
+	/* Skip the known, already mapped func */
+	fn_map &= ~(1<<PCI_FUNC(pdev->devfn));
+
+	for (fn = 0; fn_map >> fn; fn++) {
+		if (fn_map & (1<<fn)) {
+			u8 quirk_devfn = PCI_DEVFN(PCI_SLOT(devid), fn);
+			u16 quirk_devid = PCI_DEVID(pdev->bus->number,
+						    quirk_devfn);
+			amd_iommu_alias_table[devid] = quirk_devid;
+			set_dev_entry_from_acpi(amd_iommu_rlookup_table[devid],
+						quirk_devid, 0, 0);
+			dev_dbg(&pdev->dev,
+				"requester id quirk; ghost func %d mapped", fn);
+		}
+	}
+}
+
+static void quirk_map_requester_id(struct pci_dev *pdev)
+{
+	u8 quirk_devfn = pci_requester(pdev);
+	u16 devid = PCI_DEVID(pdev->bus->number, pdev->devfn);
+	u16 quirk_devid = PCI_DEVID(pdev->bus->number, quirk_devfn);
+
+	dev_dbg(&pdev->dev, "checking for incorrect pci requester id quirk..");
+
+	if (pdev->devfn == quirk_devfn)
+		return;
+
+	amd_iommu_alias_table[devid] = quirk_devid;
+	set_dev_entry_from_acpi(amd_iommu_rlookup_table[devid],
+				quirk_devid, 0, 0);
+}
+
+static void amd_iommu_pci_quirks(void)
+{
+	struct pci_dev *pdev = NULL;
+
+	for_each_pci_dev(pdev) {
+		/* Check for devices that use multiple requester IDs */
+		quirk_map_multi_requester_ids(pdev);
+
+		quirk_map_requester_id(pdev);
+	}
+}
+
 /*
  * Family15h Model 10h-1fh erratum 746 (IOMMU Logging May Stall Translations)
  * Workaround:
@@ -1336,6 +1392,8 @@ static int __init amd_iommu_init_pci(void)
 			break;
 	}
 
+	amd_iommu_pci_quirks();
+
 	ret = amd_iommu_init_devices();
 
 	print_iommu_info();
--- a/drivers/iommu/intel-iommu.c	
+++ a/drivers/iommu/intel-iommu.c	
@@ -1649,6 +1649,108 @@ static int domain_context_mapping_one(struct dmar_domain *domain, int segment,
 	return 0;
 }
 
+static void iommu_detach_dev(struct intel_iommu *iommu, u8 bus, u8 devfn);
+
+static void quirk_unmap_multi_requesters(struct pci_dev *pdev, u8 fn_map)
+{
+	int fn;
+	struct intel_iommu *iommu = device_to_iommu(pci_domain_nr(pdev->bus),
+						pdev->bus->number, pdev->devfn);
+
+	/* Skip the known, already mapped func */
+	fn_map &= ~(1<<PCI_FUNC(pdev->devfn));
+
+	for (fn = 0; fn_map >> fn; fn++) {
+		if (fn_map & (1<<fn)) {
+			iommu_detach_dev(iommu,
+					 pdev->bus->number,
+					 PCI_DEVFN(PCI_SLOT(pdev->devfn), fn));
+			dev_dbg(&pdev->dev,
+				"requester id quirk; ghost func %d unmapped",
+				fn);
+		}
+	}
+}
+
+/* For quirky devices that use multiple requester ids. */
+static int quirk_map_multi_requester_ids(struct dmar_domain *domain,
+		struct pci_dev *pdev,
+		int translation)
+{
+	int fn, err = 0;
+	u8 fn_map = pci_multi_requesters(pdev);
+
+	/* this is the common case; a non-quirky device. */
+	if (!fn_map)
+		return 0;
+
+	/* Skip the known, already mapped func */
+	fn_map &= ~(1<<PCI_FUNC(pdev->devfn));
+
+	for (fn = 0; fn_map >> fn; fn++) {
+		if (fn_map & (1<<fn)) {
+			err = domain_context_mapping_one(domain,
+					pci_domain_nr(pdev->bus),
+					pdev->bus->number,
+					PCI_DEVFN(PCI_SLOT(pdev->devfn), fn),
+					translation);
+			if (err) {
+				dev_err(&pdev->dev,
+					"mapping ghost func %d failed", fn);
+				quirk_unmap_multi_requesters(pdev,
+					fn_map & ((1<<fn)-1));
+				return err;
+			}
+			dev_dbg(&pdev->dev,
+				"requester id quirk; ghost func %d mapped", fn);
+		}
+	}
+	return 0;
+}
+
+static void quirk_unmap_requester_id(struct pci_dev *pdev)
+{
+	u8 devfn = pci_requester(pdev);
+	struct intel_iommu *iommu = device_to_iommu(pci_domain_nr(pdev->bus),
+						pdev->bus->number, pdev->devfn);
+
+	if (pdev->devfn == devfn)
+		return;
+
+	iommu_detach_dev(iommu,	pdev->bus->number, devfn);
+	dev_dbg(&pdev->dev, "requester id quirk; bugged device unmapped");
+}
+
+static int quirk_map_requester_id(struct dmar_domain *domain,
+		struct pci_dev *pdev,
+		int translation)
+{
+	u8 devfn = pci_requester(pdev);
+	int err;
+
+	dev_dbg(&pdev->dev,
+		"checking for incorrect pci requester id quirk...");
+
+	if (pdev->devfn == devfn)
+		return 0;
+
+	err = domain_context_mapping_one(domain,
+			pci_domain_nr(pdev->bus),
+			pdev->bus->number,
+			devfn,
+			translation);
+	if (err) {
+		dev_err(&pdev->dev,
+			"requester id quirk: mapping dev %02x:%02x.%d failed",
+			pdev->bus->number, PCI_SLOT(devfn), PCI_FUNC(devfn));
+		return err;
+	}
+	dev_dbg(&pdev->dev,
+		"requester id quirk; dmar context entry added: %02x:%02x.%d",
+		pdev->bus->number, PCI_SLOT(devfn), PCI_FUNC(devfn));
+	return 0;
+}
+
 static int
 domain_context_mapping(struct dmar_domain *domain, struct pci_dev *pdev,
 			int translation)
@@ -1662,6 +1764,16 @@ domain_context_mapping(struct dmar_domain *domain, struct pci_dev *pdev,
 	if (ret)
 		return ret;
 
+	/* quirk for devices using multiple pci requester ids */
+	ret = quirk_map_multi_requester_ids(domain, pdev, translation);
+	if (ret)
+		return ret;
+
+	/* quirk for devices using single incorrect pci requester id */
+	ret = quirk_map_requester_id(domain, pdev, translation);
+	if (ret)
+		return ret;
+
 	/* dependent device mapping */
 	tmp = pci_find_upstream_pcie_bridge(pdev);
 	if (!tmp)
@@ -3766,6 +3878,9 @@ static void domain_remove_one_dev_info(struct dmar_domain *domain,
 			iommu_disable_dev_iotlb(info);
 			iommu_detach_dev(iommu, info->bus, info->devfn);
 			iommu_detach_dependent_devices(iommu, pdev);
+			quirk_unmap_multi_requesters(pdev,
+						pci_multi_requesters(pdev));
+			quirk_unmap_requester_id(pdev);
 			free_devinfo_mem(info);
 
 			spin_lock_irqsave(&device_domain_lock, flags);
--- a/drivers/pci/quirks.c	
+++ a/drivers/pci/quirks.c	
@@ -3336,6 +3336,10 @@ static struct pci_dev *pci_func_0_dma_source(struct pci_dev *dev)
 	return pci_get_slot(dev->bus, PCI_DEVFN(PCI_SLOT(dev->devfn), 0));
 }
 
+/* Table of source functions for real devices. The DMA requests for the
+ * device are tagged with a different real function as source. This is
+ * relevant to multifunction devices.
+ */
 static const struct pci_dev_dma_source {
 	u16 vendor;
 	u16 device;
@@ -3362,7 +3366,8 @@ static const struct pci_dev_dma_source {
  * the device doing the DMA, but sometimes hardware is broken and will
  * tag the DMA as being sourced from a different device.  This function
  * allows that translation.  Note that the reference count of the
- * returned device is incremented on all paths.
+ * returned device is incremented on all paths. Translation is done when
+ * the device is added to an IOMMU group.
  */
 struct pci_dev *pci_get_dma_source(struct pci_dev *dev)
 {
@@ -3423,6 +3428,148 @@ static int pci_quirk_amd_sb_acs(struct pci_dev *dev, u16 acs_flags)
 #endif
 }
 
+/* Table of multiple requester IDs, aka ghost DMA source functions.
+ * Devices that may need this quirk show the following behaviour:
+ * 1. the device may use multiple PCI requester IDs during operation,
+ *     (eg. one pci transaction uses xx:yy.0, the next uses xx:yy.1)
+ * 2. the requester ID may point to an absent/ghost device.
+ *     (eg. lspci does not show xx:yy.1 to be present)
+ *
+ * The bitmap contains ALL of the functions used as requester IDs by the
+ * device, including the one known to 'lspci'. If the func is known to 'lspci',
+ * but not used for DMA, don't include it.
+ * See  https://bugzilla.redhat.com/show_bug.cgi?id=757166,
+ * https://bugzilla.kernel.org/show_bug.cgi?id=42679
+ * https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1089768
+ */
+static const struct pci_dev_dma_multi_source_map {
+	u16 vendor;
+	u16 device;
+	u8 func_map;	/* bit map. lsb is fn 0. */
+} pci_dev_dma_multi_source_map[] = {
+	 /* Reported by Patrick Bregman
+	  * https://bugzilla.redhat.com/show_bug.cgi?id=863653 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9120, (1<<0)|(1<<1)},
+
+	/* Reported by  Paweł Żak, Korneliusz Jarzębski, Daniel Mayer
+	 * https://bugzilla.kernel.org/show_bug.cgi?id=42679 and by
+	 * Justin Piszcz  https://lkml.org/lkml/2012/11/24/94 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9123, (1<<0)|(1<<1)},
+
+	/* Used in a patch by Ying Chu
+	 * https://bugzilla.redhat.com/show_bug.cgi?id=757166 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9125, (1<<0)|(1<<1)},
+
+	/* Reported by Robert Cicconetti
+	 * https://bugzilla.kernel.org/show_bug.cgi?id=42679 and by
+	 * Fernando https://bugzilla.redhat.com/show_bug.cgi?id=757166 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9128, (1<<0)|(1<<1)},
+
+	/* Reported by Stijn Tintel
+	 * https://bugzilla.kernel.org/show_bug.cgi?id=42679 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9130, (1<<0)|(1<<1)},
+
+	/* Reported by Gaudenz Steinlin
+	 * https://lkml.org/lkml/2013/3/5/288 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9143, (1<<0)|(1<<1)},
+
+	/* Reported by Andrew Cooks
+	 * https://bugzilla.kernel.org/show_bug.cgi?id=42679 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9172, (1<<0)|(1<<1)},
+
+	/* Reported by Martin Öhrling
+	 * https://bugzilla.kernel.org/show_bug.cgi?id=42679#c28 */
+	{ PCI_VENDOR_ID_MARVELL_EXT, 0x9230, (1<<0)|(1<<1)},
+
+	{ 0 }
+};
+
+/*
+ * The mapping of quirky requester ids is used when the device driver sets up
+ * dma, if iommu is enabled.
+ */
+u8 pci_multi_requesters(struct pci_dev *dev)
+{
+	const struct pci_dev_dma_multi_source_map *i;
+
+	for (i = pci_dev_dma_multi_source_map; i->func_map; i++) {
+		if ((i->vendor == dev->vendor ||
+		     i->vendor == (u16)PCI_ANY_ID) &&
+		    (i->device == dev->device ||
+		     i->device == (u16)PCI_ANY_ID)) {
+			return i->func_map;
+		}
+	}
+	return 0;
+}
+
+/* These are one-to-one translations for devices that use a single incorrect
+ * requester ID. The requester id may not be the BDF of a real device.
+ */
+static const struct pci_dev_dma_source_map {
+	u16 vendor;
+	u16 device;
+	u8  devfn;
+	u8  dma_devfn;
+} pci_dev_dma_source_map[] = {
+	/* Ricoh IEEE 1394 Controller */
+	{
+		PCI_VENDOR_ID_RICOH,
+		0xe832,
+		PCI_DEVFN(0x00, 3),
+		PCI_DEVFN(0x00, 0)
+	},
+
+	/* Nils Caspar - Adaptec 3405
+	 * http://www.mail-archive.com/centos@centos.org/msg90986.html
+	 * Jonathan McCune
+	 * http://old-list-archives.xen.org/archives/html/xen-users/2010-04/msg00535.html */
+	{
+		PCI_VENDOR_ID_ADAPTEC2,
+		0x028b,
+		PCI_DEVFN(0x0e, 0),
+		PCI_DEVFN(0x01, 0)
+	},
+
+	/* Mateusz Murawski - LSI SAS based MegaRAID
+	 * https://lkml.org/lkml/2011/9/12/104
+	 * M. Nunberg - Dell PERC 5/i Integrated RAID Controller
+	 * http://lists.xen.org/archives/html/xen-devel/2010-05/msg01563.html */
+	{
+		PCI_VENDOR_ID_LSI_LOGIC,
+		0x0411,
+		PCI_DEVFN(0x0e, 0),
+		PCI_DEVFN(0x08, 0)
+	},
+
+	/* Steven Dake, Markus Stockhausen - Mellanox 26428
+	 * https://bugzilla.redhat.com/show_bug.cgi?id=713774
+	 * Note: mellanox uses decimal product numbers, convert to hex for PCI
+	 * device ID. ie. 26428 == 0x673c */
+	{
+		PCI_VENDOR_ID_MELLANOX,
+		0x673c,
+		PCI_DEVFN(0x00, 0),
+		PCI_DEVFN(0x00, 6)
+	},
+
+	{ 0 }
+};
+
+u8 pci_requester(struct pci_dev *dev)
+{
+	const struct pci_dev_dma_source_map *i;
+
+	for (i = pci_dev_dma_source_map; i->vendor; i++) {
+		if ((i->vendor == dev->vendor) &&
+		    (i->device == dev->device) &&
+		    (i->devfn == dev->devfn)) {
+			return i->dma_devfn;
+		}
+	}
+	return dev->devfn;
+}
+
 static const struct pci_dev_acs_enabled {
 	u16 vendor;
 	u16 device;
--- a/include/linux/pci.h	
+++ a/include/linux/pci.h	
@@ -1509,6 +1509,8 @@ enum pci_fixup_pass {
 #ifdef CONFIG_PCI_QUIRKS
 void pci_fixup_device(enum pci_fixup_pass pass, struct pci_dev *dev);
 struct pci_dev *pci_get_dma_source(struct pci_dev *dev);
+u8 pci_multi_requesters(struct pci_dev *dev);
+u8 pci_requester(struct pci_dev *dev);
 int pci_dev_specific_acs_enabled(struct pci_dev *dev, u16 acs_flags);
 #else
 static inline void pci_fixup_device(enum pci_fixup_pass pass,
@@ -1517,6 +1519,14 @@ static inline struct pci_dev *pci_get_dma_source(struct pci_dev *dev)
 {
 	return pci_dev_get(dev);
 }
+u8 pci_multi_requesters(struct pci_dev *dev)
+{
+	return 0;
+}
+u8 pci_requester(struct pci_dev *dev)
+{
+	return dev->devfn;
+}
 static inline int pci_dev_specific_acs_enabled(struct pci_dev *dev,
 					       u16 acs_flags)
 {

2-1- Add_PCI_walk_function_and_PCIe_bridge_test

diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index d0627fa..0357f74 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -17,6 +17,63 @@
 DECLARE_RWSEM(pci_bus_sem);
 EXPORT_SYMBOL_GPL(pci_bus_sem);
 
+/* Test for PCIe bridges. */
+bool pci_is_pcie_bridge(struct pci_dev *pdev)
+{
+	if (!pci_is_bridge(pdev))
+		return false;
+
+	if (pci_is_pcie(pdev))
+		return true;
+
+#ifdef CONFIG_PCI_QUIRKS
+	/*
+	 * If we're not on the root bus, look one device upstream of the
+	 * current device.  If that device is PCIe and is not a PCIe-to-PCI
+	 * bridge, then the current device is effectively PCIe as it must
+	 * be the PCIe-to-PCI bridge.  This handles several bridges that
+	 * violate the PCIe spec by not exposing a PCIe capability:
+	 * https://bugzilla.kernel.org/show_bug.cgi?id=44881
+	 */
+	if (!pci_is_root_bus(pdev->bus)) {
+		struct pci_dev *parent = pdev->bus->self;
+
+		if (pci_is_pcie(parent) &&
+		    pci_pcie_type(parent) != PCI_EXP_TYPE_PCI_BRIDGE)
+
+			return true;
+	}
+#endif
+	return false;
+}
+
+/*
+ * Walk upstream from the given pdev for the first device returning
+ * true for the provided match function.  If no match is found, return
+ * NULL.  *last records the previous step in the walk.
+ */
+struct pci_dev *pci_walk_up_to_first_match(struct pci_dev *pdev,
+					   bool (*match)(struct pci_dev *),
+					   struct pci_dev **last)
+{
+	*last = NULL;
+
+	if (match(pdev))
+		return pdev;
+
+	*last = pdev;
+
+	while (!pci_is_root_bus(pdev->bus)) {
+		*last = pdev;
+		pdev = pdev->bus->self;
+
+		if (match(pdev))
+			return pdev;
+	}
+
+	return NULL;
+}
+
 /*
  * find the upstream PCIe-to-PCI bridge of a PCI device
  * if the device is PCIE, return NULL
diff --git a/include/linux/pci.h b/include/linux/pci.h
index bd8ec30..e87423a 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1855,6 +1855,29 @@ static inline struct eeh_dev *pci_dev_to_eeh_dev(struct pci_dev *pdev)
 #endif
 
 /**
+ * pci_walk_up_to_first_match - Generic upstream search function
+ * @pdev: starting PCI device to search
+ * @match: match function to call on each device (true = match)
+ * @last: last device examined prior to returned device
+ *
+ * Walk upstream from the given device, calling match() at each device.
+ * Returns the first device matching match().  If the root bus is reached
+ * without finding a match, return NULL.  last returns the N-1 step in
+ * the search path.
+ */
+struct pci_dev *pci_walk_up_to_first_match(struct pci_dev *pdev,
+					   bool (*match)(struct pci_dev *),
+					   struct pci_dev **last);
+
+/**
+ * pci_is_pcie_bridge - Match a PCIe bridge device
+ * @pdev: device to test
+ *
+ * Return true if the given device is a PCIe bridge, false otherwise.
+ */
+bool pci_is_pcie_bridge(struct pci_dev *pdev);
+
+/**
  * pci_find_upstream_pcie_bridge - find upstream PCIe-to-PCI bridge of a device
  * @pdev: the PCI device
  *

2-2-Add_PCI_walk_function_and_PCIe_bridge_test

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 0099667..277c18f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1688,9 +1688,13 @@ domain_context_mapping(struct dmar_domain *domain, struct pci_dev *pdev,
 		return ret;
 
 	/* dependent device mapping */
-	tmp = pci_find_upstream_pcie_bridge(pdev);
-	if (!tmp)
+	if (pci_is_pcie(pdev) || pci_is_root_bus(pdev->bus))
 		return 0;
+
+	tmp = pci_walk_up_to_first_match(pdev, pci_is_pcie_bridge, &parent);
+	if (!tmp)
+		tmp = parent;
+
 	/* Secondary interface's bus number and devfn 0 */
 	parent = pdev->bus->self;
 	while (parent != tmp) {
@@ -1702,7 +1706,7 @@ domain_context_mapping(struct dmar_domain *domain, struct pci_dev *pdev,
 			return ret;
 		parent = parent->bus->self;
 	}
-	if (pci_is_pcie(tmp)) /* this is a PCIe-to-PCI bridge */
+	if (pci_is_pcie_bridge(tmp)) /* this is a PCIe-to-PCI bridge */
 		return domain_context_mapping_one(domain,
 					pci_domain_nr(tmp->subordinate),
 					tmp->subordinate->number, 0,
@@ -1729,10 +1733,15 @@ static int domain_context_mapped(struct pci_dev *pdev)
 	ret = device_context_mapped(iommu, pdev->bus->number, pdev->devfn);
 	if (!ret)
 		return ret;
+
 	/* dependent device mapping */
-	tmp = pci_find_upstream_pcie_bridge(pdev);
+	if (pci_is_pcie(pdev) || pci_is_root_bus(pdev->bus))
+		return 0;
+
+	tmp = pci_walk_up_to_first_match(pdev, pci_is_pcie_bridge, &parent);
 	if (!tmp)
-		return ret;
+		tmp = parent;
+
 	/* Secondary interface's bus number and devfn 0 */
 	parent = pdev->bus->self;
 	while (parent != tmp) {
@@ -1742,7 +1751,7 @@ static int domain_context_mapped(struct pci_dev *pdev)
 			return ret;
 		parent = parent->bus->self;
 	}
-	if (pci_is_pcie(tmp))
+	if (pci_is_pcie_bridge(tmp))
 		return device_context_mapped(iommu, tmp->subordinate->number,
 					     0);
 	else
@@ -1973,7 +1982,7 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
 	struct intel_iommu *iommu;
 	struct dmar_drhd_unit *drhd;
 	struct device_domain_info *info, *tmp;
-	struct pci_dev *dev_tmp;
+	struct pci_dev *dev_tmp = NULL;
 	unsigned long flags;
 	int bus = 0, devfn = 0;
 	int segment;
@@ -1985,9 +1994,15 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
 
 	segment = pci_domain_nr(pdev->bus);
 
-	dev_tmp = pci_find_upstream_pcie_bridge(pdev);
-	if (dev_tmp) {
-		if (pci_is_pcie(dev_tmp)) {
+	if (!pci_is_pcie(pdev) && !pci_is_root_bus(pdev->bus)) {
+		struct pci_dev *last;
+
+		dev_tmp = pci_walk_up_to_first_match(pdev, pci_is_pcie_bridge,
+						     &last);
+		if (!dev_tmp)
+			dev_tmp = last;
+
+		if (pci_is_pcie_bridge(dev_tmp)) {
 			bus = dev_tmp->subordinate->number;
 			devfn = 0;
 		} else {
@@ -3738,26 +3753,24 @@ static void iommu_detach_dependent_devices(struct intel_iommu *iommu,
 {
 	struct pci_dev *tmp, *parent;
 
-	if (!iommu || !pdev)
+	if (!iommu || !pdev || pci_is_pcie(pdev) || pci_is_root_bus(pdev->bus))
 		return;
 
 	/* dependent device detach */
-	tmp = pci_find_upstream_pcie_bridge(pdev);
+	tmp = pci_walk_up_to_first_match(pdev, pci_is_pcie_bridge, &parent);
+	if (!tmp)
+		tmp = parent;
+
 	/* Secondary interface's bus number and devfn 0 */
-	if (tmp) {
-		parent = pdev->bus->self;
-		while (parent != tmp) {
-			iommu_detach_dev(iommu, parent->bus->number,
-					 parent->devfn);
-			parent = parent->bus->self;
-		}
-		if (pci_is_pcie(tmp)) /* this is a PCIe-to-PCI bridge */
-			iommu_detach_dev(iommu,
-				tmp->subordinate->number, 0);
-		else /* this is a legacy PCI bridge */
-			iommu_detach_dev(iommu, tmp->bus->number,
-					 tmp->devfn);
+	parent = pdev->bus->self;
+	while (parent != tmp) {
+		iommu_detach_dev(iommu, parent->bus->number, parent->devfn);
+		parent = parent->bus->self;
 	}
+	if (pci_is_pcie_bridge(tmp)) /* this is a PCIe-to-PCI bridge */
+		iommu_detach_dev(iommu, tmp->subordinate->number, 0);
+	else /* this is a legacy PCI bridge */
+		iommu_detach_dev(iommu, tmp->bus->number, tmp->devfn);
 }
 
 static void domain_remove_one_dev_info(struct dmar_domain *domain,
@@ -4148,7 +4161,7 @@ static void swap_pci_ref(struct pci_dev **from, struct pci_dev *to)
 static int intel_iommu_add_device(struct device *dev)
 {
 	struct pci_dev *pdev = to_pci_dev(dev);
-	struct pci_dev *bridge, *dma_pdev = NULL;
+	struct pci_dev *dma_pdev = NULL;
 	struct iommu_group *group;
 	int ret;
 
@@ -4156,9 +4169,15 @@ static int intel_iommu_add_device(struct device *dev)
 			     pdev->bus->number, pdev->devfn))
 		return -ENODEV;
 
-	bridge = pci_find_upstream_pcie_bridge(pdev);
-	if (bridge) {
-		if (pci_is_pcie(bridge))
+	if (!pci_is_pcie(pdev) && !pci_is_root_bus(pdev->bus)) {
+		struct pci_dev *bridge, *last;
+
+		bridge = pci_walk_up_to_first_match(pdev, pci_is_pcie_bridge,
+						    &last);
+		if (!bridge)
+			bridge = last;
+
+		if (pci_is_pcie_bridge(bridge))
 			dma_pdev = pci_get_domain_bus_and_slot(
 						pci_domain_nr(pdev->bus),
 						bridge->subordinate->number, 0);
diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index f3b8f23..873da76 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -373,7 +373,6 @@ static int set_hpet_sid(struct irte *irte, u8 id)
 
 static int set_msi_sid(struct irte *irte, struct pci_dev *dev)
 {
-	struct pci_dev *bridge;
 
 	if (!irte || !dev)
 		return -1;
@@ -385,9 +384,15 @@ static int set_msi_sid(struct irte *irte, struct pci_dev *dev)
 		return 0;
 	}
 
-	bridge = pci_find_upstream_pcie_bridge(dev);
-	if (bridge) {
-		if (pci_is_pcie(bridge))/* this is a PCIe-to-PCI/PCIX bridge */
+	if (!pci_is_pcie(dev) && !pci_is_root_bus(dev->bus)) {
+		struct pci_dev *bridge, *last;
+
+		bridge = pci_walk_up_to_first_match(dev,
+						    pci_is_pcie_bridge, &last);
+		if (!bridge)
+			bridge = last;
+
+		if (pci_is_pcie_bridge(bridge)) /* PCIe-to-PCI/PCIX bridge */
 			set_irte_sid(irte, SVT_VERIFY_BUS, SQ_ALL_16,
 				(bridge->bus->number << 8) | dev->bus->number);
 		else /* this is a legacy PCI bridge */

2-3-Add_PCI_walk_function_and_PCIe_bridge_test

diff --git a/drivers/pci/search.c b/drivers/pci/search.c
index 0357f74..bdbff05 100644
--- a/drivers/pci/search.c
+++ b/drivers/pci/search.c
@@ -74,41 +74,6 @@ struct pci_dev *pci_walk_up_to_first_match(struct pci_dev *pdev,
 	return NULL;
 }
 
-/*
- * find the upstream PCIe-to-PCI bridge of a PCI device
- * if the device is PCIE, return NULL
- * if the device isn't connected to a PCIe bridge (that is its parent is a
- * legacy PCI bridge and the bridge is directly connected to bus 0), return its
- * parent
- */
-struct pci_dev *
-pci_find_upstream_pcie_bridge(struct pci_dev *pdev)
-{
-	struct pci_dev *tmp = NULL;
-
-	if (pci_is_pcie(pdev))
-		return NULL;
-	while (1) {
-		if (pci_is_root_bus(pdev->bus))
-			break;
-		pdev = pdev->bus->self;
-		/* a p2p bridge */
-		if (!pci_is_pcie(pdev)) {
-			tmp = pdev;
-			continue;
-		}
-		/* PCI device should connect to a PCIe bridge */
-		if (pci_pcie_type(pdev) != PCI_EXP_TYPE_PCI_BRIDGE) {
-			/* Busted hardware? */
-			WARN_ON_ONCE(1);
-			return NULL;
-		}
-		return pdev;
-	}
-
-	return tmp;
-}
-
 static struct pci_bus *pci_do_find_bus(struct pci_bus *bus, unsigned char busnr)
 {
 	struct pci_bus* child;
diff --git a/include/linux/pci.h b/include/linux/pci.h
index e87423a..9d9dcf5 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -1877,15 +1877,4 @@ struct pci_dev *pci_walk_up_to_first_match(struct pci_dev *pdev,
  */
 bool pci_is_pcie_bridge(struct pci_dev *pdev);
 
-/**
- * pci_find_upstream_pcie_bridge - find upstream PCIe-to-PCI bridge of a device
- * @pdev: the PCI device
- *
- * if the device is PCIE, return NULL
- * if the device isn't connected to a PCIe bridge (that is its parent is a
- * legacy PCI bridge and the bridge is directly connected to bus 0), return its
- * parent
- */
-struct pci_dev *pci_find_upstream_pcie_bridge(struct pci_dev *pdev);
-
 #endif /* LINUX_PCI_H */

A patch for qemu:
1-vfio-pci_Lazy_PCI_option_ROM_loading

diff --git a/hw/misc/vfio.c b/hw/misc/vfio.c
index ede026d..730dec5 100644
--- a/hw/misc/vfio.c
+++ b/hw/misc/vfio.c
@@ -166,6 +166,7 @@ typedef struct VFIODevice {
     off_t config_offset; /* Offset of config space region within device fd */
     unsigned int rom_size;
     off_t rom_offset; /* Offset of ROM region within device fd */
+    void *rom;
     int msi_cap_size;
     VFIOMSIVector *msi_vectors;
     VFIOMSIXInfo *msix;
@@ -1058,6 +1059,125 @@ static const MemoryRegionOps vfio_bar_ops = {
     .endianness = DEVICE_LITTLE_ENDIAN,
 };
 
+static void vfio_pci_load_rom(VFIODevice *vdev)
+{
+    struct vfio_region_info reg_info = {
+        .argsz = sizeof(reg_info),
+        .index = VFIO_PCI_ROM_REGION_INDEX
+    };
+    uint64_t size;
+    off_t off = 0;
+    size_t bytes;
+
+    if (ioctl(vdev->fd, VFIO_DEVICE_GET_REGION_INFO, &reg_info)) {
+        error_report("vfio: Error getting ROM info: %m");
+        return;
+    }
+
+    DPRINTF("Device %04x:%02x:%02x.%x ROM:\n", vdev->host.domain,
+            vdev->host.bus, vdev->host.slot, vdev->host.function);
+    DPRINTF("  size: 0x%lx, offset: 0x%lx, flags: 0x%lx\n",
+            (unsigned long)reg_info.size, (unsigned long)reg_info.offset,
+            (unsigned long)reg_info.flags);
+
+    vdev->rom_size = size = reg_info.size;
+    vdev->rom_offset = reg_info.offset;
+
+    if (!vdev->rom_size) {
+        return;
+    }
+
+    vdev->rom = g_malloc(size);
+    memset(vdev->rom, 0xff, size);
+
+    while (size) {
+        bytes = pread(vdev->fd, vdev->rom + off, size, vdev->rom_offset + off);
+        if (bytes == 0) {
+            break;
+        } else if (bytes > 0) {
+            off += bytes;
+            size -= bytes;
+        } else {
+            if (errno == EINTR || errno == EAGAIN) {
+                continue;
+            }
+            error_report("vfio: Error reading device ROM: %m");
+            break;
+        }
+    }
+}
+
+static uint64_t vfio_rom_read(void *opaque, hwaddr addr, unsigned size)
+{
+    VFIODevice *vdev = opaque;
+    uint64_t val = ((uint64_t)1 << (size * 8)) - 1;
+
+    /* Load the ROM lazily when the guest tries to read it */
+    if (unlikely(!vdev->rom)) {
+        vfio_pci_load_rom(vdev);
+    }
+
+    memcpy(&val, vdev->rom + addr,
+           (addr < vdev->rom_size) ? MIN(size, vdev->rom_size - addr) : 0);
+
+    DPRINTF("%s(%04x:%02x:%02x.%x, 0x%"HWADDR_PRIx", 0x%x) = 0x%"PRIx64"\n",
+            __func__, vdev->host.domain, vdev->host.bus, vdev->host.slot,
+            vdev->host.function, addr, size, val);
+
+    return val;
+}
+
+static const MemoryRegionOps vfio_rom_ops = {
+    .read = vfio_rom_read,
+    .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void vfio_pci_size_rom(VFIODevice *vdev)
+{
+    uint32_t orig, size = (uint32_t)PCI_ROM_ADDRESS_MASK;
+    off_t offset = vdev->config_offset + PCI_ROM_ADDRESS;
+    char name[32];
+
+    if (vdev->pdev.romfile || !vdev->pdev.rom_bar) {
+        return;
+    }
+
+    /*
+     * Use the same size ROM BAR as the physical device.  The contents
+     * will get filled in later when the guest tries to read it.
+     */
+    if (pread(vdev->fd, &orig, 4, offset) != 4 ||
+        pwrite(vdev->fd, &size, 4, offset) != 4 ||
+        pread(vdev->fd, &size, 4, offset) != 4 ||
+        pwrite(vdev->fd, &orig, 4, offset) != 4) {
+        error_report("%s(%04x:%02x:%02x.%x) failed: %m",
+                     __func__, vdev->host.domain, vdev->host.bus,
+                     vdev->host.slot, vdev->host.function);
+        return;
+    }
+
+    size = ~(size & PCI_ROM_ADDRESS_MASK) + 1;
+
+    if (!size) {
+        return;
+    }
+
+    DPRINTF("%04x:%02x:%02x.%x ROM size 0x%x\n", vdev->host.domain,
+            vdev->host.bus, vdev->host.slot, vdev->host.function, size);
+
+    snprintf(name, sizeof(name), "vfio[%04x:%02x:%02x.%x].rom",
+             vdev->host.domain, vdev->host.bus, vdev->host.slot,
+             vdev->host.function);
+
+    memory_region_init_io(&vdev->pdev.rom, OBJECT(vdev),
+                          &vfio_rom_ops, vdev, name, size);
+
+    pci_register_bar(&vdev->pdev, PCI_ROM_SLOT,
+                     PCI_BASE_ADDRESS_SPACE_MEMORY, &vdev->pdev.rom);
+
+    vdev->pdev.has_rom = true;
+}
+
 static void vfio_vga_write(void *opaque, hwaddr addr,
                            uint64_t data, unsigned size)
 {
@@ -2638,51 +2758,6 @@ static int vfio_add_capabilities(VFIODevice *vdev)
     return vfio_add_std_cap(vdev, pdev->config[PCI_CAPABILITY_LIST]);
 }
 
-static int vfio_load_rom(VFIODevice *vdev)
-{
-    uint64_t size = vdev->rom_size;
-    char name[32];
-    off_t off = 0, voff = vdev->rom_offset;
-    ssize_t bytes;
-    void *ptr;
-
-    /* If loading ROM from file, pci handles it */
-    if (vdev->pdev.romfile || !vdev->pdev.rom_bar || !size) {
-        return 0;
-    }
-
-    DPRINTF("%s(%04x:%02x:%02x.%x)\n", __func__, vdev->host.domain,
-            vdev->host.bus, vdev->host.slot, vdev->host.function);
-
-    snprintf(name, sizeof(name), "vfio[%04x:%02x:%02x.%x].rom",
-             vdev->host.domain, vdev->host.bus, vdev->host.slot,
-             vdev->host.function);
-    memory_region_init_ram(&vdev->pdev.rom, OBJECT(vdev), name, size);
-    ptr = memory_region_get_ram_ptr(&vdev->pdev.rom);
-    memset(ptr, 0xff, size);
-
-    while (size) {
-        bytes = pread(vdev->fd, ptr + off, size, voff + off);
-        if (bytes == 0) {
-            break; /* expect that we could get back less than the ROM BAR */
-        } else if (bytes > 0) {
-            off += bytes;
-            size -= bytes;
-        } else {
-            if (errno == EINTR || errno == EAGAIN) {
-                continue;
-            }
-            error_report("vfio: Error reading device ROM: %m");
-            memory_region_destroy(&vdev->pdev.rom);
-            return -errno;
-        }
-    }
-
-    pci_register_bar(&vdev->pdev, PCI_ROM_SLOT, 0, &vdev->pdev.rom);
-    vdev->pdev.has_rom = true;
-    return 0;
-}
-
 static int vfio_connect_container(VFIOGroup *group)
 {
     VFIOContainer *container;
@@ -2916,22 +2991,6 @@ static int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vdev)
         QLIST_INIT(&vdev->bars[i].quirks);
     }
 
-    reg_info.index = VFIO_PCI_ROM_REGION_INDEX;
-
-    ret = ioctl(vdev->fd, VFIO_DEVICE_GET_REGION_INFO, &reg_info);
-    if (ret) {
-        error_report("vfio: Error getting ROM info: %m");
-        goto error;
-    }
-
-    DPRINTF("Device %s ROM:\n", name);
-    DPRINTF("  size: 0x%lx, offset: 0x%lx, flags: 0x%lx\n",
-            (unsigned long)reg_info.size, (unsigned long)reg_info.offset,
-            (unsigned long)reg_info.flags);
-
-    vdev->rom_size = reg_info.size;
-    vdev->rom_offset = reg_info.offset;
-
     reg_info.index = VFIO_PCI_CONFIG_REGION_INDEX;
 
     ret = ioctl(vdev->fd, VFIO_DEVICE_GET_REGION_INFO, &reg_info);
@@ -3229,7 +3288,7 @@ static int vfio_initfn(PCIDevice *pdev)
     memset(&vdev->pdev.config[PCI_BASE_ADDRESS_0], 0, 24);
     memset(&vdev->pdev.config[PCI_ROM_ADDRESS], 0, 4);
 
-    vfio_load_rom(vdev);
+    vfio_pci_size_rom(vdev);
 
     ret = vfio_early_setup_msix(vdev);
     if (ret) {
@@ -3294,6 +3353,7 @@ static void vfio_exitfn(PCIDevice *pdev)
     vfio_teardown_msi(vdev);
     vfio_unmap_bars(vdev);
     g_free(vdev->emulated_config_bits);
+    g_free(vdev->rom);
     vfio_put_device(vdev);
     vfio_put_group(group);
 } 

lspci -t

-[0000:00]-+-00.0
           +-01.0-[01]--
           +-01.1-[02]----00.0
           +-02.0
           +-14.0
           +-16.0
           +-19.0
           +-1a.0
           +-1b.0
           +-1c.0-[03]--
           +-1c.1-[04]----00.0
           +-1c.5-[05-06]----00.0-[06]----01.0
           +-1c.6-[07]----00.0
           +-1c.7-[08]----00.0
           +-1d.0
           +-1f.0
           +-1f.2
           \-1f.3

lspci -nnk

00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor DRAM Controller [8086:0150] (rev 09)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:5000]
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0151] (rev 09)
        Kernel driver in use: pcieport
        Kernel modules: shpchp
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor PCI Express Root Port [8086:0155] (rev 09)
        Kernel driver in use: pcieport
        Kernel modules: shpchp
00:02.0 VGA compatible controller [0300]: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller [8086:0152] (rev 09)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:d000]
        Kernel driver in use: i915
        Kernel modules: i915
00:14.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller [8086:1e31] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:5007]
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_hcd
00:16.0 Communication controller [0780]: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 [8086:1e3a] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:1c3a]
        Kernel driver in use: mei_me
        Kernel modules: mei_me
00:19.0 Ethernet controller [0200]: Intel Corporation 82579V Gigabit Network Connection [8086:1503] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:e000]
        Kernel driver in use: e1000e
        Kernel modules: e1000e
00:1a.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 [8086:1e2d] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:5006]
        Kernel driver in use: ehci-pci
        Kernel modules: ehci_pci
00:1b.0 Audio device [0403]: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller [8086:1e20] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:a072]
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
00:1c.0 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 [8086:1e10] (rev c4)
        Kernel driver in use: pcieport
        Kernel modules: shpchp
00:1c.1 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 [8086:1e12] (rev c4)
        Kernel driver in use: pcieport
        Kernel modules: shpchp
00:1c.5 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev c4)
00:1c.6 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 7 [8086:1e1c] (rev c4)
        Kernel driver in use: pcieport
        Kernel modules: shpchp
00:1c.7 PCI bridge [0604]: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 [8086:1e1e] (rev c4)
        Kernel driver in use: pcieport
        Kernel modules: shpchp
00:1d.0 USB controller [0c03]: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 [8086:1e26] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:5006]
        Kernel driver in use: ehci-pci
        Kernel modules: ehci_pci
00:1f.0 ISA bridge [0601]: Intel Corporation Z77 Express Chipset LPC Controller [8086:1e44] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
        Kernel driver in use: lpc_ich
        Kernel modules: lpc_ich
00:1f.2 SATA controller [0106]: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] [8086:1e02] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:b005]
        Kernel driver in use: ahci
00:1f.3 SMBus [0c05]: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller [8086:1e22] (rev 04)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:5001]
        Kernel modules: i2c_i801
02:00.0 VGA compatible controller [0300]: NVIDIA Corporation G73 [GeForce 7600 GT] [10de:0391] (rev a1)
        Subsystem: Micro-Star International Co., Ltd. Device [1462:0452]
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau
04:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller [1b4b:9172] (rev 11)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:b000]
        Kernel driver in use: ahci
05:00.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev 30)
06:01.0 FireWire (IEEE 1394) [0c00]: VIA Technologies, Inc. VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller [1106:3044] (rev c0)
        Subsystem: Gigabyte Technology Co., Ltd GA-7VT600-1394 Motherboard [1458:1000]
        Kernel driver in use: firewire_ohci
        Kernel modules: firewire_ohci
07:00.0 Ethernet controller [0200]: Qualcomm Atheros AR8151 v2.0 Gigabit Ethernet [1969:1083] (rev c0)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:e000]
        Kernel driver in use: atl1c
        Kernel modules: atl1c
08:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9172 SATA 6Gb/s Controller [1b4b:9172] (rev 11)
        Subsystem: Gigabyte Technology Co., Ltd Device [1458:b000]
        Kernel driver in use: ahci

Adding parameter to kernel

intel_iommu=on,igfx_off

Without igfx_off the system frezze.

dmesg after boot

[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.14.2 (root@kbrown) (gcc version 4.8.2 (GCC) ) #1 SMP Fri May 2 00:04:02 CDT 2014
[    0.000000] Command line: auto BOOT_IMAGE=Generic3.14.2 ro root=803 intel_iommu=on,igfx_off
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009d7ff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009d800-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001fffffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000020000000-0x00000000201fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000020200000-0x0000000040003fff] usable
[    0.000000] BIOS-e820: [mem 0x0000000040004000-0x0000000040004fff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000040005000-0x000000008d455fff] usable
[    0.000000] BIOS-e820: [mem 0x000000008d456000-0x000000008d905fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000008d906000-0x000000008d906fff] ACPI data
[    0.000000] BIOS-e820: [mem 0x000000008d907000-0x000000008da2ffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000008da30000-0x000000008e265fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000008e266000-0x000000008e266fff] usable
[    0.000000] BIOS-e820: [mem 0x000000008e267000-0x000000008e2a9fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000008e2aa000-0x000000008ec6bfff] usable
[    0.000000] BIOS-e820: [mem 0x000000008ec6c000-0x000000008efd6fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000008efd7000-0x000000008effffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000cf800000-0x00000000cf9fffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000042f5fffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.7 present.
[    0.000000] DMI: Gigabyte Technology Co., Ltd. To be filled by O.E.M./Z77X-UD5H, BIOS F14 08/22/2012
[    0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[    0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
[    0.000000] No AGP bridge found
[    0.000000] e820: last_pfn = 0x42f600 max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: uncachable
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-CFFFF write-protect
[    0.000000]   D0000-E7FFF uncachable
[    0.000000]   E8000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 000000000 mask C00000000 write-back
[    0.000000]   1 base 400000000 mask FE0000000 write-back
[    0.000000]   2 base 420000000 mask FF0000000 write-back
[    0.000000]   3 base 0C0000000 mask FC0000000 uncachable
[    0.000000]   4 base 0A0000000 mask FE0000000 uncachable
[    0.000000]   5 base 090000000 mask FF0000000 uncachable
[    0.000000]   6 base 08F800000 mask FFF800000 uncachable
[    0.000000]   7 base 42F800000 mask FFF800000 uncachable
[    0.000000]   8 base 42F600000 mask FFFE00000 uncachable
[    0.000000]   9 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] e820: update [mem 0x8f800000-0xffffffff] usable ==> reserved
[    0.000000] e820: last_pfn = 0x8f000 max_arch_pfn = 0x400000000
[    0.000000] found SMP MP-table at [mem 0x000fd830-0x000fd83f] mapped at [ffff8800000fd830]
[    0.000000] Base memory trampoline at [ffff880000097000] 97000 size 24576
[    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[    0.000000]  [mem 0x00000000-0x000fffff] page 4k
[    0.000000] BRK [0x01cac000, 0x01cacfff] PGTABLE
[    0.000000] BRK [0x01cad000, 0x01cadfff] PGTABLE
[    0.000000] BRK [0x01cae000, 0x01caefff] PGTABLE
[    0.000000] init_memory_mapping: [mem 0x42f400000-0x42f5fffff]
[    0.000000]  [mem 0x42f400000-0x42f5fffff] page 2M
[    0.000000] BRK [0x01caf000, 0x01caffff] PGTABLE
[    0.000000] init_memory_mapping: [mem 0x42c000000-0x42f3fffff]
[    0.000000]  [mem 0x42c000000-0x42f3fffff] page 2M
[    0.000000] init_memory_mapping: [mem 0x400000000-0x42bffffff]
[    0.000000]  [mem 0x400000000-0x42bffffff] page 2M
[    0.000000] init_memory_mapping: [mem 0x00100000-0x1fffffff]
[    0.000000]  [mem 0x00100000-0x001fffff] page 4k
[    0.000000]  [mem 0x00200000-0x1fffffff] page 2M
[    0.000000] init_memory_mapping: [mem 0x20200000-0x40003fff]
[    0.000000]  [mem 0x20200000-0x3fffffff] page 2M
[    0.000000]  [mem 0x40000000-0x40003fff] page 4k
[    0.000000] BRK [0x01cb0000, 0x01cb0fff] PGTABLE
[    0.000000] BRK [0x01cb1000, 0x01cb1fff] PGTABLE
[    0.000000] init_memory_mapping: [mem 0x40005000-0x8d455fff]
[    0.000000]  [mem 0x40005000-0x401fffff] page 4k
[    0.000000]  [mem 0x40200000-0x8d3fffff] page 2M
[    0.000000]  [mem 0x8d400000-0x8d455fff] page 4k
[    0.000000] init_memory_mapping: [mem 0x8e266000-0x8e266fff]
[    0.000000]  [mem 0x8e266000-0x8e266fff] page 4k
[    0.000000] init_memory_mapping: [mem 0x8e2aa000-0x8ec6bfff]
[    0.000000]  [mem 0x8e2aa000-0x8e3fffff] page 4k
[    0.000000]  [mem 0x8e400000-0x8ebfffff] page 2M
[    0.000000]  [mem 0x8ec00000-0x8ec6bfff] page 4k
[    0.000000] init_memory_mapping: [mem 0x8efd7000-0x8effffff]
[    0.000000]  [mem 0x8efd7000-0x8effffff] page 4k
[    0.000000] init_memory_mapping: [mem 0x100000000-0x3ffffffff]
[    0.000000]  [mem 0x100000000-0x3ffffffff] page 2M
[    0.000000] RAMDISK: [mem 0x7fbb0000-0x7fffffff]
[    0.000000] ACPI: RSDP 00000000000f0490 000024 (v02 ALASKA)
[    0.000000] ACPI: XSDT 000000008da0e078 00006C (v01 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: FACP 000000008da195f8 0000F4 (v04 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: DSDT 000000008da0e178 00B47F (v02 ALASKA    A M I 00000012 INTL 20051117)
[    0.000000] ACPI: FACS 000000008da2ef80 000040
[    0.000000] ACPI: APIC 000000008da196f0 000072 (v03 ALASKA    A M I 01072009 AMI  00010013)
[    0.000000] ACPI: MCFG 000000008da19768 00003C (v01                 01072009 MSFT 00000097)
[    0.000000] ACPI: HPET 000000008da197a8 000038 (v01 ALASKA    A M I 01072009 AMI. 00000005)
[    0.000000] ACPI: SSDT 000000008da197e0 00036D (v01 SataRe SataTabl 00001000 INTL 20091112)
[    0.000000] ACPI: SSDT 000000008da19b50 0009AA (v01  PmRef  Cpu0Ist 00003000 INTL 20051117)
[    0.000000] ACPI: SSDT 000000008da1a500 000A92 (v01  PmRef    CpuPm 00003000 INTL 20051117)
[    0.000000] ACPI: DMAR 000000008da1af98 0000B8 (v01 INTEL      SNB  00000001 INTL 00000001)
[    0.000000] ACPI: MATS 000000008da1b050 000034 (v02 ALASKA    A M I 00000002 w?x2 00000000)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000]  [ffffea0000000000-ffffea0010bfffff] PMD -> [ffff88041fc00000-ffff88042ebfffff] on node 0
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
[    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
[    0.000000]   Normal   [mem 0x100000000-0x42f5fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00001000-0x0009cfff]
[    0.000000]   node   0: [mem 0x00100000-0x1fffffff]
[    0.000000]   node   0: [mem 0x20200000-0x40003fff]
[    0.000000]   node   0: [mem 0x40005000-0x8d455fff]
[    0.000000]   node   0: [mem 0x8e266000-0x8e266fff]
[    0.000000]   node   0: [mem 0x8e2aa000-0x8ec6bfff]
[    0.000000]   node   0: [mem 0x8efd7000-0x8effffff]
[    0.000000]   node   0: [mem 0x100000000-0x42f5fffff]
[    0.000000] On node 0 totalpages: 3920349
[    0.000000]   DMA zone: 64 pages used for memmap
[    0.000000]   DMA zone: 21 pages reserved
[    0.000000]   DMA zone: 3996 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 9010 pages used for memmap
[    0.000000]   DMA32 zone: 576577 pages, LIFO batch:31
[    0.000000]   Normal zone: 52184 pages used for memmap
[    0.000000]   Normal zone: 3339776 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
[    0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[    0.000000] nr_irqs_gsi: 40
[    0.000000] PM: Registered nosave memory: [mem 0x0009d000-0x0009dfff]
[    0.000000] PM: Registered nosave memory: [mem 0x0009e000-0x0009ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000dffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000e0000-0x000fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x20000000-0x201fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x40004000-0x40004fff]
[    0.000000] PM: Registered nosave memory: [mem 0x8d456000-0x8d905fff]
[    0.000000] PM: Registered nosave memory: [mem 0x8d906000-0x8d906fff]
[    0.000000] PM: Registered nosave memory: [mem 0x8d907000-0x8da2ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x8da30000-0x8e265fff]
[    0.000000] PM: Registered nosave memory: [mem 0x8e267000-0x8e2a9fff]
[    0.000000] PM: Registered nosave memory: [mem 0x8ec6c000-0x8efd6fff]
[    0.000000] PM: Registered nosave memory: [mem 0x8f000000-0x8f9fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x8fa00000-0xb19fffff]
[    0.000000] PM: Registered nosave memory: [mem 0xb1a00000-0xcf7fffff]
[    0.000000] PM: Registered nosave memory: [mem 0xcf800000-0xcf9fffff]
[    0.000000] PM: Registered nosave memory: [mem 0xcfa00000-0xf7ffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xf8000000-0xfbffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfc000000-0xfebfffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfec00000-0xfec00fff]
[    0.000000] PM: Registered nosave memory: [mem 0xfec01000-0xfecfffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed00000-0xfed03fff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed04000-0xfed1bfff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed1ffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed20000-0xfedfffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfee00000-0xfee00fff]
[    0.000000] PM: Registered nosave memory: [mem 0xfee01000-0xfeffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xff000000-0xffffffff]
[    0.000000] e820: [mem 0xcfa00000-0xf7ffffff] available for PCI devices
[    0.000000] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:4 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff88042f200000 s78848 r8192 d23552 u524288
[    0.000000] pcpu-alloc: s78848 r8192 d23552 u524288 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 1 2 3 
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 3859070
[    0.000000] Kernel command line: auto BOOT_IMAGE=Generic3.14.2 ro root=803 intel_iommu=on,igfx_off
[    0.000000] Intel-IOMMU: enabled
[    0.000000] Intel-IOMMU: disable GFX device mapping
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes)
[    0.000000] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 15326756K/15681396K available (5963K kernel code, 687K rwdata, 2080K rodata, 1088K init, 924K bss, 354640K reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	CONFIG_RCU_FANOUT set to non-default value of 32
[    0.000000] 	RCU restricting CPUs from NR_CPUS=128 to nr_cpu_ids=4.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS:8448 nr_irqs:712 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [tty0] enabled
[    0.000000] hpet clockevent registered
[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] tsc: Detected 3002.957 MHz processor
[    0.000002] Calibrating delay loop (skipped), value calculated using timer frequency.. 6005.91 BogoMIPS (lpj=3002957)
[    0.000005] pid_max: default: 32768 minimum: 301
[    0.000010] ACPI: Core revision 20131218
[    0.004301] ACPI: All ACPI Tables successfully acquired
[    0.006502] Security Framework initialized
[    0.006512] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.006514] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes)
[    0.006689] Initializing cgroup subsys devices
[    0.006691] Initializing cgroup subsys freezer
[    0.006693] Initializing cgroup subsys net_cls
[    0.006695] Initializing cgroup subsys blkio
[    0.006697] Initializing cgroup subsys perf_event
[    0.006716] CPU: Physical Processor ID: 0
[    0.006717] CPU: Processor Core ID: 0
[    0.006722] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
[    0.006722] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
[    0.007019] mce: CPU supports 9 MCE banks
[    0.007030] CPU0: Thermal monitoring enabled (TM1)
[    0.007037] Last level iTLB entries: 4KB 512, 2MB 8, 4MB 8
[    0.007037] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32, 1GB 0
[    0.007037] tlb_flushall_shift: 2
[    0.007115] Freeing SMP alternatives memory: 24K (ffffffff81bbd000 - ffffffff81bc3000)
[    0.007118] ftrace: allocating 23798 entries in 93 pages
[    0.015849] dmar: Host address width 36
[    0.015852] dmar: DRHD base: 0x000000fed90000 flags: 0x0
[    0.015858] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap c0000020e60262 ecap f0101a
[    0.015860] dmar: DRHD base: 0x000000fed91000 flags: 0x1
[    0.015864] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap c9008020660262 ecap f0105a
[    0.015865] dmar: RMRR base: 0x0000008d86f000 end: 0x0000008d899fff
[    0.015867] dmar: RMRR base: 0x000000cf800000 end: 0x000000cf9fffff
[    0.015937] IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.015938] HPET id 0 under DRHD base 0xfed91000
[    0.015940] Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.016058] Enabled IRQ remapping in x2apic mode
[    0.016059] Enabling x2apic
[    0.016060] Enabled x2apic
[    0.016064] Switched APIC routing to cluster x2apic.
[    0.016474] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.026477] smpboot: CPU0: Intel(R) Core(TM) i5-3330 CPU @ 3.00GHz (fam: 06, model: 3a, stepping: 09)
[    0.026483] TSC deadline timer enabled
[    0.026489] Performance Events: PEBS fmt1+, 16-deep LBR, IvyBridge events, full-width counters, Intel PMU driver.
[    0.026496] ... version:                3
[    0.026497] ... bit width:              48
[    0.026498] ... generic registers:      8
[    0.026499] ... value mask:             0000ffffffffffff
[    0.026500] ... max period:             0000ffffffffffff
[    0.026501] ... fixed-purpose events:   3
[    0.026503] ... event mask:             00000007000000ff
[    0.026647] x86: Booting SMP configuration:
[    0.026649] .... node  #0, CPUs:      #1 #2 #3
[    0.067044] x86: Booted up 1 node, 4 CPUs
[    0.067049] smpboot: Total of 4 processors activated (24023.65 BogoMIPS)
[    0.069376] devtmpfs: initialized
[    0.071890] PM: Registering ACPI NVS region [mem 0x8d907000-0x8da2ffff] (1216512 bytes)
[    0.071903] PM: Registering ACPI NVS region [mem 0x8e267000-0x8e2a9fff] (274432 bytes)
[    0.071993] xor: automatically using best checksumming function:
[    0.081222]    avx       : 21368.000 MB/sec
[    0.081226] atomic64 test passed for x86-64 platform with CX8 and with SSE
[    0.081228] pinctrl core: initialized pinctrl subsystem
[    0.081266] regulator-dummy: no parameters
[    0.081296] NET: Registered protocol family 16
[    0.081343] cpuidle: using governor ladder
[    0.081344] cpuidle: using governor menu
[    0.081359] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[    0.081361] ACPI: bus type PCI registered
[    0.081363] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.081405] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[    0.081408] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved in E820
[    0.087132] PCI: Using configuration type 1 for base access
[    0.087919] bio: create slab <bio-0> at 0
[    0.104238] raid6: sse2x1    8238 MB/s
[    0.121252] raid6: sse2x2   10230 MB/s
[    0.138260] raid6: sse2x4   11851 MB/s
[    0.138262] raid6: using algorithm sse2x4 (11851 MB/s)
[    0.138263] raid6: using ssse3x2 recovery algorithm
[    0.138287] ACPI: Added _OSI(Module Device)
[    0.138289] ACPI: Added _OSI(Processor Device)
[    0.138290] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.138291] ACPI: Added _OSI(Processor Aggregator Device)
[    0.140554] ACPI: Executed 1 blocks of module-level executable AML code
[    0.147583] ACPI: SSDT 000000008d8a8018 00083B (v01  PmRef  Cpu0Cst 00003001 INTL 20051117)
[    0.147861] ACPI: Dynamic OEM Table Load:
[    0.147863] ACPI: SSDT           (null) 00083B (v01  PmRef  Cpu0Cst 00003001 INTL 20051117)
[    0.152472] ACPI: SSDT 000000008d8a9a98 000303 (v01  PmRef    ApIst 00003000 INTL 20051117)
[    0.152766] ACPI: Dynamic OEM Table Load:
[    0.152768] ACPI: SSDT           (null) 000303 (v01  PmRef    ApIst 00003000 INTL 20051117)
[    0.157394] ACPI: SSDT 000000008d8b5c18 000119 (v01  PmRef    ApCst 00003000 INTL 20051117)
[    0.157665] ACPI: Dynamic OEM Table Load:
[    0.157667] ACPI: SSDT           (null) 000119 (v01  PmRef    ApCst 00003000 INTL 20051117)
[    0.162731] ACPI: Interpreter enabled
[    0.162737] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20131218/hwxface-580)
[    0.162741] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20131218/hwxface-580)
[    0.162752] ACPI: (supports S0 S3 S4 S5)
[    0.162753] ACPI: Using IOAPIC for interrupt routing
[    0.162771] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.162908] ACPI: No dock devices found.
[    0.168303] ACPI: Power Resource [FN00] (off)
[    0.168363] ACPI: Power Resource [FN01] (off)
[    0.168417] ACPI: Power Resource [FN02] (off)
[    0.168472] ACPI: Power Resource [FN03] (off)
[    0.168526] ACPI: Power Resource [FN04] (off)
[    0.168897] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-3e])
[    0.168901] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    0.169051] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME]
[    0.169140] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability]
[    0.169578] PCI host bridge to bus 0000:00
[    0.169581] pci_bus 0000:00: root bus resource [bus 00-3e]
[    0.169583] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
[    0.169585] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
[    0.169587] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
[    0.169589] pci_bus 0000:00: root bus resource [mem 0x000d0000-0x000d3fff]
[    0.169590] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff]
[    0.169592] pci_bus 0000:00: root bus resource [mem 0x000d8000-0x000dbfff]
[    0.169594] pci_bus 0000:00: root bus resource [mem 0x000dc000-0x000dffff]
[    0.169596] pci_bus 0000:00: root bus resource [mem 0x000e0000-0x000e3fff]
[    0.169598] pci_bus 0000:00: root bus resource [mem 0x000e4000-0x000e7fff]
[    0.169599] pci_bus 0000:00: root bus resource [mem 0xcfa00000-0xfeafffff]
[    0.169606] pci 0000:00:00.0: [8086:0150] type 00 class 0x060000
[    0.169669] pci 0000:00:01.0: [8086:0151] type 01 class 0x060400
[    0.169695] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[    0.169724] pci 0000:00:01.0: System wakeup disabled by ACPI
[    0.169753] pci 0000:00:02.0: [8086:0152] type 00 class 0x030000
[    0.169762] pci 0000:00:02.0: reg 0x10: [mem 0xf7400000-0xf77fffff 64bit]
[    0.169767] pci 0000:00:02.0: reg 0x18: [mem 0xd0000000-0xdfffffff 64bit pref]
[    0.169771] pci 0000:00:02.0: reg 0x20: [io  0xf000-0xf03f]
[    0.169849] pci 0000:00:14.0: [8086:1e31] type 00 class 0x0c0330
[    0.169870] pci 0000:00:14.0: reg 0x10: [mem 0xf7d20000-0xf7d2ffff 64bit]
[    0.169937] pci 0000:00:14.0: PME# supported from D3hot D3cold
[    0.169970] pci 0000:00:14.0: System wakeup disabled by ACPI
[    0.169999] pci 0000:00:16.0: [8086:1e3a] type 00 class 0x078000
[    0.170020] pci 0000:00:16.0: reg 0x10: [mem 0xf7d3b000-0xf7d3b00f 64bit]
[    0.170090] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
[    0.170155] pci 0000:00:19.0: [8086:1503] type 00 class 0x020000
[    0.170171] pci 0000:00:19.0: reg 0x10: [mem 0xf7d00000-0xf7d1ffff]
[    0.170179] pci 0000:00:19.0: reg 0x14: [mem 0xf7d39000-0xf7d39fff]
[    0.170186] pci 0000:00:19.0: reg 0x18: [io  0xf080-0xf09f]
[    0.170245] pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
[    0.170278] pci 0000:00:19.0: System wakeup disabled by ACPI
[    0.170308] pci 0000:00:1a.0: [8086:1e2d] type 00 class 0x0c0320
[    0.170329] pci 0000:00:1a.0: reg 0x10: [mem 0xf7d38000-0xf7d383ff]
[    0.170415] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
[    0.170460] pci 0000:00:1a.0: System wakeup disabled by ACPI
[    0.170489] pci 0000:00:1b.0: [8086:1e20] type 00 class 0x040300
[    0.170502] pci 0000:00:1b.0: reg 0x10: [mem 0xf7d30000-0xf7d33fff 64bit]
[    0.170572] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
[    0.170606] pci 0000:00:1b.0: System wakeup disabled by ACPI
[    0.170632] pci 0000:00:1c.0: [8086:1e10] type 01 class 0x060400
[    0.170708] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
[    0.170745] pci 0000:00:1c.0: System wakeup disabled by ACPI
[    0.170770] pci 0000:00:1c.1: [8086:1e12] type 01 class 0x060400
[    0.170846] pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
[    0.170883] pci 0000:00:1c.1: System wakeup disabled by ACPI
[    0.170910] pci 0000:00:1c.3: [8086:1e16] type 01 class 0x060400
[    0.170985] pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
[    0.171023] pci 0000:00:1c.3: System wakeup disabled by ACPI
[    0.171049] pci 0000:00:1c.5: [8086:244e] type 01 class 0x060401
[    0.171125] pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold
[    0.171161] pci 0000:00:1c.5: System wakeup disabled by ACPI
[    0.171187] pci 0000:00:1c.6: [8086:1e1c] type 01 class 0x060400
[    0.171262] pci 0000:00:1c.6: PME# supported from D0 D3hot D3cold
[    0.171299] pci 0000:00:1c.6: System wakeup disabled by ACPI
[    0.171327] pci 0000:00:1c.7: [8086:1e1e] type 01 class 0x060400
[    0.171402] pci 0000:00:1c.7: PME# supported from D0 D3hot D3cold
[    0.171439] pci 0000:00:1c.7: System wakeup disabled by ACPI
[    0.171469] pci 0000:00:1d.0: [8086:1e26] type 00 class 0x0c0320
[    0.171488] pci 0000:00:1d.0: reg 0x10: [mem 0xf7d37000-0xf7d373ff]
[    0.171573] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
[    0.171616] pci 0000:00:1d.0: System wakeup disabled by ACPI
[    0.171645] pci 0000:00:1f.0: [8086:1e44] type 00 class 0x060100
[    0.171796] pci 0000:00:1f.2: [8086:1e02] type 00 class 0x010601
[    0.171812] pci 0000:00:1f.2: reg 0x10: [io  0xf0d0-0xf0d7]
[    0.171819] pci 0000:00:1f.2: reg 0x14: [io  0xf0c0-0xf0c3]
[    0.171826] pci 0000:00:1f.2: reg 0x18: [io  0xf0b0-0xf0b7]
[    0.171833] pci 0000:00:1f.2: reg 0x1c: [io  0xf0a0-0xf0a3]
[    0.171839] pci 0000:00:1f.2: reg 0x20: [io  0xf060-0xf07f]
[    0.171846] pci 0000:00:1f.2: reg 0x24: [mem 0xf7d36000-0xf7d367ff]
[    0.171887] pci 0000:00:1f.2: PME# supported from D3hot
[    0.171939] pci 0000:00:1f.3: [8086:1e22] type 00 class 0x0c0500
[    0.171953] pci 0000:00:1f.3: reg 0x10: [mem 0xf7d35000-0xf7d350ff 64bit]
[    0.171971] pci 0000:00:1f.3: reg 0x20: [io  0xf040-0xf05f]
[    0.172065] pci 0000:01:00.0: [10de:0391] type 00 class 0x030000
[    0.172071] pci 0000:01:00.0: reg 0x10: [mem 0xf6000000-0xf6ffffff]
[    0.172077] pci 0000:01:00.0: reg 0x14: [mem 0xe0000000-0xefffffff 64bit pref]
[    0.172084] pci 0000:01:00.0: reg 0x1c: [mem 0xf5000000-0xf5ffffff 64bit]
[    0.172088] pci 0000:01:00.0: reg 0x24: [io  0xe000-0xe07f]
[    0.172092] pci 0000:01:00.0: reg 0x30: [mem 0xf7000000-0xf701ffff pref]
[    0.173332] pci 0000:00:01.0: PCI bridge to [bus 01]
[    0.173338] pci 0000:00:01.0:   bridge window [io  0xe000-0xefff]
[    0.173341] pci 0000:00:01.0:   bridge window [mem 0xf5000000-0xf70fffff]
[    0.173346] pci 0000:00:01.0:   bridge window [mem 0xe0000000-0xefffffff 64bit pref]
[    0.173430] pci 0000:00:1c.0: PCI bridge to [bus 02]
[    0.173509] pci 0000:03:00.0: [1b4b:9172] type 00 class 0x010601
[    0.173526] pci 0000:03:00.0: reg 0x10: [io  0xd040-0xd047]
[    0.173538] pci 0000:03:00.0: reg 0x14: [io  0xd030-0xd033]
[    0.173549] pci 0000:03:00.0: reg 0x18: [io  0xd020-0xd027]
[    0.173561] pci 0000:03:00.0: reg 0x1c: [io  0xd010-0xd013]
[    0.173573] pci 0000:03:00.0: reg 0x20: [io  0xd000-0xd00f]
[    0.173585] pci 0000:03:00.0: reg 0x24: [mem 0xf7c10000-0xf7c101ff]
[    0.173596] pci 0000:03:00.0: reg 0x30: [mem 0xf7c00000-0xf7c0ffff pref]
[    0.173657] pci 0000:03:00.0: PME# supported from D3hot
[    0.173680] pci 0000:03:00.0: System wakeup disabled by ACPI
[    0.175340] pci 0000:00:1c.1: PCI bridge to [bus 03]
[    0.175346] pci 0000:00:1c.1:   bridge window [io  0xd000-0xdfff]
[    0.175350] pci 0000:00:1c.1:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.175457] pci 0000:04:00.0: [168c:0034] type 00 class 0x028000
[    0.175482] pci 0000:04:00.0: reg 0x10: [mem 0xf7b00000-0xf7b7ffff 64bit]
[    0.175544] pci 0000:04:00.0: reg 0x30: [mem 0xf7b80000-0xf7b8ffff pref]
[    0.175625] pci 0000:04:00.0: supports D1 D2
[    0.175627] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.175655] pci 0000:04:00.0: System wakeup disabled by ACPI
[    0.177344] pci 0000:00:1c.3: PCI bridge to [bus 04]
[    0.177353] pci 0000:00:1c.3:   bridge window [mem 0xf7b00000-0xf7bfffff]
[    0.177461] pci 0000:05:00.0: [8086:244e] type 01 class 0x060401
[    0.177616] pci 0000:05:00.0: supports D1 D2
[    0.177617] pci 0000:05:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.177642] pci 0000:05:00.0: System wakeup disabled by ACPI
[    0.177662] pci 0000:00:1c.5: PCI bridge to [bus 05-06] (subtractive decode)
[    0.177666] pci 0000:00:1c.5:   bridge window [io  0xc000-0xcfff]
[    0.177669] pci 0000:00:1c.5:   bridge window [mem 0xf7a00000-0xf7afffff]
[    0.177674] pci 0000:00:1c.5:   bridge window [io  0x0000-0x0cf7] (subtractive decode)
[    0.177675] pci 0000:00:1c.5:   bridge window [io  0x0d00-0xffff] (subtractive decode)
[    0.177677] pci 0000:00:1c.5:   bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
[    0.177678] pci 0000:00:1c.5:   bridge window [mem 0x000d0000-0x000d3fff] (subtractive decode)
[    0.177679] pci 0000:00:1c.5:   bridge window [mem 0x000d4000-0x000d7fff] (subtractive decode)
[    0.177680] pci 0000:00:1c.5:   bridge window [mem 0x000d8000-0x000dbfff] (subtractive decode)
[    0.177681] pci 0000:00:1c.5:   bridge window [mem 0x000dc000-0x000dffff] (subtractive decode)
[    0.177682] pci 0000:00:1c.5:   bridge window [mem 0x000e0000-0x000e3fff] (subtractive decode)
[    0.177684] pci 0000:00:1c.5:   bridge window [mem 0x000e4000-0x000e7fff] (subtractive decode)
[    0.177685] pci 0000:00:1c.5:   bridge window [mem 0xcfa00000-0xfeafffff] (subtractive decode)
[    0.177767] pci 0000:06:01.0: [1106:3044] type 00 class 0x0c0010
[    0.177803] pci 0000:06:01.0: reg 0x10: [mem 0xf7a00000-0xf7a007ff]
[    0.177823] pci 0000:06:01.0: reg 0x14: [io  0xc000-0xc07f]
[    0.177975] pci 0000:06:01.0: supports D2
[    0.177976] pci 0000:06:01.0: PME# supported from D2 D3hot D3cold
[    0.178088] pci 0000:05:00.0: PCI bridge to [bus 06] (subtractive decode)
[    0.178099] pci 0000:05:00.0:   bridge window [io  0xc000-0xcfff]
[    0.178105] pci 0000:05:00.0:   bridge window [mem 0xf7a00000-0xf7afffff]
[    0.178116] pci 0000:05:00.0:   bridge window [io  0xc000-0xcfff] (subtractive decode)
[    0.178117] pci 0000:05:00.0:   bridge window [mem 0xf7a00000-0xf7afffff] (subtractive decode)
[    0.178118] pci 0000:05:00.0:   bridge window [??? 0x00000000 flags 0x0] (subtractive decode)
[    0.178120] pci 0000:05:00.0:   bridge window [??? 0x00000000 flags 0x0] (subtractive decode)
[    0.178121] pci 0000:05:00.0:   bridge window [io  0x0000-0x0cf7] (subtractive decode)
[    0.178122] pci 0000:05:00.0:   bridge window [io  0x0d00-0xffff] (subtractive decode)
[    0.178123] pci 0000:05:00.0:   bridge window [mem 0x000a0000-0x000bffff] (subtractive decode)
[    0.178124] pci 0000:05:00.0:   bridge window [mem 0x000d0000-0x000d3fff] (subtractive decode)
[    0.178125] pci 0000:05:00.0:   bridge window [mem 0x000d4000-0x000d7fff] (subtractive decode)
[    0.178127] pci 0000:05:00.0:   bridge window [mem 0x000d8000-0x000dbfff] (subtractive decode)
[    0.178128] pci 0000:05:00.0:   bridge window [mem 0x000dc000-0x000dffff] (subtractive decode)
[    0.178129] pci 0000:05:00.0:   bridge window [mem 0x000e0000-0x000e3fff] (subtractive decode)
[    0.178130] pci 0000:05:00.0:   bridge window [mem 0x000e4000-0x000e7fff] (subtractive decode)
[    0.178131] pci 0000:05:00.0:   bridge window [mem 0xcfa00000-0xfeafffff] (subtractive decode)
[    0.178218] pci 0000:07:00.0: [1969:1083] type 00 class 0x020000
[    0.178244] pci 0000:07:00.0: reg 0x10: [mem 0xf7900000-0xf793ffff 64bit]
[    0.178258] pci 0000:07:00.0: reg 0x18: [io  0xb000-0xb07f]
[    0.178392] pci 0000:07:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    0.178421] pci 0000:07:00.0: System wakeup disabled by ACPI
[    0.180351] pci 0000:00:1c.6: PCI bridge to [bus 07]
[    0.180358] pci 0000:00:1c.6:   bridge window [io  0xb000-0xbfff]
[    0.180362] pci 0000:00:1c.6:   bridge window [mem 0xf7900000-0xf79fffff]
[    0.180464] pci 0000:08:00.0: [1b4b:9172] type 00 class 0x010601
[    0.180480] pci 0000:08:00.0: reg 0x10: [io  0xa040-0xa047]
[    0.180492] pci 0000:08:00.0: reg 0x14: [io  0xa030-0xa033]
[    0.180504] pci 0000:08:00.0: reg 0x18: [io  0xa020-0xa027]
[    0.180515] pci 0000:08:00.0: reg 0x1c: [io  0xa010-0xa013]
[    0.180527] pci 0000:08:00.0: reg 0x20: [io  0xa000-0xa00f]
[    0.180539] pci 0000:08:00.0: reg 0x24: [mem 0xf7810000-0xf78101ff]
[    0.180551] pci 0000:08:00.0: reg 0x30: [mem 0xf7800000-0xf780ffff pref]
[    0.180611] pci 0000:08:00.0: PME# supported from D3hot
[    0.180634] pci 0000:08:00.0: System wakeup disabled by ACPI
[    0.182352] pci 0000:00:1c.7: PCI bridge to [bus 08]
[    0.182358] pci 0000:00:1c.7:   bridge window [io  0xa000-0xafff]
[    0.182362] pci 0000:00:1c.7:   bridge window [mem 0xf7800000-0xf78fffff]
[    0.182416] pci_bus 0000:00: on NUMA node 0
[    0.182419] acpi PNP0A08:00: Disabling ASPM (FADT indicates it is unsupported)
[    0.182582] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 10 *11 12 14 15)
[    0.182619] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 *4 5 6 10 11 12 14 15)
[    0.182653] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 10 *11 12 14 15)
[    0.182687] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 *10 11 12 14 15)
[    0.182721] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 *5 6 10 11 12 14 15)
[    0.182755] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 10 11 12 14 15) *0, disabled.
[    0.182789] ACPI: PCI Interrupt Link [LNKG] (IRQs *3 4 5 6 10 11 12 14 15)
[    0.182823] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 *10 11 12 14 15)
[    0.182893] ACPI: Enabled 4 GPEs in block 00 to 3F
[    0.182943] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
[    0.182948] vgaarb: device added: PCI:0000:01:00.0,decodes=io+mem,owns=none,locks=none
[    0.182950] vgaarb: loaded
[    0.182951] vgaarb: bridge control possible 0000:01:00.0
[    0.182953] vgaarb: no bridge control possible 0000:00:02.0
[    0.182990] SCSI subsystem initialized
[    0.183014] libata version 3.00 loaded.
[    0.183017] ACPI: bus type USB registered
[    0.183030] usbcore: registered new interface driver usbfs
[    0.183038] usbcore: registered new interface driver hub
[    0.183050] usbcore: registered new device driver usb
[    0.183100] PCI: Using ACPI for IRQ routing
[    0.184475] PCI: pci_cache_line_size set to 64 bytes
[    0.184541] e820: reserve RAM buffer [mem 0x0009d800-0x0009ffff]
[    0.184542] e820: reserve RAM buffer [mem 0x40004000-0x43ffffff]
[    0.184543] e820: reserve RAM buffer [mem 0x8d456000-0x8fffffff]
[    0.184545] e820: reserve RAM buffer [mem 0x8e267000-0x8fffffff]
[    0.184546] e820: reserve RAM buffer [mem 0x8ec6c000-0x8fffffff]
[    0.184547] e820: reserve RAM buffer [mem 0x8f000000-0x8fffffff]
[    0.184548] e820: reserve RAM buffer [mem 0x42f600000-0x42fffffff]
[    0.184624] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    0.184629] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
[    0.188094] Switched to clocksource hpet
[    0.189439] pnp: PnP ACPI init
[    0.189448] ACPI: bus type PNP registered
[    0.189498] system 00:00: [mem 0xfed40000-0xfed44fff] has been reserved
[    0.189501] system 00:00: Plug and Play ACPI device, IDs PNP0c01 (active)
[    0.189509] pnp 00:01: [dma 4]
[    0.189518] pnp 00:01: Plug and Play ACPI device, IDs PNP0200 (active)
[    0.189531] pnp 00:02: Plug and Play ACPI device, IDs INT0800 (active)
[    0.189591] pnp 00:03: Plug and Play ACPI device, IDs PNP0103 (active)
[    0.189623] system 00:04: [io  0x0680-0x069f] has been reserved
[    0.189625] system 00:04: [io  0x0200-0x020f] has been reserved
[    0.189627] system 00:04: [io  0xffff] has been reserved
[    0.189629] system 00:04: [io  0xffff] has been reserved
[    0.189631] system 00:04: [io  0x0400-0x0453] could not be reserved
[    0.189633] system 00:04: [io  0x0458-0x047f] has been reserved
[    0.189635] system 00:04: [io  0x0500-0x057f] has been reserved
[    0.189637] system 00:04: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.189658] pnp 00:05: Plug and Play ACPI device, IDs PNP0b00 (active)
[    0.189689] system 00:06: [io  0x0454-0x0457] has been reserved
[    0.189692] system 00:06: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active)
[    0.189776] system 00:07: [io  0x0a00-0x0a0f] has been reserved
[    0.189778] system 00:07: [io  0x0a30-0x0a3f] has been reserved
[    0.189780] system 00:07: [io  0x0a20-0x0a2f] has been reserved
[    0.189782] system 00:07: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.189832] system 00:08: [io  0x04d0-0x04d1] has been reserved
[    0.189835] system 00:08: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.189852] pnp 00:09: Plug and Play ACPI device, IDs PNP0c04 (active)
[    0.190037] system 00:0a: [mem 0xfed1c000-0xfed1ffff] has been reserved
[    0.190040] system 00:0a: [mem 0xfed10000-0xfed17fff] has been reserved
[    0.190044] system 00:0a: [mem 0xfed18000-0xfed18fff] has been reserved
[    0.190046] system 00:0a: [mem 0xfed19000-0xfed19fff] has been reserved
[    0.190048] system 00:0a: [mem 0xf8000000-0xfbffffff] has been reserved
[    0.190050] system 00:0a: [mem 0xfed20000-0xfed3ffff] has been reserved
[    0.190053] system 00:0a: [mem 0xfed90000-0xfed93fff] could not be reserved
[    0.190055] system 00:0a: [mem 0xfed45000-0xfed8ffff] has been reserved
[    0.190057] system 00:0a: [mem 0xff000000-0xffffffff] has been reserved
[    0.190059] system 00:0a: [mem 0xfee00000-0xfeefffff] could not be reserved
[    0.190061] system 00:0a: [mem 0xcfa00000-0xcfa00fff] has been reserved
[    0.190063] system 00:0a: Plug and Play ACPI device, IDs PNP0c02 (active)
[    0.190180] system 00:0b: [mem 0x20000000-0x201fffff] has been reserved
[    0.190182] system 00:0b: [mem 0x40004000-0x40004fff] has been reserved
[    0.190184] system 00:0b: Plug and Play ACPI device, IDs PNP0c01 (active)
[    0.190198] pnp: PnP ACPI: found 12 devices
[    0.190199] ACPI: bus type PNP unregistered
[    0.196070] pci 0000:00:01.0: PCI bridge to [bus 01]
[    0.196073] pci 0000:00:01.0:   bridge window [io  0xe000-0xefff]
[    0.196076] pci 0000:00:01.0:   bridge window [mem 0xf5000000-0xf70fffff]
[    0.196078] pci 0000:00:01.0:   bridge window [mem 0xe0000000-0xefffffff 64bit pref]
[    0.196082] pci 0000:00:1c.0: PCI bridge to [bus 02]
[    0.196093] pci 0000:00:1c.1: PCI bridge to [bus 03]
[    0.196099] pci 0000:00:1c.1:   bridge window [io  0xd000-0xdfff]
[    0.196104] pci 0000:00:1c.1:   bridge window [mem 0xf7c00000-0xf7cfffff]
[    0.196111] pci 0000:00:1c.3: PCI bridge to [bus 04]
[    0.196116] pci 0000:00:1c.3:   bridge window [mem 0xf7b00000-0xf7bfffff]
[    0.196124] pci 0000:05:00.0: PCI bridge to [bus 06]
[    0.196128] pci 0000:05:00.0:   bridge window [io  0xc000-0xcfff]
[    0.196137] pci 0000:05:00.0:   bridge window [mem 0xf7a00000-0xf7afffff]
[    0.196153] pci 0000:00:1c.5: PCI bridge to [bus 05-06]
[    0.196155] pci 0000:00:1c.5:   bridge window [io  0xc000-0xcfff]
[    0.196160] pci 0000:00:1c.5:   bridge window [mem 0xf7a00000-0xf7afffff]
[    0.196168] pci 0000:00:1c.6: PCI bridge to [bus 07]
[    0.196170] pci 0000:00:1c.6:   bridge window [io  0xb000-0xbfff]
[    0.196175] pci 0000:00:1c.6:   bridge window [mem 0xf7900000-0xf79fffff]
[    0.196182] pci 0000:00:1c.7: PCI bridge to [bus 08]
[    0.196185] pci 0000:00:1c.7:   bridge window [io  0xa000-0xafff]
[    0.196189] pci 0000:00:1c.7:   bridge window [mem 0xf7800000-0xf78fffff]
[    0.196197] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
[    0.196199] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
[    0.196200] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
[    0.196201] pci_bus 0000:00: resource 7 [mem 0x000d0000-0x000d3fff]
[    0.196202] pci_bus 0000:00: resource 8 [mem 0x000d4000-0x000d7fff]
[    0.196203] pci_bus 0000:00: resource 9 [mem 0x000d8000-0x000dbfff]
[    0.196205] pci_bus 0000:00: resource 10 [mem 0x000dc000-0x000dffff]
[    0.196206] pci_bus 0000:00: resource 11 [mem 0x000e0000-0x000e3fff]
[    0.196207] pci_bus 0000:00: resource 12 [mem 0x000e4000-0x000e7fff]
[    0.196208] pci_bus 0000:00: resource 13 [mem 0xcfa00000-0xfeafffff]
[    0.196209] pci_bus 0000:01: resource 0 [io  0xe000-0xefff]
[    0.196211] pci_bus 0000:01: resource 1 [mem 0xf5000000-0xf70fffff]
[    0.196212] pci_bus 0000:01: resource 2 [mem 0xe0000000-0xefffffff 64bit pref]
[    0.196213] pci_bus 0000:03: resource 0 [io  0xd000-0xdfff]
[    0.196214] pci_bus 0000:03: resource 1 [mem 0xf7c00000-0xf7cfffff]
[    0.196216] pci_bus 0000:04: resource 1 [mem 0xf7b00000-0xf7bfffff]
[    0.196217] pci_bus 0000:05: resource 0 [io  0xc000-0xcfff]
[    0.196218] pci_bus 0000:05: resource 1 [mem 0xf7a00000-0xf7afffff]
[    0.196219] pci_bus 0000:05: resource 4 [io  0x0000-0x0cf7]
[    0.196220] pci_bus 0000:05: resource 5 [io  0x0d00-0xffff]
[    0.196221] pci_bus 0000:05: resource 6 [mem 0x000a0000-0x000bffff]
[    0.196222] pci_bus 0000:05: resource 7 [mem 0x000d0000-0x000d3fff]
[    0.196224] pci_bus 0000:05: resource 8 [mem 0x000d4000-0x000d7fff]
[    0.196225] pci_bus 0000:05: resource 9 [mem 0x000d8000-0x000dbfff]
[    0.196226] pci_bus 0000:05: resource 10 [mem 0x000dc000-0x000dffff]
[    0.196227] pci_bus 0000:05: resource 11 [mem 0x000e0000-0x000e3fff]
[    0.196228] pci_bus 0000:05: resource 12 [mem 0x000e4000-0x000e7fff]
[    0.196229] pci_bus 0000:05: resource 13 [mem 0xcfa00000-0xfeafffff]
[    0.196231] pci_bus 0000:06: resource 0 [io  0xc000-0xcfff]
[    0.196232] pci_bus 0000:06: resource 1 [mem 0xf7a00000-0xf7afffff]
[    0.196233] pci_bus 0000:06: resource 4 [io  0xc000-0xcfff]
[    0.196234] pci_bus 0000:06: resource 5 [mem 0xf7a00000-0xf7afffff]
[    0.196235] pci_bus 0000:06: resource 8 [io  0x0000-0x0cf7]
[    0.196236] pci_bus 0000:06: resource 9 [io  0x0d00-0xffff]
[    0.196238] pci_bus 0000:06: resource 10 [mem 0x000a0000-0x000bffff]
[    0.196239] pci_bus 0000:06: resource 11 [mem 0x000d0000-0x000d3fff]
[    0.196240] pci_bus 0000:06: resource 12 [mem 0x000d4000-0x000d7fff]
[    0.196241] pci_bus 0000:06: resource 13 [mem 0x000d8000-0x000dbfff]
[    0.196242] pci_bus 0000:06: resource 14 [mem 0x000dc000-0x000dffff]
[    0.196243] pci_bus 0000:06: resource 15 [mem 0x000e0000-0x000e3fff]
[    0.196245] pci_bus 0000:06: resource 16 [mem 0x000e4000-0x000e7fff]
[    0.196246] pci_bus 0000:06: resource 17 [mem 0xcfa00000-0xfeafffff]
[    0.196247] pci_bus 0000:07: resource 0 [io  0xb000-0xbfff]
[    0.196248] pci_bus 0000:07: resource 1 [mem 0xf7900000-0xf79fffff]
[    0.196249] pci_bus 0000:08: resource 0 [io  0xa000-0xafff]
[    0.196250] pci_bus 0000:08: resource 1 [mem 0xf7800000-0xf78fffff]
[    0.196267] NET: Registered protocol family 2
[    0.196351] TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.196492] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.196598] TCP: Hash tables configured (established 131072 bind 65536)
[    0.196611] TCP: reno registered
[    0.196614] UDP hash table entries: 8192 (order: 6, 262144 bytes)
[    0.196646] UDP-Lite hash table entries: 8192 (order: 6, 262144 bytes)
[    0.196708] NET: Registered protocol family 1
[    0.196757] RPC: Registered named UNIX socket transport module.
[    0.196759] RPC: Registered udp transport module.
[    0.196760] RPC: Registered tcp transport module.
[    0.196761] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.196768] pci 0000:00:02.0: Boot video device
[    0.229229] PCI: CLS 64 bytes, default 64
[    0.229264] Trying to unpack rootfs image as initramfs...
[    0.284002] Freeing initrd memory: 4416K (ffff88007fbb0000 - ffff880080000000)
[    0.284018] DMAR: No ATSR found
[    0.284039] IOMMU 1 0xfed91000: using Queued invalidation
[    0.284041] IOMMU: Setting RMRR:
[    0.284051] IOMMU: Setting identity map for device 0000:00:1d.0 [0x8d86f000 - 0x8d899fff]
[    0.284073] IOMMU: Setting identity map for device 0000:00:1a.0 [0x8d86f000 - 0x8d899fff]
[    0.284089] IOMMU: Setting identity map for device 0000:00:14.0 [0x8d86f000 - 0x8d899fff]
[    0.284110] IOMMU: Prepare 0-16MiB unity mapping for LPC
[    0.284117] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.284200] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[    0.286013] RAPL PMU detected, hw unit 2^-16 Joules, API unit is 2^-32 Joules, 3 fixed counters 163840 ms ovfl timer
[    0.286395] AVX version of gcm_enc/dec engaged.
[    0.286784] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.286800] audit: initializing netlink subsys (disabled)
[    0.286812] audit: type=2000 audit(1399140815.274:1): initialized
[    0.286984] bounce pool size: 64 pages
[    0.287917] VFS: Disk quotas dquot_6.5.2
[    0.287946] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.288061] NFS: Registering the id_resolver key type
[    0.288068] Key type id_resolver registered
[    0.288069] Key type id_legacy registered
[    0.288074] ROMFS MTD (C) 2007 Red Hat, Inc.
[    0.288093] msgmni has been set to 30072
[    0.288892] async_tx: api initialized (async)
[    0.288914] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    0.288935] io scheduler noop registered
[    0.288936] io scheduler deadline registered
[    0.288954] io scheduler cfq registered (default)
[    0.289090] pcieport 0000:00:01.0: irq 42 for MSI/MSI-X
[    0.289578] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.289596] vesafb: mode is 800x600x8, linelength=832, pages=202
[    0.289598] vesafb: scrolling: redraw
[    0.289599] vesafb: Pseudocolor: size=8:8:8:8, shift=0:0:0:0
[    0.289701] vesafb: framebuffer at 0xd0000000, mapped to 0xffffc90004100000, using 975k, total 104384k
[    0.297086] Console: switching to colour frame buffer device 100x37
[    0.305277] fb0: VESA VGA frame buffer device
[    0.305380] intel_idle: MWAIT substates: 0x1120
[    0.305380] intel_idle: v0.4 model 0x3A
[    0.305381] intel_idle: lapic_timer_reliable_states 0xffffffff
[    0.305485] GHES: HEST is not enabled!
[    0.306647] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.308085] brd: module loaded
[    0.308238] ahci 0000:00:1f.2: version 3.0
[    0.308319] ahci 0000:00:1f.2: irq 43 for MSI/MSI-X
[    0.319117] ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0xd impl SATA mode
[    0.319296] ahci 0000:00:1f.2: flags: 64bit ncq led clo pio slum part ems apst 
[    0.323564] scsi0 : ahci
[    0.326680] scsi1 : ahci
[    0.329688] scsi2 : ahci
[    0.332600] scsi3 : ahci
[    0.335430] scsi4 : ahci
[    0.338174] scsi5 : ahci
[    0.340731] ata1: SATA max UDMA/133 abar m2048@0xf7d36000 port 0xf7d36100 irq 43
[    0.343395] ata2: DUMMY
[    0.345961] ata3: SATA max UDMA/133 abar m2048@0xf7d36000 port 0xf7d36200 irq 43
[    0.348676] ata4: SATA max UDMA/133 abar m2048@0xf7d36000 port 0xf7d36280 irq 43
[    0.351370] ata5: DUMMY
[    0.354009] ata6: DUMMY
[    0.356681] ahci 0000:03:00.0: irq 44 for MSI/MSI-X
[    0.356730] ahci 0000:03:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    0.359509] ahci 0000:03:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    0.362537] scsi6 : ahci
[    0.365330] scsi7 : ahci
[    0.367926] ata7: SATA max UDMA/133 abar m512@0xf7c10000 port 0xf7c10100 irq 44
[    0.370685] ata8: SATA max UDMA/133 abar m512@0xf7c10000 port 0xf7c10180 irq 44
[    0.373486] ahci 0000:08:00.0: irq 45 for MSI/MSI-X
[    0.373533] ahci 0000:08:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    0.376323] ahci 0000:08:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    0.379379] scsi8 : ahci
[    0.382250] scsi9 : ahci
[    0.384920] ata9: SATA max UDMA/133 abar m512@0xf7810000 port 0xf7810100 irq 45
[    0.387686] ata10: SATA max UDMA/133 abar m512@0xf7810000 port 0xf7810180 irq 45
[    0.391105] scsi10 : pata_legacy
[    0.393829] ata11: PATA max PIO4 cmd 0x1f0 ctl 0x3f6 irq 14
[    0.662132] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    0.664922] ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    0.667591] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    0.670237] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20131218/psargs-359)
[    0.672885] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT2._GTF] (Node ffff88041e04f2d0), AE_NOT_FOUND (20131218/psparse-536)
[    0.678218] ata3.00: ATAPI: HL-DT-ST DVDRAM GH22NS50, TN01, max UDMA/100
[    0.681062] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20131218/psargs-359)
[    0.684126] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT0._GTF] (Node ffff88041e04f1e0), AE_NOT_FOUND (20131218/psparse-536)
[    0.690623] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20131218/psargs-359)
[    0.694115] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT3._GTF] (Node ffff88041e04f348)
[    0.696131] ata9: SATA link down (SStatus 0 SControl 300)
[    0.701136] , AE_NOT_FOUND<6>[    0.701179] ata10: SATA link down (SStatus 0 SControl 300)
[    0.708170]  (20131218/psparse-536)
[    0.711757] ata4.00: ATAPI: HL-DT-ST BD-RE  BH14NS40, 1.01, max UDMA/100
[    0.715538] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20131218/psargs-359)
[    0.719420] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT2._GTF] (Node ffff88041e04f2d0), AE_NOT_FOUND (20131218/psparse-536)
[    0.727197] ata3.00: configured for UDMA/100
[    0.731264] ata1.00: ATA-7: SAMSUNG HD080HJ/P, ZH100-34, max UDMA7
[    0.735257] ata1.00: 156250000 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
[    0.739341] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20131218/psargs-359)
[    0.743433] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT3._GTF] (Node ffff88041e04f348), AE_NOT_FOUND (20131218/psparse-536)
[    0.751643] ata4.00: configured for UDMA/100
[    0.760719] ACPI Error: [DSSP] Namespace lookup failure, AE_NOT_FOUND (20131218/psargs-359)
[    0.765068] ACPI Error: Method parse/execution failed [\_SB_.PCI0.SAT0.SPT0._GTF] (Node ffff88041e04f1e0), AE_NOT_FOUND (20131218/psparse-536)
[    0.773876] ata1.00: configured for UDMA/133
[    0.833136] ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    0.837532] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[    0.842064] ata8.00: ATA-8: ST2000DL003-9VT166, CC32, max UDMA/133
[    0.846537] ata8.00: 3907029168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    0.851070] ata7.00: ATA-8: ST3500418AS, CC38, max UDMA/133
[    0.855392] ata7.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    0.860254] ata8.00: configured for UDMA/133
[    0.864716] ata7.00: configured for UDMA/133
[    1.286130] tsc: Refined TSC clocksource calibration: 3002.953 MHz
[    2.290320] Switched to clocksource tsc
[    3.322312] floppy0: no floppy controllers found
[    3.326349] scsi 0:0:0:0: Direct-Access     ATA      SAMSUNG HD080HJ/ ZH10 PQ: 0 ANSI: 5
[    3.330315] sd 0:0:0:0: [sda] 156250000 512-byte logical blocks: (80.0 GB/74.5 GiB)
[    3.334167] sd 0:0:0:0: [sda] Write Protect is off
[    3.337946] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    3.337957] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.337979] scsi 2:0:0:0: CD-ROM            HL-DT-ST DVDRAM GH22NS50  TN01 PQ: 0 ANSI: 5
[    3.353791]  sda: sda1 sda3 sda4
[    3.357854] sd 0:0:0:0: [sda] Attached SCSI disk
[    3.365291] sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray
[    3.368997] cdrom: Uniform CD-ROM driver Revision: 3.20
[    3.372768] sr 2:0:0:0: Attached scsi CD-ROM sr0
[    3.373177] scsi 3:0:0:0: CD-ROM            HL-DT-ST BD-RE  BH14NS40  1.01 PQ: 0 ANSI: 5
[    3.377682] sr1: scsi3-mmc drive: 153x/306x writer dvd-ram cd/rw xa/form2 cdda tray
[    3.381622] sr 3:0:0:0: Attached scsi CD-ROM sr1
[    3.381877] scsi 6:0:0:0: Direct-Access     ATA      ST3500418AS      CC38 PQ: 0 ANSI: 5
[    3.385838] sd 6:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[    3.389624] sd 6:0:0:0: [sdb] Write Protect is off
[    3.389635] scsi 7:0:0:0: Direct-Access     ATA      ST2000DL003-9VT1 CC32 PQ: 0 ANSI: 5
[    3.389735] sd 7:0:0:0: [sdc] 3907029168 512-byte logical blocks: (2.00 TB/1.81 TiB)
[    3.389754] sd 7:0:0:0: [sdc] Write Protect is off
[    3.389755] sd 7:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[    3.389763] sd 7:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.411312] sd 6:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    3.411323] sd 6:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    3.411335]  sdc: sdc1 sdc2 sdc3
[    3.411642] sd 7:0:0:0: [sdc] Attached SCSI disk
[    3.435094]  sdb: sdb1 sdb2 sdb3
[    3.438999] sd 6:0:0:0: [sdb] Attached SCSI disk
[    3.443001] scsi11 : pata_legacy
[    3.446568] ata12: PATA max PIO4 cmd 0x170 ctl 0x376 irq 15
[    3.601504] i8042: PNP: No PS/2 controller found. Probing ports directly.
[    3.605604] serio: i8042 KBD port at 0x60,0x64 irq 1
[    3.609295] serio: i8042 AUX port at 0x60,0x64 irq 12
[    3.613049] mousedev: PS/2 mouse device common for all mice
[    3.616696] rtc_cmos 00:05: RTC can wake from S4
[    3.620425] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0
[    3.624007] rtc_cmos 00:05: alarms up to one month, y3k, 242 bytes nvram, hpet irqs
[    3.627596] md: linear personality registered for level -1
[    3.631105] md: raid0 personality registered for level 0
[    3.634573] md: raid1 personality registered for level 1
[    3.637938] md: raid10 personality registered for level 10
[    3.641443] md: raid6 personality registered for level 6
[    3.644779] md: raid5 personality registered for level 5
[    3.647992] md: raid4 personality registered for level 4
[    3.651082] md: multipath personality registered for level -4
[    3.654177] device-mapper: uevent: version 1.0.3
[    3.657269] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
[    3.660357] Intel P-state driver initializing.
[    3.663393] Intel pstate controlling: cpu 0
[    3.666361] Intel pstate controlling: cpu 1
[    3.669207] Intel pstate controlling: cpu 2
[    3.671961] Intel pstate controlling: cpu 3
[    3.674703] TCP: cubic registered
[    3.677170] Initializing XFRM netlink socket
[    3.679568] NET: Registered protocol family 17
[    3.681896] Key type dns_resolver registered
[    3.684630] registered taskstats version 1
[    3.687252] regulator-dummy: disabling
[    3.741594] Freeing unused kernel memory: 1088K (ffffffff81aad000 - ffffffff81bbd000)
[    3.746033] Write protecting the kernel read-only data: 10240k
[    3.751286] Freeing unused kernel memory: 168K (ffff8800015d6000 - ffff880001600000)
[    3.761439] Freeing unused kernel memory: 2016K (ffff880001808000 - ffff880001a00000)
[    3.778921] udevd[177]: starting version 182
[    3.909538] random: nonblocking pool is initialized
[    5.366297] EXT4-fs (sda3): mounted filesystem with ordered data mode. Opts: (null)
[    6.535776] loop: module loaded
[    6.741465] udevd[302]: starting version 182
[    7.436152] microcode: CPU0 sig=0x306a9, pf=0x2, revision=0x12
[    7.451666] microcode: CPU1 sig=0x306a9, pf=0x2, revision=0x12
[    7.456544] microcode: CPU2 sig=0x306a9, pf=0x2, revision=0x12
[    7.461232] microcode: CPU3 sig=0x306a9, pf=0x2, revision=0x12
[    7.465689] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    7.580269] input: Power Button as /devices/LNXSYSTM:00/device:00/PNP0C0C:00/input/input2
[    7.585231] ACPI: Power Button [PWRB]
[    7.590320] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3
[    7.592892] ACPI Warning: SystemIO range 0x0000000000000428-0x000000000000042f conflicts with OpRegion 0x0000000000000400-0x000000000000047f (\PMIO) (20131218/utaddress-258)
[    7.592893] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[    7.592898] ACPI Warning: SystemIO range 0x0000000000000530-0x000000000000053f conflicts with OpRegion 0x0000000000000500-0x0000000000000563 (\GPIO) (20131218/utaddress-258)
[    7.592899] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[    7.592902] ACPI Warning: SystemIO range 0x0000000000000500-0x000000000000052f conflicts with OpRegion 0x0000000000000500-0x000000000000051f (\LED_) (20131218/utaddress-258)
[    7.592904] ACPI Warning: SystemIO range 0x0000000000000500-0x000000000000052f conflicts with OpRegion 0x0000000000000500-0x0000000000000563 (\GPIO) (20131218/utaddress-258)
[    7.592905] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[    7.592905] lpc_ich: Resource conflict(s) found affecting gpio_ich
[    7.694470] ACPI: Power Button [PWRF]
[    7.706431] atl1c 0000:07:00.0: version 1.0.1.1-NAPI
[    7.758058] ACPI Warning: SystemIO range 0x000000000000f040-0x000000000000f05f conflicts with OpRegion 0x000000000000f040-0x000000000000f04f (\_SB_.PCI0.SBUS.SMBI) (20131218/utaddress-258)
[    7.773406] ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[    7.966567] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    7.974710] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 1
[    7.982932] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported
[    7.982959] xhci_hcd 0000:00:14.0: irq 46 for MSI/MSI-X
[    7.983029] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    7.990685] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    7.995799] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    8.002720] ehci-pci: EHCI PCI platform driver
[    8.014456] usb usb1: Product: xHCI Host Controller
[    8.021993] usb usb1: Manufacturer: Linux 3.14.2 xhci_hcd
[    8.029408] usb usb1: SerialNumber: 0000:00:14.0
[    8.036423] hub 1-0:1.0: USB hub found
[    8.039177] Linux agpgart interface v0.103
[    8.049809] hub 1-0:1.0: 4 ports detected
[    8.056273] xhci_hcd 0000:00:14.0: xHCI Host Controller
[    8.059294] wmi: Mapper loaded
[    8.059334] pps_core: LinuxPPS API ver. 1 registered
[    8.059334] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    8.072047] PTP clock support registered
[    8.091050] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 2
[    8.096048] e1000e: Intel(R) PRO/1000 Network Driver - 2.3.2-k
[    8.096048] e1000e: Copyright(c) 1999 - 2013 Intel Corporation.
[    8.106748] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    8.111574] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.116295] usb usb2: Product: xHCI Host Controller
[    8.120786] usb usb2: Manufacturer: Linux 3.14.2 xhci_hcd
[    8.125158] usb usb2: SerialNumber: 0000:00:14.0
[    8.129602] hub 2-0:1.0: USB hub found
[    8.133700] hub 2-0:1.0: 4 ports detected
[    8.137998] ehci-pci 0000:00:1a.0: EHCI Host Controller
[    8.143612] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 3
[    8.149229] ACPI: Fan [FAN0] (off)
[    8.149248] ehci-pci 0000:00:1a.0: debug port 2
[    8.153167] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported
[    8.153187] ehci-pci 0000:00:1a.0: irq 16, io mem 0xf7d38000
[    8.159159] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00
[    8.159194] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[    8.159196] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.159197] usb usb3: Product: EHCI Host Controller
[    8.159199] usb usb3: Manufacturer: Linux 3.14.2 ehci_hcd
[    8.159200] usb usb3: SerialNumber: 0000:00:1a.0
[    8.159347] hub 3-0:1.0: USB hub found
[    8.159353] hub 3-0:1.0: 2 ports detected
[    8.159619] ehci-pci 0000:00:1d.0: EHCI Host Controller
[    8.159674] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 4
[    8.159687] ehci-pci 0000:00:1d.0: debug port 2
[    8.163604] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported
[    8.163619] ehci-pci 0000:00:1d.0: irq 23, io mem 0xf7d37000
[    8.169159] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00
[    8.169188] usb usb4: New USB device found, idVendor=1d6b, idProduct=0002
[    8.169189] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.169190] usb usb4: Product: EHCI Host Controller
[    8.169192] usb usb4: Manufacturer: Linux 3.14.2 ehci_hcd
[    8.169193] usb usb4: SerialNumber: 0000:00:1d.0
[    8.169327] hub 4-0:1.0: USB hub found
[    8.169334] hub 4-0:1.0: 2 ports detected
[    8.169617] mei_me 0000:00:16.0: irq 47 for MSI/MSI-X
[    8.179432] [drm] Initialized drm 1.1.0 20060810
[    8.268729] ACPI: Fan [FAN1] (off)
[    8.269408] e1000e 0000:00:19.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
[    8.269429] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[    8.281495] ACPI: Fan [FAN2] (off)
[    8.285682] ACPI: Fan [FAN3] (off)
[    8.289827] ACPI: Fan [FAN4] (off)
[    8.300765] thermal LNXTHERM:00: registered as thermal_zone0
[    8.305258] ACPI: Thermal Zone [TZ00] (28 C)
[    8.309988] thermal LNXTHERM:01: registered as thermal_zone1
[    8.314526] ACPI: Thermal Zone [TZ01] (30 C)
[    8.351233] firewire_ohci 0000:06:01.0: added OHCI v1.10 device as card 0, 4 IR + 8 IT contexts, quirks 0x11
[    8.411175] usb 1-4: new low-speed USB device number 2 using xhci_hcd
[    8.529359] e1000e 0000:00:19.0 eth1: registered PHC clock
[    8.534400] e1000e 0000:00:19.0 eth1: (PCI Express:2.5GT/s:Width x1) 90:2b:34:39:9e:ad
[    8.539538] e1000e 0000:00:19.0 eth1: Intel(R) PRO/1000 Network Connection
[    8.544508] e1000e 0000:00:19.0 eth1: MAC: 10, PHY: 11, PBA No: FFFFFF-0FF
[    8.550596] [drm] Memory usable by graphics device = 2048M
[    8.555463] checking generic (d0000000 65f0000) vs hw (d0000000 10000000)
[    8.555465] fb: conflicting fb hw usage inteldrmfb vs VESA VGA - removing generic driver
[    8.560905] Console: switching to colour dummy device 80x25
[    8.583427] usb 1-4: New USB device found, idVendor=04f3, idProduct=0801
[    8.583433] usb 1-4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    8.583517] usb 1-4: ep 0x81 - rounding interval to 64 microframes, ep desc says 80 microframes
[    8.583524] usb 1-4: ep 0x82 - rounding interval to 64 microframes, ep desc says 80 microframes
[    8.606219] i915 0000:00:02.0: irq 49 for MSI/MSI-X
[    8.606227] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    8.606231] [drm] Driver supports precise vblank timestamp query.
[    8.606294] vgaarb: device changed decodes: PCI:0000:00:02.0,olddecodes=io+mem,decodes=none:owns=none
[    8.626354] [drm] Wrong MCH_SSKPD value: 0x20100406
[    8.626358] [drm] This can cause pipe underruns and display issues.
[    8.626360] [drm] Please upgrade your BIOS to fix this.
[    8.651677] fbcon: inteldrmfb (fb0) is primary device
[    8.687695] usb 2-3: new SuperSpeed USB device number 2 using xhci_hcd
[    8.719763] cfg80211: Calling CRDA to update world regulatory domain
[    8.751917] usb 2-3: New USB device found, idVendor=2109, idProduct=0810
[    8.751919] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    8.751921] usb 2-3: Product: 4-Port USB 3.0 Hub
[    8.751922] usb 2-3: Manufacturer: VIA Labs, Inc.
[    8.772738] hub 2-3:1.0: USB hub found
[    8.776085] hub 2-3:1.0: 4 ports detected
[    8.814223] Console: switching to colour frame buffer device 180x56
[    8.818959] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
[    8.818995] i915 0000:00:02.0: registered panic notifier
[    8.824499] ACPI: Video Device [GFX0] (multi-head: yes  rom: no  post: no)
[    8.824853] acpi device:66: registered as cooling_device10
[    8.824978] input: Video Bus as /devices/LNXSYSTM:00/device:00/PNP0A08:00/LNXVIDEO:00/input/input5
[    8.825146] [drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0
[    8.825612] snd_hda_intel 0000:00:1b.0: irq 50 for MSI/MSI-X
[    8.861258] firewire_core 0000:06:01.0: created device fw0: GUID 0049e550cda64b00, S400
[    8.898050] ath: EEPROM regdomain: 0x6a
[    8.898053] ath: EEPROM indicates we should expect a direct regpair map
[    8.898056] ath: Country alpha2 being used: 00
[    8.898057] ath: Regpair used: 0x6a
[    8.898837] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht'
[    8.899024] ieee80211 phy0: Atheros AR9462 Rev:2 mem=0xffffc90004480000, irq=19
[    8.915833] usb 2-4: new SuperSpeed USB device number 3 using xhci_hcd
[    8.980242] usb 2-4: New USB device found, idVendor=2109, idProduct=0810
[    8.980287] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    8.980329] usb 2-4: Product: 4-Port USB 3.0 Hub
[    8.980358] usb 2-4: Manufacturer: VIA Labs, Inc.
[    8.988807] i2c /dev entries driver
[    9.003347] hub 2-4:1.0: USB hub found
[    9.006782] hub 2-4:1.0: 4 ports detected
[    9.146202] usb 3-1: new high-speed USB device number 2 using ehci-pci
[    9.185839] input: HDA Intel PCH HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:1b.0/sound/card0/input15
[    9.185976] input: HDA Intel PCH HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:1b.0/sound/card0/input14
[    9.186084] input: HDA Intel PCH Front Headphone as /devices/pci0000:00/0000:00:1b.0/sound/card0/input13
[    9.186196] input: HDA Intel PCH Line Out Side as /devices/pci0000:00/0000:00:1b.0/sound/card0/input12
[    9.187282] input: HDA Intel PCH Line Out CLFE as /devices/pci0000:00/0000:00:1b.0/sound/card0/input11
[    9.188872] input: HDA Intel PCH Line Out Surround as /devices/pci0000:00/0000:00:1b.0/sound/card0/input10
[    9.190445] input: HDA Intel PCH Line Out Front as /devices/pci0000:00/0000:00:1b.0/sound/card0/input9
[    9.192000] input: HDA Intel PCH Line as /devices/pci0000:00/0000:00:1b.0/sound/card0/input8
[    9.193553] input: HDA Intel PCH Rear Mic as /devices/pci0000:00/0000:00:1b.0/sound/card0/input7
[    9.195110] input: HDA Intel PCH Front Mic as /devices/pci0000:00/0000:00:1b.0/sound/card0/input6
[    9.260650] usb 3-1: New USB device found, idVendor=8087, idProduct=0024
[    9.262356] usb 3-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    9.264405] hub 3-1:1.0: USB hub found
[    9.266230] hub 3-1:1.0: 6 ports detected
[    9.309531] hidraw: raw HID events driver (C) Jiri Kosina
[    9.371185] usb 4-1: new high-speed USB device number 2 using ehci-pci
[    9.486646] usb 4-1: New USB device found, idVendor=8087, idProduct=0024
[    9.488338] usb 4-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[    9.490310] hub 4-1:1.0: USB hub found
[    9.492127] hub 4-1:1.0: 8 ports detected
[    9.516451] cfg80211: World regulatory domain updated:
[    9.518097] cfg80211:  DFS Master region: unset
[    9.518125] cfg80211:   (start_freq - end_freq @ bandwidth), (max_antenna_gain, max_eirp)
[    9.521303] cfg80211:   (2402000 KHz - 2472000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[    9.522913] cfg80211:   (2457000 KHz - 2482000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[    9.524501] cfg80211:   (2474000 KHz - 2494000 KHz @ 20000 KHz), (300 mBi, 2000 mBm)
[    9.526067] cfg80211:   (5170000 KHz - 5250000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[    9.527625] cfg80211:   (5735000 KHz - 5835000 KHz @ 40000 KHz), (300 mBi, 2000 mBm)
[    9.568256] usb 3-1.3: new high-speed USB device number 3 using ehci-pci
[    9.628152] usbcore: registered new interface driver usbhid
[    9.629701] usbhid: USB HID core driver
[    9.657128] usb 3-1.3: New USB device found, idVendor=05e3, idProduct=0605
[    9.658674] usb 3-1.3: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[    9.660213] usb 3-1.3: Product: USB2.0 Hub
[    9.662193] hub 3-1.3:1.0: USB hub found
[    9.664002] hub 3-1.3:1.0: 4 ports detected
[    9.714546] [drm] Enabling RC6 states: RC6 on, RC6p on, RC6pp off
[    9.740246] usb 3-1.4: new high-speed USB device number 4 using ehci-pci
[    9.846628] usb 3-1.4: New USB device found, idVendor=0ac8, idProduct=3450
[    9.848192] usb 3-1.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    9.849744] usb 3-1.4: Product: Vimicro USB Camera (Altair)
[    9.851285] usb 3-1.4: Manufacturer: Vimicro
[    9.852413] input: HID 04f3:0801 as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.0/0003:04F3:0801.0001/input/input16
[    9.852695] hid-generic 0003:04F3:0801.0001: input,hidraw0: USB HID v1.10 Keyboard [HID 04f3:0801] on usb-0000:00:14.0-4/input0
[    9.856922] input: HID 04f3:0801 as /devices/pci0000:00/0000:00:14.0/usb1/1-4/1-4:1.1/0003:04F3:0801.0002/input/input17
[    9.860061] hid-generic 0003:04F3:0801.0002: input,hidraw1: USB HID v1.10 Mouse [HID 04f3:0801] on usb-0000:00:14.0-4/input1
[    9.931236] usb 3-1.5: new high-speed USB device number 5 using ehci-pci
[   10.019355] usb 3-1.5: New USB device found, idVendor=2109, idProduct=0810
[   10.020976] usb 3-1.5: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[   10.022588] usb 3-1.5: Product: USB2.0 Hub
[   10.024515] hub 3-1.5:1.0: USB hub found
[   10.026355] hub 3-1.5:1.0: 4 ports detected
[   10.039206] Linux video capture interface: v2.00
[   10.107235] usb 3-1.6: new high-speed USB device number 6 using ehci-pci
[   10.143896] uvcvideo: Found UVC 1.00 device Vimicro USB Camera (Altair) (0ac8:3450)
[   10.147333] input: Vimicro USB Camera (Altair) as /devices/pci0000:00/0000:00:1a.0/usb3/3-1/3-1.4/3-1.4:1.0/input/input18
[   10.149091] usbcore: registered new interface driver uvcvideo
[   10.150767] USB Video Class driver (1.1.1)
[   10.195785] usb 3-1.6: New USB device found, idVendor=2109, idProduct=0810
[   10.197458] usb 3-1.6: New USB device strings: Mfr=0, Product=1, SerialNumber=0
[   10.199128] usb 3-1.6: Product: USB2.0 Hub
[   10.201202] hub 3-1.6:1.0: USB hub found
[   10.202996] hub 3-1.6:1.0: 4 ports detected
[   10.228181] [drm] GMBUS [i915 gmbus panel] timed out, falling back to bit banging on pin 3
[   10.284289] usb 4-1.6: new high-speed USB device number 3 using ehci-pci
[   10.289182] [drm] GMBUS [i915 gmbus ssc] timed out, falling back to bit banging on pin 1
[   10.375394] usb 4-1.6: New USB device found, idVendor=05e3, idProduct=0716
[   10.377077] usb 4-1.6: New USB device strings: Mfr=0, Product=1, SerialNumber=2
[   10.378739] usb 4-1.6: Product: USB Storage
[   10.380398] usb 4-1.6: SerialNumber: 000000009727
[   10.402102] usb-storage 4-1.6:1.0: USB Mass Storage device detected
[   10.404082] scsi12 : usb-storage 4-1.6:1.0
[   10.405833] usbcore: registered new interface driver usb-storage
[   10.456235] usb 4-1.7: new full-speed USB device number 4 using ehci-pci
[   10.533852] usb 4-1.7: string descriptor 0 read error: -22
[   10.535546] usb 4-1.7: New USB device found, idVendor=04ca, idProduct=3006
[   10.537224] usb 4-1.7: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[   10.606803] Bluetooth: Core ver 2.18
[   10.608501] NET: Registered protocol family 31
[   10.610164] Bluetooth: HCI device and connection manager initialized
[   10.611865] Bluetooth: HCI socket layer initialized
[   10.613585] Bluetooth: L2CAP socket layer initialized
[   10.615315] Bluetooth: SCO socket layer initialized
[   10.615340] usb 3-1.3.4: new low-speed USB device number 7 using ehci-pci
[   10.651159] usbcore: registered new interface driver btusb
[   10.656054] usbcore: registered new interface driver ath3k
[   10.708515] usb 3-1.3.4: New USB device found, idVendor=0bc7, idProduct=0004
[   10.710244] usb 3-1.3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   10.711983] usb 3-1.3.4: Product: USB Receiver
[   10.713692] usb 3-1.3.4: Manufacturer: X10 Wireless Technology Inc
[   10.814199] Registered IR keymap rc-ati-x10
[   10.816029] input: X10 Wireless Technology Inc USB Receiver as /devices/pci0000:00/0000:00:1a.0/usb3/3-1/3-1.3/3-1.3.4/3-1.3.4:1.0/rc/rc0/input19
[   10.817931] rc0: X10 Wireless Technology Inc USB Receiver as /devices/pci0000:00/0000:00:1a.0/usb3/3-1/3-1.3/3-1.3.4/3-1.3.4:1.0/rc/rc0
[   10.819812] input: X10 Wireless Technology Inc USB Receiver mouse as /devices/pci0000:00/0000:00:1a.0/usb3/3-1/3-1.3/3-1.3.4/3-1.3.4:1.0/input/input20
[   10.821919] usbcore: registered new interface driver ati_remote
[   10.902866] fuse init (API version 7.22)
[   11.407682] scsi 12:0:0:0: Direct-Access     Generic  STORAGE DEVICE-A 9727 PQ: 0 ANSI: 0
[   11.412102] scsi 12:0:0:1: Direct-Access     Generic  STORAGE DEVICE-A 9727 PQ: 0 ANSI: 0
[   11.414125] sd 12:0:0:0: [sdd] Attached SCSI removable disk
[   11.418118] scsi 12:0:0:2: Direct-Access     Generic  STORAGE DEVICE-A 9727 PQ: 0 ANSI: 0
[   11.423617] scsi 12:0:0:3: Direct-Access     Generic  STORAGE DEVICE-A 9727 PQ: 0 ANSI: 0 CCS
[   11.425883] sd 12:0:0:1: [sde] Attached SCSI removable disk
[   11.429368] scsi 12:0:0:4: Direct-Access     Generic  STORAGE DEVICE-A 9727 PQ: 0 ANSI: 0
[   11.434365] sd 12:0:0:2: [sdf] Attached SCSI removable disk
[   11.442237] sd 12:0:0:3: [sdg] Attached SCSI removable disk
[   11.450737] sd 12:0:0:4: [sdh] Attached SCSI removable disk
[   11.916324] EXT4-fs (sda3): re-mounted. Opts: (null)
[   12.374753] ppdev: user-space parallel port driver
[   12.382409] lp: driver loaded but no devices found
[   15.609650] EXT4-fs (sda4): mounted filesystem with ordered data mode. Opts: (null)
[   20.465461] NET: Registered protocol family 10
[   31.151029] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   31.251373] e1000e 0000:00:19.0: irq 48 for MSI/MSI-X
[   31.251481] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[   31.261402] atl1c 0000:07:00.0: irq 51 for MSI/MSI-X
[   31.261949] atl1c 0000:07:00.0: atl1c: eth0 NIC Link is Up<100 Mbps Full Duplex>
[   32.787211] e1000e: eth1 NIC Link is Up 100 Mbps Full Duplex, Flow Control: Rx/Tx
[   32.787219] e1000e 0000:00:19.0 eth1: 10/100 speed: disabling TSO
[   32.787255] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[   51.481723] Guest personality initialized and is inactive
[   51.481982] VMCI host device registered (name=vmci, major=10, minor=56)
[   51.481985] Initialized host personality
[   51.520320] NET: Registered protocol family 40
[   53.298288] tun: Universal TUN/TAP device driver, 1.6
[   53.298292] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[   56.579323] Ebtables v2.0 registered
[   57.557764] ip_tables: (C) 2000-2006 Netfilter Core Team
[   58.163779] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   79.261291] Bridge firewalling registered
[   79.289732] device virbr4-nic entered promiscuous mode
[   79.429923] virbr4: port 1(virbr4-nic) entered listening state
[   79.429939] virbr4: port 1(virbr4-nic) entered listening state
[   79.429986] IPv6: ADDRCONF(NETDEV_UP): virbr4: link is not ready
[   79.442113] virbr4: port 1(virbr4-nic) entered disabled state
[   79.544700] device virbr2-nic entered promiscuous mode
[   79.624098] virbr2: port 1(virbr2-nic) entered listening state
[   79.624113] virbr2: port 1(virbr2-nic) entered listening state
[   79.624179] IPv6: ADDRCONF(NETDEV_UP): virbr2: link is not ready
[   79.639076] virbr2: port 1(virbr2-nic) entered disabled state
[   79.712701] device virbr3-nic entered promiscuous mode
[   79.758094] virbr3: port 1(virbr3-nic) entered listening state
[   79.758109] virbr3: port 1(virbr3-nic) entered listening state
[   79.758129] IPv6: ADDRCONF(NETDEV_UP): virbr3: link is not ready
[   79.764835] virbr3: port 1(virbr3-nic) entered disabled state
[   79.819701] device virbr1-nic entered promiscuous mode
[   79.871879] virbr1: port 1(virbr1-nic) entered listening state
[   79.871893] virbr1: port 1(virbr1-nic) entered listening state
[   79.871965] IPv6: ADDRCONF(NETDEV_UP): virbr1: link is not ready
[   79.883150] virbr1: port 1(virbr1-nic) entered disabled state
[   80.001807] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[   80.061613] IPv6: ADDRCONF(NETDEV_UP): virbr0: link is not ready

Then

#vfio-pci 0000:02:00.0
# qemu-system-x86_64 -enable-kvm -M q35 -m 1024 -cpu host -device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 -device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=/media/800/destroy_varios/donwload2/virtualizacion/MSI.7600GT.256.060310.rom -bios /var/lib/libvirt/bios/seabios-hp2.bin -vga none -drive file=/media/Utils/Sistemas\ Operativos/linux/Express-3.1-rc5-x86_64.iso,id=isocd -device ide-cd,bus=ide.1,drive=isocd -vnc 127.0.0.1:0 -vga none

and the result is in my "intel video" that color changed and in my "virtual Nvidia video" has not signal, the second monitor is in stand by mode . If a go to Console 1 (CONTROL+ALT+F1) and back to Grafic consle (ALT+F7) all seems fine.

All I see in dmesg is

l# dmesg 
[ 1873.878446] vfio-pci 0000:02:00.0: enabling device (0400 -> 0403)

Any clue why the nvdia is not sending signal?

Then when I open the VNC session all is in blank.

Last edited by Destroy (2014-05-06 00:51:03)

Offline

#1656 2014-05-06 00:53:11

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Destroy wrote:

Any clue why the nvdia is not sending signal?

You need the i915 vga arbiter patches


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1657 2014-05-07 05:23:54

Destroy
Member
Registered: 2014-05-05
Posts: 10

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:

You need the i915 vga arbiter patches

Thanks!!! It is sending signal!!!


Now I just need a mouse and keyboard usb to attach to VM

Offline

#1658 2014-05-07 12:48:48

powerhouse
Member
Registered: 2013-05-22
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

Thanks nbhs for this excellent tutorial! I've wanted to try kvm for a long time, but only recently found time to do it. I have successfully passed through a Sapphire Radeon HD 7770 using primary passthrough to a Windows 7 Pro 64 bit guest. Below some more info:
OS: Ubuntu 14.04
Guest OS: Windows 7 Pro 64 bit
M/B: Asus Sabertooth X79
CPU: Intel i7 3930K (LGA 2011)
Host OS graphics card: Nvidia Quadro 2000 with Nvidia proprietary driver
bridged network configuration
vfio driver for network and disk
Disk is a LVM drive using raw format

Here some remarks and questions:
Passmark PerformanceTest benchmark only works in SAFEMODE, else Windows BSODs. Under Xen there is no such issue. Furthermore Passmark is unable to detect the SSE extensions of the CPU (again under Xen it works fine). CPU-Z, however, does recognize the SSE extensions.
PCmark has similar issues.
Is there anything I do wrong with the configuration?

EDIT: Here the configuration:
sudo qemu-system-x86_64 \
-bios /usr/share/qemu/bios.bin -vga none \
-name win7 \
-cpu host \
-smp 10,sockets=1,cores=5,threads=2 \
-enable-kvm \
-m 24G \
-rtc clock=host \
-vga none \
-serial null \
-parallel null \
-monitor none \
-display none \
-k en-us \
-machine type=q35,accel=kvm \
-boot order=cd \
-device ahci,id=ahci \
-device virtio-scsi-pci,id=scsi \
-drive file=/dev/external/win7amd,cache=none,if=virtio \
-drive file=/home/user/Win7.iso,id=isocd -device ide-cd,bus=ide.1,drive=isocd \
-drive file=/home/user/virtio-win-0.1-74.iso,id=virtiocd -device ide-cd,bus=ide.2,drive=virtiocd \
-device ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \
-device vfio-pci,host=01:00.1,bus=pcie.0 \
-device vfio-pci,host=00:1a.0,bus=pcie.0 \
-net nic,model=virtio,macaddr=00:16:3e:00:03:03 -net tap

Last edited by powerhouse (2014-05-07 18:54:16)

Offline

#1659 2014-05-07 13:19:49

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

powerhouse wrote:

Thanks nbhs for this excellent tutorial! I've wanted to try kvm for a long time, but only recently found time to do it. I have successfully passed through a Sapphire Radeon HD 7770 using primary passthrough to a Windows 7 Pro 64 bit guest. Below some more info:
OS: Ubuntu 14.04
Guest OS: Windows 7 Pro 64 bit
M/B: Asus Sabertooth X79
CPU: Intel i7 3930K (LGA 2011)
Host OS graphics card: Nvidia Quadro 2000 with Nvidia proprietary driver
bridged network configuration
vfio driver for network and disk
Disk is a LVM drive using raw format

Here some remarks and questions:
Passmark PerformanceTest benchmark only works in SAFEMODE, else Windows BSODs. Under Xen there is no such issue. Furthermore Passmark is unable to detect the SSE extensions of the CPU (again under Xen it works fine). CPU-Z, however, does recognize the SSE extensions.
PCmark has similar issues.
Is there anything I do wrong with the configuration?

See http://www.passmark.com/forum/showthrea … r-Number-4 comment #6 and beyond.  My suspicion is that Passmark makes some platform assumptions based on the CPU family.  It would be a great data point if you could confirm or deny that with Xen.  Thanks.


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1660 2014-05-07 14:46:17

anickname
Member
Registered: 2014-01-07
Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

adejour wrote:
gneville wrote:

Hi adejour,

Yes I managed to fix the bsods on reboots by following the instructions this guy had in his blog http://blog.ktz.me/?p=219 it works perfectly now.

adejour wrote:

Hello,

I have the same error as gneville ( https://bbs.archlinux.org/viewtopic.php … 8#p1403248 ).
VGA Passthrough works when I launch windows but if I reboot or stop and then launch again, I have this message&BSOD (PAGE_FAULT_IN_NONPAGED_AREA). It is just after the loading of windows (black background) and just before the launch of the desktop. It is exactly at this moment that the screen resolution must change.

Résumé:
- If I launch windows with my VM then stop windows and the VM, then reboot the host, and launch the VM again, it works.
- If I launch windows with my VM then stop windows and the VM, then launch the VM again, there is this BSOD.

Ok, it seems to be a good software workaround and I will try that. Thank you very much !

Anyway it doesn't explain the real problem. When the VM shutdown, the VM turns off but the screen still display the last image (windows blue background with "windows is shuting down" (or something like that, my windows is not in english)).
=> qemu doesn't switch off my GPU, and the screen stays on !

Do you know if it is a pbroblem with qemu? Or may be with the bios used with qemu?
Any idea to fix this ?!

I'm also having this problem.
Do you have any updates regarding it ?
This is my lspci diff before and after restart.

--- /ok 2014-05-06 21:17:34.720433511 +0300
+++ /nok        2014-05-06 21:13:16.259217401 +0300
@@ -1,9 +1,8 @@
 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Bonaire XT [Radeon HD 7790/8770] (prog-if 00 [VGA controller])
        Subsystem: PC Partner Limited / Sapphire Technology Radeon HD 7790 Dual-X OC
-       Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
+       Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
-       Latency: 0, Cache Line Size: 64 bytes
-       Interrupt: pin A routed to IRQ 71
+       Interrupt: pin A routed to IRQ 16
        Region 0: Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Region 2: Memory at f0000000 (64-bit, prefetchable) [size=8M]
        Region 4: I/O ports at e000 [size=256]
@@ -16,8 +15,8 @@
        Capabilities: [58] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
-               DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported-
-                       RlxdOrd- ExtTag+ PhantFunc- AuxPwr- NoSnoop+
+               DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
+                       RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
                        MaxPayload 128 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
@@ -32,14 +31,14 @@
                         Compliance De-emphasis: -6dB
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
                         EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
-       Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
-               Address: 00000000fee004b8  Data: 0000
+       Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
+               Address: 0000000000000000  Data: 0000
        Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
-               CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
+               CESta:  RxErr- BadTLP+ BadDLLP- Rollover+ Timeout+ NonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
                AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
        Capabilities: [270 v1] #19

Last edited by anickname (2014-05-07 14:47:10)

Offline

#1661 2014-05-07 14:59:33

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

anickname wrote:
adejour wrote:
gneville wrote:

Hi adejour,

Yes I managed to fix the bsods on reboots by following the instructions this guy had in his blog http://blog.ktz.me/?p=219 it works perfectly now.

Ok, it seems to be a good software workaround and I will try that. Thank you very much !

Anyway it doesn't explain the real problem. When the VM shutdown, the VM turns off but the screen still display the last image (windows blue background with "windows is shuting down" (or something like that, my windows is not in english)).
=> qemu doesn't switch off my GPU, and the screen stays on !

Do you know if it is a pbroblem with qemu? Or may be with the bios used with qemu?
Any idea to fix this ?!

I'm also having this problem.
Do you have any updates regarding it ?

It's complicated.  The only way we can reset these GPUs is via a bus reset, but there are multiple devices on the bus.  We therefore have an interface for QEMU to say "I own all of these devices and I want a reset", that's what happens on startup and on each VM reset.  When QEMU is shutdown, there is no final reset (maybe there should be).  VFIO sees the devices getting released, but it can't tell the difference between a single device being hot-unplugged from the guest and all of the devices getting released.  So, we probably need to add another reset when the group is being closed, but the device interface doesn't really have any visibility to that event.  Code contributions welcome...


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1662 2014-05-07 15:16:20

anickname
Member
Registered: 2014-01-07
Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
anickname wrote:
adejour wrote:

Ok, it seems to be a good software workaround and I will try that. Thank you very much !

Anyway it doesn't explain the real problem. When the VM shutdown, the VM turns off but the screen still display the last image (windows blue background with "windows is shuting down" (or something like that, my windows is not in english)).
=> qemu doesn't switch off my GPU, and the screen stays on !

Do you know if it is a pbroblem with qemu? Or may be with the bios used with qemu?
Any idea to fix this ?!

I'm also having this problem.
Do you have any updates regarding it ?

It's complicated.  The only way we can reset these GPUs is via a bus reset, but there are multiple devices on the bus.  We therefore have an interface for QEMU to say "I own all of these devices and I want a reset", that's what happens on startup and on each VM reset.  When QEMU is shutdown, there is no final reset (maybe there should be).  VFIO sees the devices getting released, but it can't tell the difference between a single device being hot-unplugged from the guest and all of the devices getting released.  So, we probably need to add another reset when the group is being closed, but the device interface doesn't really have any visibility to that event.  Code contributions welcome...

Thanks for the answer.
Still, I can't understand what is the difference between my setup and a setup which doesn't have this problem.
Looking at lspci's output the video card is the only device on that bus and in that specific group :

00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
00:02.0 Display controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04)
00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
00:1b.0 Audio device: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)
00:1c.4 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 5 (rev c4)
00:1c.7 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 (rev c4)
00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation Z77 Express Chipset LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller (rev 04)
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Bonaire XT [Radeon HD 7790/8770]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device 0002
02:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
04:00.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
05:01.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
05:04.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
05:05.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
05:06.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
05:07.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
05:08.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
05:09.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
07:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 01)
09:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
0a:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)
0b:00.0 USB controller: Etron Technology, Inc. EJ188/EJ198 USB 3.0 Host Controller
0c:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)
### Group 0 ###
    00:00.0 Host bridge: Intel Corporation 2nd Generation Core Processor Family DRAM Controller (rev 09)
### Group 1 ###
    00:01.0 PCI bridge: Intel Corporation Xeon E3-1200/2nd Generation Core Processor Family PCI Express Root Port (rev 09)
### Group 2 ###
    00:02.0 Display controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)
### Group 3 ###
    00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
### Group 4 ###
    00:16.0 Communication controller: Intel Corporation 7 Series/C210 Series Chipset Family MEI Controller #1 (rev 04)
### Group 5 ###
    00:1a.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #2 (rev 04)
### Group 6 ###
    00:1b.0 Audio device: Intel Corporation 7 Series/C210 Series Chipset Family High Definition Audio Controller (rev 04)
### Group 7 ###
    00:1c.0 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 1 (rev c4)
### Group 8 ###
    00:1c.4 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 5 (rev c4)
### Group 9 ###
    00:1c.7 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 8 (rev c4)
### Group 10 ###
    00:1d.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB Enhanced Host Controller #1 (rev 04)
### Group 11 ###
    00:1f.0 ISA bridge: Intel Corporation Z77 Express Chipset LPC Controller (rev 04)
    00:1f.2 SATA controller: Intel Corporation 7 Series/C210 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
    00:1f.3 SMBus: Intel Corporation 7 Series/C210 Series Chipset Family SMBus Controller (rev 04)
### Group 12 ###
    01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Bonaire XT [Radeon HD 7790/8770]
    01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device 0002
### Group 13 ###
    02:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
### Group 14 ###
    03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)
### Group 15 ###
    04:00.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 16 ###
    05:01.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 17 ###
    05:04.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 18 ###
    05:05.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 19 ###
    05:06.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 20 ###
    05:07.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 21 ###
    05:08.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 22 ###
    05:09.0 PCI bridge: PLX Technology, Inc. PEX 8608 8-lane, 8-Port PCI Express Gen 2 (5.0 GT/s) Switch (rev ba)
### Group 23 ###
    06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
### Group 24 ###
    07:00.0 PCI bridge: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge (rev 01)
### Group 25 ###
    09:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
### Group 26 ###
    0a:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)
### Group 27 ###
    0b:00.0 USB controller: Etron Technology, Inc. EJ188/EJ198 USB 3.0 Host Controller
### Group 28 ###
    0c:00.0 Ethernet controller: Broadcom Corporation NetLink BCM57781 Gigabit Ethernet PCIe (rev 10)

Offline

#1663 2014-05-07 15:20:38

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

anickname wrote:
aw wrote:
anickname wrote:

I'm also having this problem.
Do you have any updates regarding it ?

It's complicated.  The only way we can reset these GPUs is via a bus reset, but there are multiple devices on the bus.  We therefore have an interface for QEMU to say "I own all of these devices and I want a reset", that's what happens on startup and on each VM reset.  When QEMU is shutdown, there is no final reset (maybe there should be).  VFIO sees the devices getting released, but it can't tell the difference between a single device being hot-unplugged from the guest and all of the devices getting released.  So, we probably need to add another reset when the group is being closed, but the device interface doesn't really have any visibility to that event.  Code contributions welcome...

Thanks for the answer.
Still, I can't understand what is the difference between my setup and a setup which doesn't have this problem.
Looking at lspci's output the video card is the only device on that bus and in that specific group :

01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Bonaire XT [Radeon HD 7790/8770]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device 0002
### Group 12 ###
    01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Bonaire XT [Radeon HD 7790/8770]
    01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Device 0002

The only way you would not have this problem would be if your VGA device was a single function device (ie. it lacked an audio function).  For our purposes, function 0 and function 1 are two separate devices on the same bus.


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1664 2014-05-07 15:38:17

anickname
Member
Registered: 2014-01-07
Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

@aw

Regarding "hv-time" problem do you know if will it be fixed in 3.15 ?

https://bugzilla.kernel.org/show_bug.cgi?id=73721

Thanks.

Offline

#1665 2014-05-07 15:59:21

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

anickname wrote:

@aw

Regarding "hv-time" problem do you know if will it be fixed in 3.15 ?

https://bugzilla.kernel.org/show_bug.cgi?id=73721

I don't know, but I just poked one of the KVM maintainers about it and he's going to investigate.


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1666 2014-05-07 16:18:23

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
anickname wrote:

@aw

Regarding "hv-time" problem do you know if will it be fixed in 3.15 ?

https://bugzilla.kernel.org/show_bug.cgi?id=73721

I don't know, but I just poked one of the KVM maintainers about it and he's going to investigate.

Apparently it's already fixed in current 3.15-rc and Paolo plans send it for stable as well.


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1667 2014-05-07 18:08:46

Norcoen
Member
From: Norway
Registered: 2013-11-02
Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

I just bought a SSD for my virtual Win7 because Steam keeps telling me that my HDD is busy when downloading games/updates,
but now I wonder what might be the best way to pass it to the VM? (enabling TRIM and stuff)

I can't pass the whole SATA-Controller, so I guess passing the device using Q35 sata controller for installation and VIRTIO after installation would be the way to go? (I never managed to use VIRTIO during installation).

Any experiences doing this? Will TRIM work?
If not, anyone passing through a pcie-sata controller card? Recommendations for buying one?
I tried a Transcend TS-PDC3 Combocard last year, didn't exactly work since it always lost connection to the drives.

Offline

#1668 2014-05-07 18:51:25

powerhouse
Member
Registered: 2013-05-22
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
powerhouse wrote:

Thanks nbhs for this excellent tutorial! I've wanted to try kvm for a long time, but only recently found time to do it. I have successfully passed through a Sapphire Radeon HD 7770 using primary passthrough to a Windows 7 Pro 64 bit guest. Below some more info:
OS: Ubuntu 14.04
Guest OS: Windows 7 Pro 64 bit
M/B: Asus Sabertooth X79
CPU: Intel i7 3930K (LGA 2011)
Host OS graphics card: Nvidia Quadro 2000 with Nvidia proprietary driver
bridged network configuration
vfio driver for network and disk
Disk is a LVM drive using raw format

Here some remarks and questions:
Passmark PerformanceTest benchmark only works in SAFEMODE, else Windows BSODs. Under Xen there is no such issue. Furthermore Passmark is unable to detect the SSE extensions of the CPU (again under Xen it works fine). CPU-Z, however, does recognize the SSE extensions.
PCmark has similar issues.
Is there anything I do wrong with the configuration?

See http://www.passmark.com/forum/showthrea … r-Number-4 comment #6 and beyond.  My suspicion is that Passmark makes some platform assumptions based on the CPU family.  It would be a great data point if you could confirm or deny that with Xen.  Thanks.

Thanks for the link. I had read that thread and tried -cpu=SandyBridge, but it didn't make a difference - the same BSOD when running Passmark. I did not try any model= parameters though, nor other options like "-cpu Penryn,+popcnt,+sse4_2".

With Xen I did not have any issue running Passmark 8 - it just works. See my passmark results here. In that same thread I posted a CPU performance comparison between KVM and Xen, see here.

However, I'm a total newbie to KVM and haven't found any good source of information on it, except this tutorial/ thread!!!

Can you or someone point me to some good sources for information, in particular an overview of the zillion+1 qemu-system-x86_64 command line options with some useful explanations? (See also my next post - to keep things separate.)

Offline

#1669 2014-05-07 19:17:00

aw
Member
Registered: 2013-10-04
Posts: 921
Website

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

powerhouse wrote:
aw wrote:
powerhouse wrote:

Thanks nbhs for this excellent tutorial! I've wanted to try kvm for a long time, but only recently found time to do it. I have successfully passed through a Sapphire Radeon HD 7770 using primary passthrough to a Windows 7 Pro 64 bit guest. Below some more info:
OS: Ubuntu 14.04
Guest OS: Windows 7 Pro 64 bit
M/B: Asus Sabertooth X79
CPU: Intel i7 3930K (LGA 2011)
Host OS graphics card: Nvidia Quadro 2000 with Nvidia proprietary driver
bridged network configuration
vfio driver for network and disk
Disk is a LVM drive using raw format

Here some remarks and questions:
Passmark PerformanceTest benchmark only works in SAFEMODE, else Windows BSODs. Under Xen there is no such issue. Furthermore Passmark is unable to detect the SSE extensions of the CPU (again under Xen it works fine). CPU-Z, however, does recognize the SSE extensions.
PCmark has similar issues.
Is there anything I do wrong with the configuration?

See http://www.passmark.com/forum/showthrea … r-Number-4 comment #6 and beyond.  My suspicion is that Passmark makes some platform assumptions based on the CPU family.  It would be a great data point if you could confirm or deny that with Xen.  Thanks.

Thanks for the link. I had read that thread and tried -cpu=SandyBridge, but it didn't make a difference - the same BSOD when running Passmark. I did not try any model= parameters though, nor other options like "-cpu Penryn,+popcnt,+sse4_2".

-cpu=SandyBridge isn't going to prove anything, SandyBridge is newer than Nehalem and I'd expect the same problem.  If it's the same BSOD, -cpu=Penryn should work.

With Xen I did not have any issue running Passmark 8 - it just works. See my passmark results here.

I notice in the CPU-z screen that Xen is reporting the CPU as family 6, model D.  So that's not accurate either.  It would be really interesting if you could change that model to match your physical CPU and see if Xen gets the same BSOD.  I expect KVM would also work with -cpu SandyBridge,model=13 (aka 'D' in decimal)

In that same thread I posted a CPU performance comparison between KVM and Xen, see here.

Questionable comparison with one running in safemode.  Safemode may exclude SSE, since you note it didn't run that test.  I'm confused though that your model is also listed as D under KVM, maybe my hypothesis is completely off.  You also seem to have nested VT-x support enabled.  Some here seem to think that hurts performance (I've never tried it myself).  To disable, unload and reload the kvm-intel module with nested=0 (modprobe -r kvm-intel; modprobe kvm-intel nested=0).

However, I'm a total newbie to KVM and haven't found any good source of information on it, except this tutorial/ thread!!!

Can you or someone point me to some good sources for information, in particular an overview of the zillion+1 qemu-system-x86_64 command line options with some useful explanations? (See also my next post - to keep things separate.)


http://vfio.blogspot.com
Looking for a more open forum to discuss vfio related uses?  Try https://www.redhat.com/mailman/listinfo/vfio-users

Offline

#1670 2014-05-07 19:20:04

nbhs
Member
From: Montevideo, Uruguay
Registered: 2013-05-02
Posts: 402

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

powerhouse wrote:
aw wrote:
powerhouse wrote:

Thanks nbhs for this excellent tutorial! I've wanted to try kvm for a long time, but only recently found time to do it. I have successfully passed through a Sapphire Radeon HD 7770 using primary passthrough to a Windows 7 Pro 64 bit guest. Below some more info:
OS: Ubuntu 14.04
Guest OS: Windows 7 Pro 64 bit
M/B: Asus Sabertooth X79
CPU: Intel i7 3930K (LGA 2011)
Host OS graphics card: Nvidia Quadro 2000 with Nvidia proprietary driver
bridged network configuration
vfio driver for network and disk
Disk is a LVM drive using raw format

Here some remarks and questions:
Passmark PerformanceTest benchmark only works in SAFEMODE, else Windows BSODs. Under Xen there is no such issue. Furthermore Passmark is unable to detect the SSE extensions of the CPU (again under Xen it works fine). CPU-Z, however, does recognize the SSE extensions.
PCmark has similar issues.
Is there anything I do wrong with the configuration?

See http://www.passmark.com/forum/showthrea … r-Number-4 comment #6 and beyond.  My suspicion is that Passmark makes some platform assumptions based on the CPU family.  It would be a great data point if you could confirm or deny that with Xen.  Thanks.

Thanks for the link. I had read that thread and tried -cpu=SandyBridge, but it didn't make a difference - the same BSOD when running Passmark. I did not try any model= parameters though, nor other options like "-cpu Penryn,+popcnt,+sse4_2".

With Xen I did not have any issue running Passmark 8 - it just works. See my passmark results here. In that same thread I posted a CPU performance comparison between KVM and Xen, see here.

However, I'm a total newbie to KVM and haven't found any good source of information on it, except this tutorial/ thread!!!

Can you or someone point me to some good sources for information, in particular an overview of the zillion+1 qemu-system-x86_64 command line options with some useful explanations? (See also my next post - to keep things separate.)

I had a similar problem with passmark (or was it pc mark?) on my amd system, what solved it for me was doing this

echo "options kvm ignore_msrs=1" >> /etc/modprobe.d/kvm.conf

Offline

#1671 2014-05-07 19:43:38

powerhouse
Member
Registered: 2013-05-22
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

In addition to my previous observations and questions above, here some more:

1. I found it difficult to find useful information on the command line options for qemu-system-x86_64, or any other kvm related documentation. Are there any good sources of information (except this excellent thread)?

2. With KVM I managed to pass through an AMD Radeon HD 7770 using primary passthrough, under Xen I never succeeded with primary passthrough, only secondary (booting up using the Cirrus virtual adapter). As a result, with KVM the Windows guest boots faster.

3. With KVM, I only succeeded to pass through the AMD 7770 card, I had no success with my Nvidia Quadro 2000. Under Xen, it's almost the opposite - the Nvidia Quadro 2000 in secondary passthrough is a piece of cake, but the AMD 7770 can be a challenge (it works with Xen 4.1.2 and an old kernel, but not with Xen 4.4 and a 3.13 kernel, i.e. Ubuntu 14.04). With Xen, kernel changes can break VGA passthrough support (I've seen that with AMD cards like the Radeon HD 6450 and the Radeon HD 7770), though with the Nvidia Quadro 2000 I only once had a problem (error 22) after a Xen security update (Xen update). Also under Xen, often xm works whereas xl toolstack doesn't.

4. When trying to pass through the Nvidia Quadro 2000 (following nbhs tutorial/post #1), I got primary passthrough to a low res screen when installing Windows. Only after installing the Nvidia driver under Windows and rebooting would the boot process hang with a black screen. I wasn't able to boot Windows again, as somehow also the PCI passed through USB keyboard/mouse would not respond anymore when rebooting Windows into a recovery screen (normal start, etc...). I tried to load the VGA BIOS (I had prepared a .rom file running Xen and using GPU-Z to save the file), but it didn't work, nor did the rombar=0 option - like here:
-device vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,rombar=0,romfile=/home/user/GF106.rom \

Is there anything special to consider when passing through a Nvidia Quadro card? I vaguely remember having read something somewhere, but cannot find the info.

5. I also gave virt-manager a (short) try. Somehow most kvm information sources refer to it (or virt-install or libvirt). I had some slim hope that it might work, but I my trials were in vain. Besides, I still cannot figure out where virt-manager stores its guest config files which is quite unnerving.

6. With Xen I got either the xl or the xm toolstack, giving me enough control over the guest(s). I'm still trying to figure out how to do things in KVM, but haven't found much useful information (except references to virt-manager - see above). Is there any simple tool - preferably command line or web, or a GUI if it must be? I don't need a cloud orchestration suite, just a simple tool to get the job done?

7. Last not least, I haven't applied any patches or compiled anything. The only thing I did was loading the VGA BIOS for the AMD 7770 card, but it wasn't necessary.

Please forgive me when asking perhaps redundant questions, but I haven't found the time to read the entire thread, yet.

Offline

#1672 2014-05-07 20:00:32

powerhouse
Member
Registered: 2013-05-22
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:

-cpu=SandyBridge isn't going to prove anything, SandyBridge is newer than Nehalem and I'd expect the same problem.  If it's the same BSOD, -cpu=Penryn should work.

Thanks for that hint - I will give it a try, but it might take a few days (need to reconfigure the test drive for AMD passthrough and have little time these days). Will report back.

aw wrote:
powerhouse wrote:

With Xen I did not have any issue running Passmark 8 - it just works. See my passmark results here.

I notice in the CPU-z screen that Xen is reporting the CPU as family 6, model D.  So that's not accurate either.  It would be really interesting if you could change that model to match your physical CPU and see if Xen gets the same BSOD.  I expect KVM would also work with -cpu SandyBridge,model=13 (aka 'D' in decimal)

I will try. Where can I find an overview of these command line options?

aw wrote:
powerhouse wrote:

In that same thread I posted a CPU performance comparison between KVM and Xen, see here.

Questionable comparison with one running in safemode.  Safemode may exclude SSE, since you note it didn't run that test.  I'm confused though that your model is also listed as D under KVM, maybe my hypothesis is completely off.  You also seem to have nested VT-x support enabled.  Some here seem to think that hurts performance (I've never tried it myself).  To disable, unload and reload the kvm-intel module with nested=0 (modprobe -r kvm-intel; modprobe kvm-intel nested=0).

SSE was definitely excluded, it doesn't show up in the graph. For that reason there is no CPU performance summary under the KVM column. By the way, this Passmark CPU test (in SAFEMODE) was run using the -cpu=host option. I was under the impression that that option would pass through the host CPU capabilities? As I said I have absolutely no clew as to which options there are and what they do (and --help doesn't seem to list all of them).
I'll give your suggestion a try - thanks!

Offline

#1673 2014-05-07 20:07:25

powerhouse
Member
Registered: 2013-05-22
Posts: 15

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

nbhs wrote:

I had a similar problem with passmark (or was it pc mark?) on my amd system, what solved it for me was doing this

echo "options kvm ignore_msrs=1" >> /etc/modprobe.d/kvm.conf

Thanks for the reply, and for your great tutorial! I wouldn't have made it without.

I remember that I've seen this option but never tried it since you said you are using an AMD machine. What does this option do?

By the way, what other configuration options are there for the kvm.conf file - any link to further documentation?

Offline

#1674 2014-05-08 07:52:25

anickname
Member
Registered: 2014-01-07
Posts: 23

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

aw wrote:
aw wrote:
anickname wrote:

@aw

Regarding "hv-time" problem do you know if will it be fixed in 3.15 ?

https://bugzilla.kernel.org/show_bug.cgi?id=73721

I don't know, but I just poked one of the KVM maintainers about it and he's going to investigate.

Apparently it's already fixed in current 3.15-rc and Paolo plans send it for stable as well.

I applied the patch to 3.14.1 and now everything works good. The cpu load dropped from ~60-65% to 40-45% for an
idle windows 8.1 vm (with and without hv-time). Also the vm (when using hv-time) seems to have a more "native" feeling.

--- a/arch/x86/kvm/x86.c    2014-05-07 21:15:22.099260195 +0300
+++ b/arch/x86/kvm/x86.c    2014-05-07 21:19:56.712586783 +0300
@@ -1111,7 +1111,6 @@
 {
    struct timespec ts;

-   WARN_ON(preemptible());
    ktime_get_ts(&ts);
    monotonic_to_bootbased(&ts);
    return timespec_to_ns(&ts);

Thanks.

Offline

#1675 2014-05-08 22:08:20

th3flyboy
Member
Registered: 2014-03-18
Posts: 1

Re: KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9

belliash wrote:

I have managed to pass through my GeForce GTX 770M into KVM.
The GPU showed Error 10 for the very beginning and after I have installed latest ForceWare drivers, it showed error 43.
However if i disable and re-enable the card in Device Manager it showed that the device is working correctly.
Unfortunately as a second device and I was unable to boot Windows with nvidia card set as primary due to error 43, which i come across every time i reboot the VM.
All the time i need to disable and re-enable the dGPU in order to make Windows marking it as working.


I am not sure if this is connected with nVidia Optimus, or just GTX 770M. I have found that some of you managed to get GF 7xx working, while others not.
Also there is no such problem with ATI cards. Due to that i think that more people having notebooks should give it a try, especially with older GeForce cards and with mobile ATI cards.

As the prove I want to show you a screenshot:

http://wstaw.org/m/2014/05/04/snapshot2_1_png_750x750_q85.jpg


Could you please put up some detailed instructions on how you did that? I tried a while back with my 780M and wasn't even able to boot the VM when the device was attached to the VM.

Offline

Board footer

Powered by FluxBB