890FXA-GD70 IOMMU Causes Freezing

MichaelManley · 2013-05-11 06:05:52

When IOMMU is enabled the following shows up on my screen

[ 0.028164] [Firmware Bug]: AMD-Vi: IOAPIC[24] not in IVRS table
[ 0.028170] [Firmware Bug]: AMD-Vi: No southbridge IOAPIC found in IVRS table
[ 0.028174] AMD-Vi: Disabling interrupt remapping due to BIOS bug(s)

In which the machine freezes. I read 1.8 BIOS doesn't have these issues but the problem is I have an AMD-FX CPU, which is more then likely not going to function on 1.8. Is there any way I can get IOMMU working as I need it for PCI Passthrough.

resuxunil · 2013-05-20 13:48:39

Yes, the last stable firmware (BIOS) for the 890FXA-GD70 (MS-7640) mainboard – that does not cause IOMMU problems – is version 1.8.
I just updated from this version to 1F0 aka 1.15, released on 31st of October 2012, to see if they fixed it. But as it turns out the bug is still around.

I successfully booted into Gentoo Linux (kernel 3.9.2) using the kernel command line parameter "iommu=soft" which also seems to ignore the enabled IOMMU in the BIOS, but which at leasts boots up and is usable. Before that I tried with "iommu=off" which printed some iommu errors while booting and had my keyboard and mouse disabled, so I had no means to do anything (like looking at the logs, dmesg). I had to shut the computer down using the power button, indicating that the ACPI power button event worked as expected.

I also tried "amd_iommu=isolate", "amd_iommu_size=128M", "acpi=rsdt", "irqpoll", "acpi_apic_instance=2" and "iommu=biomerge", all with no success.

Workaround: Using "iommu=soft" the system is usable even with IOMMU enabled in the BIOS setup. The IOMMU seems to not be used by Linux though, but maybe another workaround/bugfix will be successful eventually.

To me it looks as if the ACPI table is somehow broken. A user DSDT (or whatever is the right ACPI table) may be able to fix it. Extracting the working ACPI table from BIOS 1.8 and comparing it to 1.15 may give a clue that can lead to a solution.
Does anyone know how this is done?

Last edited by resuxunil (2013-05-20 19:02:49)

nbhs · 2013-05-25 17:48:21

Using kernel 3.10-rc2 or a patched kernel 3.9 try passing this to grub

ivrs_ioapic[24]=00:14.0

Kernel 3.9 patch:

amd_iommu_fixes.patch.tar.gz

Last edited by nbhs (2013-05-25 17:49:53)

resuxunil · 2013-05-25 21:22:54

nbhs wrote:

ivrs_ioapic[24]=00:14.0

00:14.0 is the SMBus. I’m compiling kernel 3.10-rc2 and hope that it works, but I don’t understand what the SMBus has got to do with the I/O APIC.

# lspci -nn
00:00.0 Host bridge [0600]: Advanced Micro Devices [AMD] nee ATI RD890 Northbridge only single slot PCI-e GFX Hydra part [1002:5a11] (rev 02)
00:00.2 IOMMU [0806]: Advanced Micro Devices [AMD] nee ATI RD990 I/O Memory Management Unit (IOMMU) [1002:5a23]
00:02.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port B) [1002:5a16]
00:04.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port D) [1002:5a18]
00:05.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port E) [1002:5a19]
00:06.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port F) [1002:5a1a]
00:09.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (PCI express gpp port H) [1002:5a1c]
00:0b.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (NB-SB link) [1002:5a1f]
00:0d.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI RD890 PCI to PCI bridge (external gfx1 port B) [1002:5a1e]
00:11.0 SATA controller [0106]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] [1002:4391] (rev 40)
00:12.0 USB controller [0c03]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
00:12.2 USB controller [0c03]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
00:13.0 USB controller [0c03]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
00:13.2 USB controller [0c03]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
00:14.0 SMBus [0c05]: Advanced Micro Devices [AMD] nee ATI SBx00 SMBus Controller [1002:4385] (rev 41)
00:14.2 Audio device [0403]: Advanced Micro Devices [AMD] nee ATI SBx00 Azalia (Intel HDA) [1002:4383] (rev 40)
00:14.3 ISA bridge [0601]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 LPC host controller [1002:439d] (rev 40)
00:14.4 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI SBx00 PCI to PCI Bridge [1002:4384] (rev 40)
00:14.5 USB controller [0c03]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI2 Controller [1002:4399]
00:15.0 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0) [1002:43a0]
00:15.1 PCI bridge [0604]: Advanced Micro Devices [AMD] nee ATI SB700/SB800/SB900 PCI to PCI bridge (PCIE port 1) [1002:43a1]
00:16.0 USB controller [0c03]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB OHCI0 Controller [1002:4397]
00:16.2 USB controller [0c03]: Advanced Micro Devices [AMD] nee ATI SB7x0/SB8x0/SB9x0 USB EHCI Controller [1002:4396]
00:18.0 Host bridge [0600]: Advanced Micro Devices [AMD] Family 10h Processor HyperTransport Configuration [1022:1200]
00:18.1 Host bridge [0600]: Advanced Micro Devices [AMD] Family 10h Processor Address Map [1022:1201]
00:18.2 Host bridge [0600]: Advanced Micro Devices [AMD] Family 10h Processor DRAM Controller [1022:1202]
00:18.3 Host bridge [0600]: Advanced Micro Devices [AMD] Family 10h Processor Miscellaneous Control [1022:1203]
00:18.4 Host bridge [0600]: Advanced Micro Devices [AMD] Family 10h Processor Link Control [1022:1204]
01:00.0 SATA controller [0106]: JMicron Technology Corp. JMB363 SATA/IDE Controller [197b:2363] (rev 03)
01:00.1 IDE interface [0101]: JMicron Technology Corp. JMB363 SATA/IDE Controller [197b:2363] (rev 03)
04:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev ff)
05:00.0 USB controller [0c03]: NEC Corporation uPD720200 USB 3.0 Host Controller [1033:0194] (rev 03)
06:00.0 USB controller [0c03]: NEC Corporation uPD720200 USB 3.0 Host Controller [1033:0194] (rev 03)
07:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller [10ec:8168] (rev 03)
08:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168 PCI Express Gigabit Ethernet controller [10ec:8168] (rev 03)
09:00.0 FireWire (IEEE 1394) [0c00]: VIA Technologies, Inc. VT6315 Series Firewire Controller [1106:3403]
0a:00.0 VGA compatible controller [0300]: Advanced Micro Devices [AMD] nee ATI Turks XT [Radeon HD 6670] [1002:6758]
0a:00.1 Audio device [0403]: Advanced Micro Devices [AMD] nee ATI Turks/Whistler HDMI Audio [Radeon HD 6000 Series] [1002:aa90]

Edit: I've tried booting this kernel now with the mentioned kernel command line added to my GRUB 2 config. It freezes. After adding "iommu=soft" it booted like 3.9.2/.3 without IOMMU support...

Last edited by resuxunil (2013-05-25 22:23:02)

nbhs · 2013-05-25 23:55:00

Booting on my board with broken tables (like yours apparently) produces this warning:

[    0.297481] [Firmware Bug]: AMD-Vi: IOAPIC[9] not in IVRS table
[    0.297485] [Firmware Bug]: AMD-Vi: IOAPIC[10] not in IVRS table
[    0.297487] [Firmware Bug]: AMD-Vi: No southbridge IOAPIC found in IVRS table
[    0.297490] AMD-Vi: Disabling interrupt remapping due to BIOS Bug(s)

IOAPIC[9] correspons to southbridge ioapic and IOAPIC[10] northbridge ioapic

Booting with amd_iommu_dump

[    0.297434] AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
[    0.297435] AMD-Vi:        mmio-addr: 00000000feb20000
[    0.297444] AMD-Vi:   DEV_SELECT_RANGE_START  devid: 00:00.0 flags: 00
[    0.297445] AMD-Vi:   DEV_RANGE_END           devid: 00:00.2
[    0.297446] AMD-Vi:   DEV_SELECT                      devid: 00:02.0 flags: 00
[    0.297447] AMD-Vi:   DEV_SELECT_RANGE_START  devid: 01:00.0 flags: 00
[    0.297448] AMD-Vi:   DEV_RANGE_END           devid: 01:00.1
[    0.297449] AMD-Vi:   DEV_SELECT                      devid: 00:04.0 flags: 00
[    0.297450] AMD-Vi:   DEV_SELECT                      devid: 02:00.0 flags: 00
[    0.297450] AMD-Vi:   DEV_SELECT                      devid: 00:05.0 flags: 00
[    0.297451] AMD-Vi:   DEV_SELECT                      devid: 03:00.0 flags: 00
[    0.297452] AMD-Vi:   DEV_SELECT                      devid: 00:06.0 flags: 00
[    0.297453] AMD-Vi:   DEV_SELECT                      devid: 04:00.0 flags: 00
[    0.297454] AMD-Vi:   DEV_SELECT                      devid: 00:07.0 flags: 00
[    0.297454] AMD-Vi:   DEV_SELECT                      devid: 05:00.0 flags: 00
[    0.297455] AMD-Vi:   DEV_SELECT                      devid: 00:09.0 flags: 00
[    0.297456] AMD-Vi:   DEV_SELECT                      devid: 06:00.0 flags: 00
[    0.297457] AMD-Vi:   DEV_SELECT                      devid: 00:0b.0 flags: 00
[    0.297458] AMD-Vi:   DEV_SELECT_RANGE_START  devid: 07:00.0 flags: 00
[    0.297458] AMD-Vi:   DEV_RANGE_END           devid: 07:00.1
[    0.297459] AMD-Vi:   DEV_SELECT                      devid: 00:11.0 flags: 00
[    0.297460] AMD-Vi:   DEV_SELECT_RANGE_START  devid: 00:12.0 flags: 00
[    0.297461] AMD-Vi:   DEV_RANGE_END           devid: 00:12.2
[    0.297462] AMD-Vi:   DEV_SELECT_RANGE_START  devid: 00:13.0 flags: 00
[    0.297462] AMD-Vi:   DEV_RANGE_END           devid: 00:13.2
[    0.297463] AMD-Vi:   DEV_SELECT                      devid: 00:14.0 flags: d7
[    0.297464] AMD-Vi:   DEV_SELECT                      devid: 00:14.2 flags: 00
[    0.297465] AMD-Vi:   DEV_SELECT                      devid: 00:14.3 flags: 00
[    0.297466] AMD-Vi:   DEV_SELECT                      devid: 00:14.4 flags: 00
[    0.297467] AMD-Vi:   DEV_ALIAS_RANGE                 devid: 08:00.0 flags: 00 devid_to: 00:14.4
[    0.297468] AMD-Vi:   DEV_RANGE_END           devid: 08:1f.7
[    0.297474] AMD-Vi:   DEV_SELECT                      devid: 00:14.5 flags: 00
[    0.297475] AMD-Vi:   DEV_SELECT_RANGE_START  devid: 00:16.0 flags: 00
[    0.297476] AMD-Vi:   DEV_RANGE_END           devid: 00:16.2
[    0.297477] AMD-Vi:   DEV_SPECIAL(IOAPIC[0])         devid: 00:14.0
[    0.297478] AMD-Vi:   DEV_SPECIAL(HPET[0])           devid: 00:14.0
[    0.297479] AMD-Vi:   DEV_SPECIAL(IOAPIC[255])               devid: 00:00.1
[    0.297481] [Firmware Bug]: AMD-Vi: IOAPIC[9] not in IVRS table
[    0.297485] [Firmware Bug]: AMD-Vi: IOAPIC[10] not in IVRS table
[    0.297487] [Firmware Bug]: AMD-Vi: No southbridge IOAPIC found in IVRS table
[    0.297490] AMD-Vi: Disabling interrupt remapping due to BIOS Bug(s)

The ivrs table is wrong, it points to non existant IOAPIC[0] and IOAPIC[255], so to override this i use this commandline in grub:

ivrs_ioapic[9]=00:14.0 ivrs_ioapic[10]=00:00.1

Booting with it:

[    1.750889] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
[    1.750890] AMD-Vi: Interrupt remapping enabled
[    1.750992] AMD-Vi: Initialized for Passthrough Mode

lspci

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (external gfx0 port B) (rev 02)
00:00.2 IOMMU: Advanced Micro Devices, Inc. [AMD/ATI] RD990 I/O Memory Management Unit (IOMMU)
00:02.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port B)
00:04.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port D)
00:05.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port E)
00:06.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port F)
00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port G)
00:09.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (PCI express gpp port H)
00:0b.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890 PCI to PCI bridge (NB-SB link)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 42)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA) (rev 40)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:16.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 0
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 1
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 2
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 3
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 4
00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h Processor Function 5
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cedar [Radeon HD 5000/6000/7350 Series]
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300 Series]
02:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
03:00.0 Ethernet controller: Intel Corporation 82583V Gigabit Network Connection
04:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
05:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
06:00.0 USB controller: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller
07:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Cayman PRO [Radeon HD 6950]
07:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Cayman/Antilles HDMI Audio [Radeon HD 6900 Series]

Last edited by nbhs (2013-05-26 00:07:25)

resuxunil · 2013-05-26 09:19:24

Strange enough, my system stalls before printing a message about the IOMMU… It even stays in the early printk screen, so it doesn’t go as far as setting up a console.

I used "amd_iommu_debug" one time and it displays a lot of information, then stalls. The last two lines tell me this: HPET[0] is device 00:00.1, IOAPIC[7] is device 00:14.0. But I cannot scroll up to see all the messages, nor have I set up a serial console to copy&paste its output here.

I don’t know what to do next…

nbhs · 2013-05-26 09:33:01

HPET[0] should be 00:14.0 i believe try:

ivrs_hpet[0]=00:14.0

resuxunil · 2013-05-26 09:53:28

I now get this:

AMD-Vi: Command-line override present for HPET id 0 - ignoring

The same thing happens if I want to override the IOAPIC setting: apparently it is ignored and stalls.

It comes to my mind that this error on my system may be different to the error reported by the original poster…

I leave it at "iommu=soft" for now to keep my system running.

nbhs · 2013-05-26 09:57:48

Well you can hack the kernel to override it, take a look at drivers/iommu/amd_iommu_init.c

resuxunil · 2013-05-26 11:57:11

I patched the function add_special_device like this (only the very part I changed):

	list_for_each_entry(entry, list, list) {
		if (!(entry->id == id && entry->cmd_line))
			continue;

		pr_info("AMD-Vi: Command-line override present for %s id %d\n",
			type == IVHD_SPECIAL_IOAPIC ? "IOAPIC" : "HPET", id);

/*		return 0; */
	}

So now if I use the override it doesn’t return anymore but continues inside the function.

It still freezes. The last message that is displayed is the very override pr_info as in the code above. I used "ivrs_hpet[0]=00:14.0" as the command line override argument.

nbhs · 2013-05-26 12:31:46

i was thinking more like this:

--- a/amd_iommu_init.c	2013-04-29 00:36:01.000000000 +0000
+++ b/amd_iommu_init.c	2013-05-26 09:33:11.452098000 +0000
@@ -751,6 +751,10 @@ static void __init set_device_exclusion_
 	}
 }
 
+/* SB IOAPIC is always on this device in AMD systems */
+#define IOAPIC_SB_DEVID		((0x00 << 8) | PCI_DEVFN(0x14, 0))
+#define IOAPIC_NB_DEVID		((0x00 << 8) | PCI_DEVFN(0x00, 1))
+
 /*
  * Takes a pointer to an AMD IOMMU entry in the ACPI table and
  * initializes the hardware and our data structures with it.
@@ -922,6 +926,10 @@ static int __init init_iommu_from_acpi(s
 			else
 				var = "UNKNOWN";
 
+			if (type == IVHD_SPECIAL_HPET ){
+				devid = IOAPIC_SB_DEVID;
+			}
+
 			DUMP_printk("  DEV_SPECIAL(%s[%d])\t\tdevid: %02x:%02x.%x\n",
 				    var, (int)handle,
 				    PCI_BUS(devid),
@@ -1633,9 +1641,6 @@ static void __init free_on_init_error(vo
 #endif
 }
 
-/* SB IOAPIC is always on this device in AMD systems */
-#define IOAPIC_SB_DEVID		((0x00 << 8) | PCI_DEVFN(0x14, 0))
-
 static bool __init check_ioapic_information(void)
 {
 	bool ret, has_sb_ioapic;

this is an ugly hack but w/e, you could do the the same for the 2 ioapics

EDIT: Perhaps its hanging on these lines?

			set_dev_entry_from_acpi(iommu, devid, e->flags, 0);
			ret = add_special_device(type, handle, devid, false);
			if (ret)
				return ret;
			break;

OR

static int __init add_early_maps(void)
{
	int i, ret;

	for (i = 0; i < early_ioapic_map_size; ++i) {
		ret = add_special_device(IVHD_SPECIAL_IOAPIC,
					 early_ioapic_map[i].id,
					 early_ioapic_map[i].devid,
					 early_ioapic_map[i].cmd_line);
		if (ret)
			return ret;
	}

	for (i = 0; i < early_hpet_map_size; ++i) {
		ret = add_special_device(IVHD_SPECIAL_HPET,
					 early_hpet_map[i].id,
					 early_hpet_map[i].devid,
					 early_hpet_map[i].cmd_line);
		if (ret)
			return ret;
	}

	return 0;
}

Last edited by nbhs (2013-05-26 13:36:12)

pam · 2013-05-27 03:53:25

Same thing, no luck with IOMMU with kernels 3.8/9.
I just disable IOMMU from bios.

resuxunil · 2015-01-17 21:24:45

Long time no see. Very long time…

I’ve been enjoying my life, working, and just using my computer in my scarce spare time: BIOS 1.15 (1.F0) and Linux without IOMMU (using iommu=soft).
During X-Mas holiday I had a bit more spare time and decided to try BIOS 1.80 again. It boots and initializes the IOMMU, without the necessity of using a iommu kernel paramenter, obviously with the IOMMU enabled in the BIOS.

I have to confess I’d rather stay with BIOS 1.80 than again update just to find it’s still not possible to work around this issue. BUT, I’ve booted kernel 3.18.2-gentoo with kernel parameter amd_iommu_dump=1 and it gives me the following information:

AMD-Vi: device: 00:00.2 cap: 0040 seg: 0 flags: 3e info 1300
AMD-Vi:        mmio-addr: 00000000f6000000
AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END		 devid: 00:00.2
AMD-Vi:   DEV_SELECT			 devid: 00:02.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 09:00.0 flags: 00
AMD-Vi:   DEV_RANGE_END		 devid: 09:00.1
AMD-Vi:   DEV_SELECT			 devid: 00:04.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 08:00.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:05.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 07:00.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:06.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 06:00.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:09.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 05:00.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:0b.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 04:00.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:11.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:12.0 flags: 00
AMD-Vi:   DEV_RANGE_END		 devid: 00:12.2
AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:13.0 flags: 00
AMD-Vi:   DEV_RANGE_END		 devid: 00:13.2
AMD-Vi:   DEV_SELECT			 devid: 00:14.0 flags: d7
AMD-Vi:   DEV_SELECT			 devid: 00:14.2 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:14.3 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:14.4 flags: 00
AMD-Vi:   DEV_ALIAS_RANGE		 devid: 03:00.0 flags: 00 devid_to: 00:14.4
AMD-Vi:   DEV_RANGE_END		 devid: 03:1f.7
AMD-Vi:   DEV_SELECT			 devid: 00:14.5 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:15.0 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 00:15.1 flags: 00
AMD-Vi:   DEV_SELECT			 devid: 01:00.0 flags: 00
AMD-Vi:   DEV_SELECT_RANGE_START	 devid: 00:16.0 flags: 00
AMD-Vi:   DEV_RANGE_END		 devid: 00:16.2
AMD-Vi:   DEV_SPECIAL(HPET[0])		devid: 00:14.0
AMD-Vi:   DEV_SPECIAL(IOAPIC[7])		devid: 00:00.1
AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
pci 0000:00:00.2: irq 27 for MSI/MSI-X
AMD-Vi: Lazy IO/TLB flushing enabled

So, I’d have to use ivrs_hpet[0]=00:14.0 and ivrs_ioapic[7]=00:00.1, right? I’m uncertain if I should risk it to update my BIOS to 1.15 once again only to find trouble…

Anyway, maybe I will, and I’ll keep you informed.
BTW, others have the same problem: ASUS Crosshair IV Formula (Ubuntu), ASUS M4A89TD PRO USB3 (Fedora)… an my MSI 890FXA-GD70… a problem shared is a problem halved, right?

What makes me wonder is if the HPET and the IOAPIC index numbers and device IDs stay the same throu BIOS updates. Hopefully, otherwise these numbers are null and void. That said, why isn’t there a convenient way to probe for these settings, in case the IVRS table is invalid – apparently every board has different settings, so probing for them seems to be the cleanest way to solve this issue on buggy BIOSes. But I guess that would be something for a kernel developer…

And one more thing: does the kernel still have to be patched in order to not ignore those IVRS override settings? Because this LKML post from January 2015 still shows the line

AMD-Vi: Command-line override present for HPET id 0 - ignoring

…and as a side note: why would ivrs_hpet and ivrs_ioapic, whatever index they are set to, be using the same device? In this case the author of this post used ivrs_ioapic[6]=00:14.0 and ivrs_hpet[0]=00:14.0, but that doesn’t make any sense to me. (It too doesn’t make any sense to use overrides at all if they are ignored afterwards.)

Anyway, I’d be happy to hear some more advice on this issue. Thanks.

Last edited by resuxunil (2015-01-17 21:26:23)

resuxunil · 2015-04-01 14:39:56

I just wanted to report that I found a couple of hopefully working patches:

https://lkml.org/lkml/2014/9/5/325 by Joerg Roedel, 5 Sep 2014 (as reaction to the one from Su Friendly, maybe simpler)
https://lkml.org/lkml/2014/9/5/248 by Su Friendy, 5 Sep 2014
http://lists.xen.org/archives/html/xen- … 01552.html by Jan Beulich, 13 Sep 2013 (for Xen, not for the mainstream Linux kernel)

I didn’t test them yet, but I plan to. The plan is to patch the kernel and see if it uses the overrides. Then I will flash the BIOS once again and see if it still works then. Hopefully it will, otherwise I'll presumably finally give up…

I checked my running system. I currently run gentoo-sources-3.19.3 and it still ignores the command line override i.e. "early mapped IOPIC/HPET devid"…

UPDATE/CORRECTION: The patch by Joerg Roedel is already applied to gentoo-sources-3.19.3. So it must have been merged into the main kernel at some point after September 2014. In my defense, it seems noteworthy that the message "AMD-Vi: Command-line override present for HPET id 0 - ignoring" is still present, suggesting that it is still ignored…

Last edited by resuxunil (2015-04-01 15:11:01)

zwa · 2015-05-02 19:38:01

Hi, I also have board 890FXA-GD70. I just booted the archlinux-2015.04.01-dual.iso and have the same problem.
Is there any solution for this that does not involve kernel patching?

Btw, sometimes the pci-e 4 slot (where I have a wifi card) is not recognized in windows 7, and a reboot is necessary in order to get the card recognized again. IIRC this also happened with another pci-e wifi card and I also have had similar problems with the pci slot. However, this problems get solved either by a reboot or by clearing the cmos.
So I guess we all bought a buggy motherboard.

resuxunil · 2015-07-12 08:44:27

I’m sorry to report that I broke my mainboard. At least I think I did. I had a USB KVM connected, which routed my USB keyboard and mouse to either the PC or the nettop and one day the KVM made a very load noise (like an explosion of a capacitor or such) and stopped working. I opened the USB KVM up but I cannot see any visible defect… strange. Anyway, after that my PC randomly restarted (between 30 minutes after turning it on and a couple of hours). It is no longer productive to use it. I don’t know if parts of the mainboard or other parts (PCI-Express cards, PSU) are affected, but I highly assume it is the mainboard, since the USB KVM was connected to one of the USB 2.0 ports on the mainboard.

In other words: I’m out.

I’ll look for a replacement in the near future.

I wish you guys good luck!

Last edited by resuxunil (2015-07-12 08:47:18)

FishB8 · 2015-07-13 03:09:41

Use BIOS ver. 1.8

Add the following to the kernel boot command line:
iommu=1 ivrs_ioapic[6]=00:14.0

Note that if you use an Nvidia video card with their proprietary driver, it will not work correctly with the IOMMU enabled. It will work if you add an additional option "iommu=pt" but that kinda defeats the purpose.

resuxunil · 2015-07-13 15:38:13

FishB8 wrote:

Use BIOS ver. 1.8

I don’t really get your point here.

If I use BIOS 1.8 the IOMMU works, yes.
It doesn’t work with version 1.9 and newer, including the latest update 1.F=1.15.

Since BIOS version 1.9 AM3+ CPUs are supported. Owners of FX-CPUs (AM3+) have no choice and cannot use BIOS 1.8, so they cannot get a working IOMMU setup.

See also: http://www.msi.com/support/mb/890FXAGD7 … upport-cpu

FishB8 · 2015-07-13 23:29:54

resuxunil wrote:

FishB8 wrote:
Use BIOS ver. 1.8
I don’t really get your point here.
If I use BIOS 1.8 the IOMMU works, yes.
It doesn’t work with version 1.9 and newer, including the latest update 1.F=1.15.
Since BIOS version 1.9 AM3+ CPUs are supported. Owners of FX-CPUs (AM3+) have no choice and cannot use BIOS 1.8, so they cannot get a working IOMMU setup.
See also: http://www.msi.com/support/mb/890FXAGD7 … upport-cpu

Yes, I think it's too broken in anything other than 1.8 to get a working IOMMU. I tried quite a few things and eventually downgraded to 1.8, but I have a Phenom II X6 so that works.

I didn't see anything explicitly stating that an AM3+ CPU was being used. My point is that if you can, downgrade to 1.8. Nothing significant is really gained from higher versions aside from CPU support. The clowns that write the BIOS seem to break 6 things for every single "fix" in newer BIOS.

resuxunil · 2015-07-14 17:22:49

FishB8 wrote:

Yes, I think it's too broken in anything other than 1.8 to get a working IOMMU. I tried quite a few things and eventually downgraded to 1.8, but I have a Phenom II X6 so that works.

Yes, I did the same thing. Only, I was eager to find a workaround. At least for me it did not work around, apparently.

FishB8 wrote:

I didn't see anything explicitly stating that an AM3+ CPU was being used. My point is that if you can, downgrade to 1.8. Nothing significant is really gained from higher versions aside from CPU support. The clowns that write the BIOS seem to break 6 things for every single "fix" in newer BIOS.

Well, even I was thinking at one time if it may be wise to upgrade to an FX CPU. That would get me once again multi-core (8 maybe?!?) and some additional CPU features, like AVX and FMA.

The question was of course "do I really need this?" And the answer was, in my case, no. But there are people who did upgrade and there are likely some who got their mainboard with a newer CPU to start with. Those are now stuck with a non-working IOMMU under Linux.

I thought to myself, since I can run both the older version 1.8 BIOS and the newer one, maybe I can figure out the correct configuration and thus not only help myself but also others. I failed. Sorry.

As for the BIOS "clowns" – I wouldn't dare call them that. I don't know what the problem is, but under Windows it all seems to work fine. Maybe they just didn't test under Linux. If I remember correctly, the bug was reported to MSI, but they didn't fix it. In that sense you're right – they definitely didn't take this bug seriously!

In defense for MSI I must say that it may be the wrong type of mainboard. After I bought it I watched a video of the presentation of this mainboard on YouTube (which I cannot find right now). The target for it was clearly gamers. The main feature of the MSI 890FXA-GD70 is that it features five full-speed PCIe-2.0-x16 expansion slots. They were, according to the presentation, intended for the use of graphics cards, even if only 2 are also electrically x16 and 2 electrically only x8 (and the last one electrically only x4). Graphics for games. For gamers. And gamers who with 99.9% certainty run Windows wouldn't need IOMMU on Linux anyway…

Arch Linux

#1 2013-05-11 06:05:52

890FXA-GD70 IOMMU Causes Freezing

#2 2013-05-20 13:48:39

Re: 890FXA-GD70 IOMMU Causes Freezing

#3 2013-05-25 17:48:21

Re: 890FXA-GD70 IOMMU Causes Freezing

#4 2013-05-25 21:22:54

Re: 890FXA-GD70 IOMMU Causes Freezing

#5 2013-05-25 23:55:00

Re: 890FXA-GD70 IOMMU Causes Freezing

#6 2013-05-26 09:19:24

Re: 890FXA-GD70 IOMMU Causes Freezing

#7 2013-05-26 09:33:01

Re: 890FXA-GD70 IOMMU Causes Freezing

#8 2013-05-26 09:53:28

Re: 890FXA-GD70 IOMMU Causes Freezing

#9 2013-05-26 09:57:48

Re: 890FXA-GD70 IOMMU Causes Freezing

#10 2013-05-26 11:57:11

Re: 890FXA-GD70 IOMMU Causes Freezing

#11 2013-05-26 12:31:46

Re: 890FXA-GD70 IOMMU Causes Freezing

#12 2013-05-27 03:53:25

Re: 890FXA-GD70 IOMMU Causes Freezing

#13 2015-01-17 21:24:45

Re: 890FXA-GD70 IOMMU Causes Freezing

#14 2015-04-01 14:39:56

Re: 890FXA-GD70 IOMMU Causes Freezing

#15 2015-05-02 19:38:01

Re: 890FXA-GD70 IOMMU Causes Freezing

#16 2015-07-12 08:44:27

Re: 890FXA-GD70 IOMMU Causes Freezing

#17 2015-07-13 03:09:41

Re: 890FXA-GD70 IOMMU Causes Freezing

#18 2015-07-13 15:38:13

Re: 890FXA-GD70 IOMMU Causes Freezing

#19 2015-07-13 23:29:54

Re: 890FXA-GD70 IOMMU Causes Freezing

#20 2015-07-14 17:22:49

Re: 890FXA-GD70 IOMMU Causes Freezing

Board footer