You are not logged in.

#1 2017-10-19 16:17:28

fede
Member
Registered: 2007-06-29
Posts: 126

[SOLVED] All USB devices powered off by error from inside KVM guest

Hello,

I've been experiencing a problem where all my USB devices are suddenly powered off after certain actions inside a KVM Windows 10 Guest (particularly by using certain web pages inside Google Chrome within this guest). I believe the logs show that something is effectively not working between the kernel an my usb devices after this point, but I don't know where to go from here.

Any suggestions on how to avoid this situation? Do you think I should report this as a bug somewhere? Thanks!

Oct 19 09:18:26 luminous kernel: uurb->buffer_length is too short 6144 vs 0
Oct 19 09:18:26 luminous kernel: ------------[ cut here ]------------
Oct 19 09:18:26 luminous kernel: WARNING: CPU: 3 PID: 31961 at drivers/usb/core/devio.c:1583 proc_do_submiturb+0x993/0xd60 [usbcore]
Oct 19 09:18:26 luminous kernel: Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_conntrack_netbios_ns nf_conntrack_broadcas
Oct 19 09:18:26 luminous kernel:  snd_usbmidi_lib videobuf2_v4l2 snd_rawmidi hid_logitech_dj videobuf2_core snd_seq_device iTCO_wdt iTCO_vendor_support mxm_wmi
Oct 19 09:18:26 luminous kernel:  media intel_lpss tpm_infineon acpi_als tpm_tis kfifo_buf tpm_tis_core industrialio tpm button acpi_pad sch_fq_codel crypto_us
Oct 19 09:18:26 luminous kernel: CPU: 3 PID: 31961 Comm: qemu-system-x86 Tainted: G         C      4.13.7-1-ARCH #1
Oct 19 09:18:26 luminous kernel: Hardware name: Gigabyte Technology Co., Ltd. Z270X-Gaming 5/Z270X-Gaming 5, BIOS F4 11/18/2016
Oct 19 09:18:26 luminous kernel: task: ffff8d37a2900000 task.stack: ffffa4cf06b68000
Oct 19 09:18:26 luminous kernel: RIP: 0010:proc_do_submiturb+0x993/0xd60 [usbcore]
Oct 19 09:18:26 luminous kernel: RSP: 0018:ffffa4cf06b6bd38 EFLAGS: 00010282
Oct 19 09:18:26 luminous kernel: RAX: 000000000000002a RBX: ffff8d37bc3b60c0 RCX: ffffffffb7a55328
Oct 19 09:18:26 luminous kernel: RDX: 0000000000000000 RSI: 0000000000000096 RDI: 0000000000000283
Oct 19 09:18:26 luminous kernel: RBP: ffffa4cf06b6bdc0 R08: 00000000000003c3 R09: ffffffffb7ca68e0
Oct 19 09:18:26 luminous kernel: R10: 0000000000000200 R11: 0000000000000000 R12: 0000000000000001
Oct 19 09:18:26 luminous kernel: R13: ffff8d37b8781f60 R14: 00007fef3b435c38 R15: ffffa4cf06b6be00
Oct 19 09:18:26 luminous kernel: FS:  00007fef4cc77340(0000) GS:ffff8d37cecc0000(0000) knlGS:0000000000000000
Oct 19 09:18:26 luminous kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 19 09:18:26 luminous kernel: CR2: 00007fed35a4f000 CR3: 0000000320a1e000 CR4: 00000000003426e0
Oct 19 09:18:26 luminous kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Oct 19 09:18:26 luminous kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Oct 19 09:18:26 luminous kernel: Call Trace:
Oct 19 09:18:26 luminous kernel:  ? compat_poll_select_copy_remaining+0x120/0x120
Oct 19 09:18:26 luminous kernel:  usbdev_do_ioctl+0xa34/0x1170 [usbcore]
Oct 19 09:18:26 luminous kernel:  ? compat_poll_select_copy_remaining+0x120/0x120
Oct 19 09:18:26 luminous kernel:  usbdev_ioctl+0xe/0x20 [usbcore]
Oct 19 09:18:26 luminous kernel:  do_vfs_ioctl+0xa5/0x600
Oct 19 09:18:26 luminous kernel:  ? __fget+0x6e/0x90
Oct 19 09:18:26 luminous kernel:  SyS_ioctl+0x79/0x90
Oct 19 09:18:26 luminous kernel:  entry_SYSCALL_64_fastpath+0x1a/0xa5
Oct 19 09:18:26 luminous kernel: RIP: 0033:0x7fef45a45157
Oct 19 09:18:26 luminous kernel: RSP: 002b:00007ffc4f45f2b8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Oct 19 09:18:26 luminous kernel: RAX: ffffffffffffffda RBX: 00000000000000c0 RCX: 00007fef45a45157
Oct 19 09:18:26 luminous kernel: RDX: 00007fef3b435c00 RSI: 000000008038550a RDI: 000000000000002f
Oct 19 09:18:26 luminous kernel: RBP: 00007fef39582900 R08: 0000000000000020 R09: 0000000000001800
Oct 19 09:18:26 luminous kernel: R10: 00007fef3b4c7680 R11: 0000000000000246 R12: 00000000bffcaa70
Oct 19 09:18:26 luminous kernel: R13: 0000000000000000 R14: 00007fef3b4f1d00 R15: 00000000000000c0
Oct 19 09:18:26 luminous kernel: Code: 00 c7 45 a8 00 00 00 00 48 c7 45 b8 00 00 00 00 0f 85 5e f8 ff ff 89 c2 48 c7 c7 40 df 32 c0 c6 05 37 2b 01 00 01 e8 ce 
Oct 19 09:18:26 luminous kernel: ---[ end trace d4fb3b0080281849 ]---
Oct 19 09:18:26 luminous kernel: DMAR: DRHD: handling fault status reg 3
Oct 19 09:18:26 luminous kernel: DMAR: [DMA Read] Request device [00:14.0] fault addr 0 [fault reason 06] PTE Read access is not set
Oct 19 09:18:44 luminous dbus[398]: [system] Activating via systemd: service name='org.bluez' unit='dbus-org.bluez.service'
Oct 19 09:18:44 luminous dbus[398]: [system] Activation via systemd failed for unit 'dbus-org.bluez.service': Unit dbus-org.bluez.service not found.
Oct 19 09:18:46 luminous kernel: xhci_hcd 0000:00:14.0: xHCI host not responding to stop endpoint command.
Oct 19 09:18:46 luminous kernel: xhci_hcd 0000:00:14.0: xHCI host controller not responding, assume dead
Oct 19 09:18:46 luminous kernel: xhci_hcd 0000:00:14.0: HC died; cleaning up
Oct 19 09:18:46 luminous kernel: usb 1-3: USB disconnect, device number 6
Oct 19 09:18:46 luminous kernel: usb 1-6: Not enough bandwidth for altsetting 0
Oct 19 09:18:46 luminous kernel: usb 1-4: USB disconnect, device number 2
Oct 19 09:18:46 luminous kernel: usb 1-6: USB disconnect, device number 3
Oct 19 09:18:46 luminous kernel: usb 1-11: USB disconnect, device number 4
Oct 19 09:18:46 luminous kernel: usb 1-12: USB disconnect, device number 5

Last edited by fede (2017-10-19 17:30:32)

Offline

#2 2017-10-19 16:39:53

mich41
Member
Registered: 2012-06-22
Posts: 796

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

Looks like this: http://www.spinics.net/lists/linux-usb/msg161659.html

It seems that all current releases (LTS too) have this issue. So either: disconnect webcams/audio or disable them in Chrome if possible, downgrade to 4.13.5 if you aren't worried about bugs/security, wait for 4.13.9 or whatever version will get the fix, apply the patch manually and recompile.

EDIT:

I scrolled your log down and it seems that after the warning you also get an IOMMU fault and the USB controller hangs. Not sure if it's caused by the above bug, recently multiple people reported IOMMU problems because IOMMU got enabled by default for the first time in Arch history AFAIK.

Last edited by mich41 (2017-10-19 16:52:55)

Offline

#3 2017-10-19 17:15:45

fede
Member
Registered: 2007-06-29
Posts: 126

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

Thanks for the reply! Yes this seems spot on. The problem does not happen on 4.13.5 so I'll try to recompile it with this patch soon.
By the way, my IOMMU is on  and has been on since I've installed the system since I use KVM with IOMMU hardware redirection and would like to keep on doing so. So I suppose the default change must have been meaningless for me as it was already on.

Thanks!

Offline

#4 2017-10-19 17:40:01

mich41
Member
Registered: 2012-06-22
Posts: 796

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

OK, so IOMMU fault may really be caused by this buffer size check. That, if confirmed, is something you may want to report to this mailing list. This check is going to be removed, but perhaps somebody should verify that there are no similar checks in this code which could also result in a NULL pointer being submitted to the hardware. A single error with some webcam is not a good reason to screw up the whole USB bus.

Offline

#5 2017-10-23 07:53:44

ulteq
Member
Registered: 2016-07-16
Posts: 24

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

I'm suffering from the same issue. How did you solve it?

The problem occurs every time I try to redirect my audio interface (USB 1.1) into a KVM Windows 10 Guest.

Oct 22 21:44:29 arch kernel: uurb->buffer_length is too short 18432 vs 0
Oct 22 21:44:29 arch kernel: ------------[ cut here ]------------
Oct 22 21:44:29 arch kernel: WARNING: CPU: 7 PID: 2388 at drivers/usb/core/devio.c:1583 proc_do_submiturb+0x993/0xd60 [usbcore]
Oct 22 21:44:29 arch kernel: Modules linked in: snd_usb_mytek(O) snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_device fuse uas usb_storage ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_conntrack xt_CHECKSUM ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebta
Oct 22 21:44:29 arch kernel:  video coretemp iTCO_wdt snd_timer iTCO_vendor_support kvm_intel snd wmi_bmof mxm_wmi soundcore kvm e1000e input_leds evdev mei_me tpm_tis ch341 joydev ptp led_class tpm_tis_core mousedev intel_cstate intel_rapl_perf mac_hid u
Oct 22 21:44:29 arch kernel: CPU: 7 PID: 2388 Comm: qemu-system-x86 Tainted: P           O    4.13.8-1-ARCH #1
Oct 22 21:44:29 arch kernel: Hardware name: ASUS All Series/STRIX X99 GAMING, BIOS 1701 03/31/2017
Oct 22 21:44:29 arch kernel: task: ffff9c4d694ee900 task.stack: ffffb02a09d6c000
Oct 22 21:44:29 arch kernel: RIP: 0010:proc_do_submiturb+0x993/0xd60 [usbcore]
Oct 22 21:44:29 arch kernel: RSP: 0018:ffffb02a09d6fd38 EFLAGS: 00010282
Oct 22 21:44:29 arch kernel: RAX: 000000000000002b RBX: ffff9c4d76bcbe00 RCX: ffffffff8aa55328
Oct 22 21:44:29 arch kernel: RDX: 0000000000000000 RSI: 0000000000000096 RDI: 0000000000000283
Oct 22 21:44:29 arch kernel: RBP: ffffb02a09d6fdc0 R08: 00000000000004cd R09: ffffffff8aca68e0
Oct 22 21:44:29 arch kernel: R10: 0000000000000200 R11: 0000000000000000 R12: 0000000000000001
Oct 22 21:44:29 arch kernel: R13: ffff9c4d90a48900 R14: 00007f6f0bfa6078 R15: ffffb02a09d6fe00
Oct 22 21:44:29 arch kernel: FS:  00007f6f20a27340(0000) GS:ffff9c4d9f3c0000(0000) knlGS:0000000000000000
Oct 22 21:44:29 arch kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Oct 22 21:44:29 arch kernel: CR2: 00007f66fe50c420 CR3: 0000000307711000 CR4: 00000000001426e0
Oct 22 21:44:29 arch kernel: Call Trace:
Oct 22 21:44:29 arch kernel:  ? mem_cgroup_commit_charge+0x82/0x510
Oct 22 21:44:29 arch kernel:  usbdev_do_ioctl+0xa34/0x1170 [usbcore]
Oct 22 21:44:29 arch kernel:  usbdev_ioctl+0xe/0x20 [usbcore]
Oct 22 21:44:29 arch kernel:  do_vfs_ioctl+0xa5/0x600
Oct 22 21:44:29 arch kernel:  ? __fget+0x6e/0x90
Oct 22 21:44:29 arch kernel:  SyS_ioctl+0x79/0x90
Oct 22 21:44:29 arch kernel:  entry_SYSCALL_64_fastpath+0x1a/0xa5
Oct 22 21:44:29 arch kernel: RIP: 0033:0x7f6f197e4157
Oct 22 21:44:29 arch kernel: RSP: 002b:00007ffcb2d718d8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
Oct 22 21:44:29 arch kernel: RAX: ffffffffffffffda RBX: 00007f6f037519c0 RCX: 00007f6f197e4157
Oct 22 21:44:29 arch kernel: RDX: 00007f6f0bfa6040 RSI: 000000008038550a RDI: 000000000000007a
Oct 22 21:44:29 arch kernel: RBP: 0000000000000240 R08: 0000000000000018 R09: 0000000000001800
Oct 22 21:44:29 arch kernel: R10: 00007f6f08f3fd00 R11: 0000000000000246 R12: 00007f66fe50c400
Oct 22 21:44:29 arch kernel: R13: 0000000000000240 R14: 0000000000000240 R15: 00007f6f037519d0
Oct 22 21:44:29 arch kernel: Code: 00 c7 45 a8 00 00 00 00 48 c7 45 b8 00 00 00 00 0f 85 5e f8 ff ff 89 c2 48 c7 c7 40 ff 1d c0 c6 05 37 2b 01 00 01 e8 ce 9c f0 c9 <0f> ff 49 63 47 18 e9 3e f8 ff ff c7 45 8c 00 00 00 00 c7 45 98 
Oct 22 21:44:29 arch kernel: ---[ end trace 2aa5c597b48c422f ]---
Oct 22 21:44:29 arch kernel: DMAR: DRHD: handling fault status reg 2
Oct 22 21:44:29 arch kernel: DMAR: [DMA Read] Request device [00:14.0] fault addr 0 [fault reason 06] PTE Read access is not set

Offline

#6 2017-10-23 08:04:26

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

I actually made a bug report for this not knowing I had to send to a mailing list (https://bugzilla.kernel.org/show_bug.cgi?id=197275) but haven't gotten around to retesting on the latest git yet.
Does your Webcam have an embedded microphone by any chance? From what I noticed it seems to be contained to audio devices.

Offline

#7 2017-10-23 09:03:01

ulteq
Member
Registered: 2016-07-16
Posts: 24

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

Mouse and Keyboard redirects work just fine. The problem only occurs when I pass through my audio card.

Offline

#8 2017-10-23 14:39:18

fede
Member
Registered: 2007-06-29
Posts: 126

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

My webcam has an embedded microphone.
On the other hand, I'm almost certain I tried removing the webcam hardware redirect from the VM and my USB ports still crashed. Perhaps I could test that again, if that's meaningful for someone.

Last edited by fede (2017-10-23 14:39:54)

Offline

#9 2017-10-24 15:00:32

Omar007
Member
Registered: 2015-04-09
Posts: 368

Re: [SOLVED] All USB devices powered off by error from inside KVM guest

That would probably be most meaningful to yourself tbh. If you don't attempt to forward the device it should not pose a problem. If it does for you, maybe something else is going on as well.
I still need to retest on a newer kernel but will not be able to do so until, at the very earliest, next weekend.

Sidenote/FYI:
For anyone ending up in this situation, you can recover the XHCI Host Controller without a reboot. (though you may need to SSH in if your system has no working keyboard available because of the crash)
While it will not allow you to suddenly assign the device to a VM or anything, it will at least allow you to continue working without having to reboot completely. (replace '0000:00:14.0' with your crashed Host Controller's address)
As root run:

echo -n "0000:00:14.0" | tee /sys/bus/pci/drivers/xhci_hcd/unbind
echo -n "0000:00:14.0" | tee /sys/bus/pci/drivers/xhci_hcd/bind

UPDATE:
I've retried with 4.13.9 (as of this moment in testing) and everything seems to work perfectly fine for me now. I let it attach and detach the audio device every 5 seconds for about 10 minutes, no crashes at all.

UPDATE2:
Turns out the crash is just postponed. As soon as the device is actively used by the VM, the host controller still crashed with the mentioned error.
It did spam the following just before though, which I had not seen before:

xhci_hcd 0000:00:14.0: WARN Event TRB for slot 24 ep 2 with no TDs queued?

But that may be completely unrelated. I did not have the time to investigate any further.

UPDATE3:
The commit in question should be reverted in 4.13.10; https://lwn.net/Articles/737577/

Last edited by Omar007 (2017-10-29 23:18:41)

Offline

Board footer

Powered by FluxBB