Unable to set I/O scheduler for a USB stick reliably

post-factum · 2020-11-29 17:05:01

So, I wanna be able to set an I/O scheduler for a USB stick. What I have is this udev rule:

ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/scheduler}="bfq"
ACTION=="add|change", KERNEL=="nvme[0-9]n[0-9]", ATTR{queue/scheduler}="bfq"
ACTION=="add|change", KERNEL=="sr[0-9]", ATTR{queue/scheduler}="bfq"
ACTION=="add|change", KERNEL=="vd[a-z]", ATTR{queue/scheduler}="bfq"

It should cover all the use cases I have. For NVMe devices it works fine, I always see BFQ set for them. With USB sticks that are /dev/sda* it's another story. Consider I insert a USB stick and see this printout in the udev monitor:

KERNEL[34193.415250] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4 (usb)
KERNEL[34193.415437] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0 (usb)
KERNEL[34193.415541] add      /devices/virtual/workqueue/scsi_tmf_2 (workqueue)
KERNEL[34193.415586] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2 (scsi)
KERNEL[34193.415596] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/scsi_host/host2 (scsi_host)
KERNEL[34193.415619] bind     /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0 (usb)
KERNEL[34193.415635] bind     /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4 (usb)
UDEV  [34193.423622] add      /devices/virtual/workqueue/scsi_tmf_2 (workqueue)
UDEV  [34193.423967] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4 (usb)
UDEV  [34193.424786] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0 (usb)
UDEV  [34193.425307] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2 (scsi)
UDEV  [34193.425903] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/scsi_host/host2 (scsi_host)
UDEV  [34193.426457] bind     /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0 (usb)
UDEV  [34193.427231] bind     /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4 (usb)
KERNEL[34194.418354] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0 (scsi)
KERNEL[34194.418371] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0 (scsi)
KERNEL[34194.418380] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/scsi_device/2:0:0:0 (scsi_device)
KERNEL[34194.418429] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/bsg/2:0:0:0 (bsg)
KERNEL[34194.418451] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/scsi_disk/2:0:0:0 (scsi_disk)
UDEV  [34194.419144] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0 (scsi)
UDEV  [34194.419843] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0 (scsi)
UDEV  [34194.421107] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/scsi_device/2:0:0:0 (scsi_device)
UDEV  [34194.421329] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/bsg/2:0:0:0 (bsg)
UDEV  [34194.421687] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/scsi_disk/2:0:0:0 (scsi_disk)
KERNEL[34194.454720] add      /devices/virtual/bdi/8:0 (bdi)
UDEV  [34194.455036] add      /devices/virtual/bdi/8:0 (bdi)
KERNEL[34194.455850] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/block/sda (block)
KERNEL[34194.455876] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/block/sda/sda1 (block)
KERNEL[34194.456635] bind     /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0 (scsi)
UDEV  [34194.488989] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/block/sda (block)
UDEV  [34194.511516] add      /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/block/sda/sda1 (block)
UDEV  [34194.512338] bind     /devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0 (scsi)

So I expect that once "ADD" event is emitted from kernel, the "/sys" hierarchy is ready to use. It is not the case, though:

systemd-udevd[67192]: sda: /usr/lib/udev/rules.d/10-io-scheduler.rules:1 Failed to write ATTR{/sys/devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:08.0/0000:07:00.1/usb3/3-4/3-4:1.0/host2/target2:0:0/2:0:0:0/block/sda/queue/scheduler}, ignoring: No such file or directory

Consequently, the scheduler is not set for this device. If I trigger the rules manually later, it works. Also, sometimes it works just after USB stick insertion, but rarely, so this is a race of some kind.

What do I miss and how to fix this?

Thanks.

jonathon · 2020-11-29 17:32:52

Just to confirm, I see a very similar same thing in `journalctl -u systemd-udevd` when inserting a USB stick:

Nov 29 17:02:07 host systemd-udevd[113791]: sdc2: /etc/udev/rules.d/60-scheduler.rules:3 Failed to write ATTR{/sys/devices/pci0000:00/0000:00:01.3/0000:03:00.0/usb1/1-4/1-4:1.0/host12/target12:0:0/12:0:0:0/block/sdc/sdc2/queue/scheduler}, ignoring: No such file or directory
Nov 29 17:02:07 host systemd-udevd[113788]: sdc1: /etc/udev/rules.d/60-scheduler.rules:3 Failed to write ATTR{/sys/devices/pci0000:00/0000:00:01.3/0000:03:00.0/usb1/1-4/1-4:1.0/host12/target12:0:0/12:0:0:0/block/sdc/sdc1/queue/scheduler}, ignoring: No such file or directory
Nov 29 17:02:07 host systemd-udevd[113788]: sdc1: /etc/udev/rules.d/60-scheduler.rules:3 Failed to write ATTR{/sys/devices/pci0000:00/0000:00:01.3/0000:03:00.0/usb1/1-4/1-4:1.0/host12/target12:0:0/12:0:0:0/block/sdc/sdc1/queue/scheduler}, ignoring: No such file or directory

(In my case it's trying to write for a partition, not just for the disk...)

However - setting the scheduler works so this might be down to the run ordering/priority? That is, my rules are running at 60, yours are much earlier at 10?

Last edited by jonathon (2020-11-29 17:44:09)

post-factum · 2020-11-29 17:51:07

This must have something to do with priorities. I've renamed the file to start with 60, and the error is not there any more.

Arch Linux

#1 2020-11-29 17:05:01

Unable to set I/O scheduler for a USB stick reliably

#2 2020-11-29 17:32:52

Re: Unable to set I/O scheduler for a USB stick reliably

#3 2020-11-29 17:51:07

Re: Unable to set I/O scheduler for a USB stick reliably

Board footer