Arch Linux

marscher

Member

Registered: 2020-03-27

Posts: 2

LVM cache: lots of dirty blocks after reboot (Could not detach DM)

Hi everybody,

recently I wanted to check out the lvm caching feature in combination with large mechanical drive with a small ssd.
I created a cache pool and backed my root partition to use it. But now I figured, that after every reboot my hdd led is
constantly blinking due to 99% "dirty cache blocks" being written back.

commands used to create the caching:

lvcreate -n meta -L 256MB vg /dev/sdc # sdc is the SSD
lvcreate -n cache -l 99%PVs vg /dev/sdc
lvconvert --type cache-pool --cachemode writeback --poolmetadata vg/meta vg/cache

After that I played around a bit and was impressed about new application startup timing etc.
However I now decided to increase my rootfs by 100G from the available extends on the volume group.
So I needed to uncache the root partition again, grew my fs and performed the caching again like this:

lvcreate -n cache -l 99%PVs vg /dev/sdc
lvcreate -n meta -L 256MB vg /dev/sdc
lvconvert --type cache-pool --cachemode writeback --poolmetadata vg/meta vg/cache

The caching stats have been obtained via:

lvdisplay vg/arch_xfs|rg Cache

  LV Cache pool name     cache
  LV Cache origin name   arch_xfs_corig
  Cache used blocks      40.61%
  Cache metadata blocks  4.34%
  Cache dirty blocks     0.00%
  Cache read hits/misses 20057 / 448
  Cache wrt hits/misses  15259 / 3191
  Cache demotions        0
  Cache promotions       73

After a reboot (nothing done in the meantime), repeating this command I get

  LV Cache pool name     cache
  LV Cache origin name   arch_xfs_corig
  Cache used blocks      40.61%
  Cache metadata blocks  4.35%
  Cache dirty blocks     96.88%
  Cache read hits/misses 7935 / 55
  Cache wrt hits/misses  1362 / 20
  Cache demotions        0
  Cache promotions       8

  As you see all the blocks are being marked as dirty, which indicates some kind of error.

  I did another reboot from text console to be able to spot any messages. Now there seems to be a culprit:

  sd-umoun[1157]: Failed to unmount /oldroot: Device or resource busy
  sd-unmou[1158]: Failed to unmount /oldroot/sys: Device or resource busy
  shutdown[1]: Could not detach DM /dev/dm-4: Device or resource busy
  shutdown[1]: Could not detach DM /dev/dm-3: Device or resource busy
  shutdown[1]: Could not detach DM /dev/dm-2: Device or resource busy
  shutdown[1]: Could not detach DM /dev/dm-1: Device or resource busy
  shutdown[1]: Failed to finalize  filesystems, DM devices, ignoring

The main question is now if I should blame systemd for just rebooting the system without flushing the DM devices?
I'd like to report it, but I would rather do it in the right spot (e.g. arch bug tracker, systemd github or even lvm (kernel) tracker).

Without caching I had no problems, but I cannot tell, if this detach error showed up during shutdown, because I was not looking for it.

Any advice would be very much appreciated!

Best regards,
Martin

PS: this is my first post to this forum, so please forgive me, if I missed some rules.

Last edited by marscher (2020-03-27 15:34:18)

marscher

Member

Registered: 2020-03-27

Posts: 2

Re: LVM cache: lots of dirty blocks after reboot (Could not detach DM)

I think I'm suffering from this bug: https://bugs.archlinux.org/task/63697