You are not logged in.

#1 2014-07-09 23:18:12

Registered: 2013-11-15
Posts: 18

non-root-ZFS over LUKS: failure of systemd understanding

I have a RAID-Z2 pool on six HDDs which is not used for my root partition, and I'd like to encrypt each of them. My process is to wipe a drive, format the dm-crypt device via LUKS, append appropriately to /etc/crypttab, and then replace the now stale unencrypted drive entry in the zpool with the new /dev/mapper name and wait for resilvering to finish. Having RAID-Z2 allows me to safely add this encryption "in-place" (i.e. without blowing the entire pool away) as long as I do it one disk at a time.

My strategy worked well for the first drive, until I rebooted my system as a sanity check before I moved on to the second drive. Though the new LUKS device is available ZFS, is not bringing it online when it imports the pool (the device is listed UNAVAIL and thus the pool DEGRADED). I've come to believe that ZFS is importing the pool before dm-crypt has unlocked any/all devices, but I'm stuck on how to configure this dependency - this is the first time I've really worked with systemd.

From what I can tell, both ZFS and dm-crypt are being managed by systemd:

    ~ $ systemctl list-units | grep "crypt\|zfs"
    systemd-cryptsetup@locker\x2d1.service  loaded active exited  ...
    zfs-import-cache.service                loaded active exited  ...
    zfs-mount.service                       loaded active exited  ...
    zfs-share.service                       loaded active exited  ...
    system-systemd\x2dcryptsetup.slice      loaded active active  ...                       loaded active active  ...                              loaded active active  ...

    ~ $ find /usr/lib/systemd/ -name *crypt*

    ~ $ find /usr/lib/systemd/ -name *zfs*

Yet I see no boot log entries for cryptsetup (even though the encrypted block device is available under /dev/mapper as soon as I log in):

    ~ $ journalctl -b | grep 'zed\['
    Jul 09 05:20:03 zed[720]: ZFS Event Daemon 0.6.3-1
    Jul 09 05:20:03 zed[720]: Processing events since eid=0
    Jul 09 05:20:03 zed[720]: Invoking "" eid=1 pid=726
    Jul 09 05:20:03 zed[727]: eid=1 class=statechange
    Jul 09 05:20:03 zed[720]: Finished "" eid=1 pid=726 exit=0
    Jul 09 05:20:03 zed[720]: Invoking "" eid=2 pid=729

    ~ $ journalctl -b | grep 'crypt'
    ~ $

I attempted to customize the ZFS import service to add cryptsetup as a dependency:

    ~ $ cat /etc/systemd/system/zfs-import-cache.service.d/luks.conf

but no joy. I did the same (same file contents) for the zfs-import-scan.service as well - again, no luck. However, systemd-delta does see these types of changes:

    ~ $ systemd-delta
    [EXTENDED]   /run/systemd/system/session-c2.scope → /run/systemd/system/session-c2.scope.d/90-SendSIGHUP.conf
    [EXTENDED]   /run/systemd/system/session-c2.scope → /run/systemd/system/session-c2.scope.d/90-After-systemd-user-sessions\x2eservic
    [EXTENDED]   /run/systemd/system/session-c2.scope → /run/systemd/system/session-c2.scope.d/90-After-systemd-logind\x2eservice.conf
    [EXTENDED]   /run/systemd/system/session-c2.scope → /run/systemd/system/session-c2.scope.d/90-Description.conf
    [EXTENDED]   /run/systemd/system/session-c2.scope → /run/systemd/system/session-c2.scope.d/90-Slice.conf
    [EXTENDED]   /usr/lib/systemd/system/zfs-import-cache.service → /etc/systemd/system/zfs-import-cache.service.d/luks.conf

From my reading it doesn't look like I can define dependencies on targets, so I'm not really sure where to go for further research. I even went as far as adding encrypt to mkinitcpio HOOKS before filesystems as suggested for encrypted root partitions, but that didn't seem to matter either.

Any direction would be greatly appreciated. All of the documentation I've found (mailing list archives, forums, web search) between the communities of systemd, Arch, ZFS, and dm-crypt (even though I'm fairly sure this is all due to a lack of systemd knowledge on my part) all seem to assume either that you have your root filesystem in ZFS or that you're using LVM on top of LUKS and none of this advice seems to translate directly to what I'm doing.


#2 2014-07-11 09:37:29

Registered: 2014-07-11
Posts: 2

Re: non-root-ZFS over LUKS: failure of systemd understanding

Running into the same problem here with a RAIDZ1 pool of 2 harddisks on top of dm-crypt with LUKS. I tried the trick with a:

ostkind@somerandomarchbox ~ % cat /usr/lib/systemd/system/

Description=ZFS startup target


but no success.


#3 2014-07-11 20:05:41

Registered: 2013-11-15
Posts: 18

Re: non-root-ZFS over LUKS: failure of systemd understanding

@ostkind, did you try the same with After *and* Requires? The docs claim you need both to get systemd dependencies working the way you "expect".

Last edited by cmtonkinson (2014-07-11 20:06:34)


#4 2014-07-11 21:05:32

Registered: 2014-07-11
Posts: 2

Re: non-root-ZFS over LUKS: failure of systemd understanding

@cmtonkinson, tried right know to have in After and Requires. No success: After reboot zfs pool still not mounted and dm-crypt drives ready in /dev/mapper/.


#5 2014-07-12 14:52:08

From: Non-paged pool
Registered: 2014-02-21
Posts: 366

Re: non-root-ZFS over LUKS: failure of systemd understanding

You should be able to see if is before with:

systemd-analyze critical-chain --fuzz=1m


#6 2014-09-04 17:25:06

Registered: 2012-10-05
Posts: 23

Re: non-root-ZFS over LUKS: failure of systemd understanding

I had the same problem and I used

systemd-analyze plot > sys.svg

to reveal that a service named systemd-cryptsetup@cryptstorage.service (where cryptstorage is my LUKS mapped device) is started before zfs but takes about 3 seconds to finish.
The service zfs-import-cache.service which imports the zpools using /etc/zfs/zpool.cache does not wait for this to finish so cryptroot is not available and the import fails silently.

Here is what I did to get that straight:

mkdir /etc/systemd/system/zfs-import-cache.service.d
echo "[Unit]
After=systemd-cryptsetup@cryptstorage.service" >/etc/systemd/system/zfs-import-cache.service.d/cryptstorage.conf

You have to replace "cryptstorage" by the actual LUKS device, of course.
If you can confirm this solution or even have a more general one please let me know so I can update the wiki to include (this hopefully temporary) workaround.
Google gives a lot of hits on this problem but at least I didn't find any working solution so I started debugging the problem myself.

More general solution (pretty obvious):

mkdir /etc/systemd/system/zfs-import-cache.service.d
echo "[Unit]" >/etc/systemd/system/zfs-import-cache.service.d/cryptsetup.conf
mkdir /etc/systemd/system/zfs-import-scan.service.d
echo "[Unit]" >/etc/systemd/system/zfs-import-scan.service.d/cryptsetup.conf

Last edited by float (2014-09-04 17:41:50)


Board footer

Powered by FluxBB