You are not logged in.

#1 2024-10-07 02:46:16

ectospasm
Member
Registered: 2015-08-28
Posts: 292

[WORKAROUND] ACL error in snapper for systemd-homed Btrfs subvolume

WORKAROUND
Not sure how or why this changed as it was working on a previous boot, but my unlocked systemd-homed home directory is mounted with the noacl option.  I assume the .snapshots subvolume inherits this option.  Thus, any operation to change the ACL will result in the error "Operation not permitted.". 

To work around this, I set SYNC_ACL="no" in my /etc/snapper/configs/home-trey snapper config.  That allowed me to work around the problem, and snapper-timeline.service and snapper-cleanup.service to run successfully.

Interesting tidbit:  since cleanup hadn't actually been running since I first set up these snapshots, timeline had been adding 1-3GiB per day.  When cleanup finally ran, I was able to free almost 400GiB of snapshot space (my systemd-homed home volume is 1.6TiB, only user on this laptop).

Original Post for posterity
Whenever snapper-timeline.service or snapper-cleanup.service run, I see the following error in the systemd journal:

$ systemctl status snapper-timeline.service
× snapper-timeline.service - Timeline of Snapper Snapshots
     Loaded: loaded (/usr/lib/systemd/system/snapper-timeline.service; static)
    Drop-In: /etc/systemd/system/snapper-timeline.service.d
             └─override.conf
     Active: failed (Result: exit-code) since Sun 2024-10-06 22:07:36 EDT; 22min ago
   Duration: 123ms
 Invocation: dfc9e82ee18c4950bf7eb7550e769522
TriggeredBy: ● snapper-timeline.timer
       Docs: man:snapper(8)
             man:snapper-configs(5)
   Main PID: 861349 (code=exited, status=1/FAILURE)
   Mem peak: 5.6M
        CPU: 112ms

Oct 06 22:07:36 fluorine systemd[1]: Starting Timeline of Snapper Snapshots...
Oct 06 22:07:36 fluorine systemd-helper[861349]: Running timeline for 'home-trey'.
Oct 06 22:07:36 fluorine systemd-helper[861349]: ACL error.
Oct 06 22:07:36 fluorine systemd-helper[861349]: timeline for 'home-trey' failed.
Oct 06 22:07:36 fluorine systemd-helper[861349]: Running timeline for 'main'.
Oct 06 22:07:36 fluorine systemd[1]: Started Timeline of Snapper Snapshots.

$ systemctl status snapper-cleanup.service
× snapper-cleanup.service - Daily Cleanup of Snapper Snapshots
     Loaded: loaded (/usr/lib/systemd/system/snapper-cleanup.service; static)
     Active: failed (Result: exit-code) since Sun 2024-10-06 22:33:52 EDT; 32s ago
   Duration: 4.233s
 Invocation: f9b504f596c24f469744c98cb45aa9e1
TriggeredBy: ● snapper-cleanup.timer
       Docs: man:snapper(8)
             man:snapper-configs(5)
    Process: 899320 ExecStart=/usr/lib/snapper/systemd-helper --cleanup (code=exited, status=1/FAILURE)
   Main PID: 899320 (code=exited, status=1/FAILURE)
   Mem peak: 88.9M
        CPU: 1.913s

Oct 06 22:33:47 fluorine systemd[1]: Started Daily Cleanup of Snapper Snapshots.
Oct 06 22:33:47 fluorine systemd-helper[899320]: Running cleanup for 'home-trey'.
Oct 06 22:33:47 fluorine systemd-helper[899320]: ACL error.
Oct 06 22:33:47 fluorine systemd-helper[899320]: cleanup for 'home-trey' failed.
Oct 06 22:33:47 fluorine systemd-helper[899320]: Running cleanup for 'main'.
Oct 06 22:33:47 fluorine systemd-helper[899320]: Running number cleanup for 'main'.
Oct 06 22:33:47 fluorine systemd-helper[899320]: Running timeline cleanup for 'main'.
Oct 06 22:33:47 fluorine systemd-helper[899320]: Running empty-pre-post cleanup for 'main'.
Oct 06 22:33:51 fluorine systemd-helper[899320]: Running 'btrfs qgroup clear-stale /.snapshots'.
Oct 06 22:33:51 fluorine systemd-helper[899320]: 'btrfs qgroup clear-stale /.snapshots' failed.
Oct 06 22:33:52 fluorine systemd[1]: snapper-cleanup.service: Main process exited, code=exited, status=1/FAILURE
Oct 06 22:33:52 fluorine systemd[1]: snapper-cleanup.service: Failed with result 'exit-code'.
Oct 06 22:33:52 fluorine systemd[1]: snapper-cleanup.service: Consumed 1.913s CPU time, 88.9M memory peak.

It seems to be working for my main configuration, which is for my root Btrfs filesystem.  At least the home-trey config had been working up until I rebooted after a standard system upgrade last week, as the journal attests:

Sep 30 16:04:42 fluorine systemd[1]: Starting Timeline of Snapper Snapshots...
Sep 30 16:04:42 fluorine systemd-helper[3470245]: Running timeline for 'home-trey'.
Sep 30 16:04:42 fluorine systemd[1]: Started Timeline of Snapper Snapshots.
Sep 30 16:04:42 fluorine systemd-helper[3470245]: Running timeline for 'main'.
Sep 30 16:04:42 fluorine systemd[1]: snapper-timeline.service: Deactivated successfully.
Sep 30 20:00:46 fluorine systemd[1]: Starting Timeline of Snapper Snapshots...
Sep 30 20:00:47 fluorine systemd-helper[3753199]: Running timeline for 'home-trey'.
Sep 30 20:00:47 fluorine systemd[1]: Started Timeline of Snapper Snapshots.
Sep 30 20:00:47 fluorine systemd-helper[3753199]: Running timeline for 'main'.
Sep 30 20:00:47 fluorine systemd[1]: snapper-timeline.service: Deactivated successfully.
-- Boot ec55c5b307244db0a91ece52691e4dc0 --
Oct 01 04:08:22 fluorine systemd[1]: Starting Timeline of Snapper Snapshots...
Oct 01 04:08:23 fluorine systemd-helper[334009]: Running timeline for 'home-trey'.
Oct 01 04:08:23 fluorine systemd-helper[334009]: ACL error.
Oct 01 04:08:23 fluorine systemd-helper[334009]: timeline for 'home-trey' failed.
Oct 01 04:08:23 fluorine systemd-helper[334009]: Running timeline for 'main'.
Oct 01 04:08:23 fluorine systemd[1]: Started Timeline of Snapper Snapshots.
Oct 01 04:08:23 fluorine systemd[1]: snapper-timeline.service: Main process exited, code=exited, status=1/FAILURE
Oct 01 04:08:23 fluorine systemd[1]: snapper-timeline.service: Failed with result 'exit-code'.

I hadn't changed anything about the config for home-trey, but here it is for good measure:

# subvolume to snapshot
SUBVOLUME="/home/trey"

# filesystem type
FSTYPE="btrfs"


# btrfs qgroup for space aware cleanup algorithms
QGROUP=""


# fraction or absolute size of the filesystems space the snapshots may use
SPACE_LIMIT="0.5"

# fraction or absolute size of the filesystems space that should be free
FREE_LIMIT="0.2"


# users and groups allowed to work with config
ALLOW_USERS="trey"
ALLOW_GROUPS="trey"

# sync users and groups from ALLOW_USERS and ALLOW_GROUPS to .snapshots
# directory
SYNC_ACL="yes"


# start comparing pre- and post-snapshot in background after creating
# post-snapshot
BACKGROUND_COMPARISON="yes"


# run daily number cleanup
NUMBER_CLEANUP="yes"

# limit for number cleanup
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="30"
NUMBER_LIMIT_IMPORTANT="10"


# create hourly snapshots
TIMELINE_CREATE="yes"

# cleanup hourly snapshots after some time
TIMELINE_CLEANUP="yes"

# limits for timeline cleanup
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="10"
TIMELINE_LIMIT_DAILY="7"
TIMELINE_LIMIT_WEEKLY="4"
TIMELINE_LIMIT_MONTHLY="6"
TIMELINE_LIMIT_YEARLY="3"


# cleanup empty pre-post-pairs
EMPTY_PRE_POST_CLEANUP="yes"

# limits for empty pre-post-pair cleanup
EMPTY_PRE_POST_MIN_AGE="1800"

Here's the ACL on /home/trey/.snapshots:

getfacl -e /home/trey/.snapshots
getfacl: Removing leading '/' from absolute path names
# file: home/trey/.snapshots
# owner: root
# group: root
user::r-x
group::r-x
other::r-x

I've tried running sudo setfacl -m u:root:rwX /home/trey/.snapshots, but I get Operation not permitted.

I didn't realize until today that I hadn't enabled snapper-cleanup.timer, so that hasn't been running until just a few minutes ago.

Last edited by ectospasm (2024-10-13 17:02:28)

Offline

#2 2024-10-13 04:02:11

ectospasm
Member
Registered: 2015-08-28
Posts: 292

Re: [WORKAROUND] ACL error in snapper for systemd-homed Btrfs subvolume

OK, so it turns out my home directory is mounted with the noacl option, and because I have SYNC_ACL="yes" it was failing.  Judging by the journal before and after the last boot I reported, systemd-homed must now mount the unlocked home directory with noacl.  So for now, I've set the option to SYNC_ACL="no", and now my snapshots and cleanup are working.

I searched briefly, but I didn't see any mention of acl or noacl for systemd-homed.

Offline

Board footer

Powered by FluxBB