You are not logged in.

#1 Yesterday 17:53:24

860lacov
Member
Registered: 2020-05-02
Posts: 491

System failed to boot – Btrfs log tree error

Hi,

Today my system failed to boot with the following error:

BTRFS: error [...] failed to recover log tree
Failed to mount /sysroot

What I did:

    Mounted the root volume using:
   

    mount -o ro,rescue=all /dev/mapper/cryptroot /mnt
    

    Subvolumes were accessible, so I made a full backup using:
   

    rsync -aAX /mnt /external/backup
    

   
    Entered arch-chroot, ran:
   

    mkinitcpio -P
    

    → system still wouldn't boot

    Then I ran:
   

    btrfs check --repair /dev/mapper/cryptroot
    

    → log tree was cleared, system became mountable again

    After that I used:
   

    btrfs send -e /mnt/@ /mnt/@home | btrfs receive -e /external/snapshots
    

    → to create clean subvolume backups

Noticed:
btrfs check output before and after repair showed this difference:

Before: 747492561376 bytes used
After: 747492040704 bytes used

Current status:
System boots normally now and seems functional.
But I’m not sure if the filesystem is in a safe and consistent state long-term.

I'd appreciate any advice on whether everything looks OK.
Here are the outputs I've gathered:

$ sudo btrfs device stats /
[sudo] hasło użytkownika jm: 
[/dev/mapper/cryptroot].write_io_errs    0
[/dev/mapper/cryptroot].read_io_errs     0
[/dev/mapper/cryptroot].flush_io_errs    0
[/dev/mapper/cryptroot].corruption_errs  0
[/dev/mapper/cryptroot].generation_errs  0
$ sudo btrfs scrub status /
Total to scrub:   698.14GiB
Rate:             0.00B/s
Error summary:    no errors found
$ journalctl -p err..alert -b
lip 13 17:51:23 p14s systemd[1]: Failed to start Refresh Pacman mirrorlist with Reflector..
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: Failed to find connector name for /dev/i2c-10, set_connector_for_businfo_using_edid at line 1236 in file i2c_bus_core.c.
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    I2C_Bus_Info at: 0x5572c96ba490
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Flags:                   I2C_BUS_EXISTS | I2C_BUS_ACCESSIBLE | I2C_BUS_PROBED | I2C_BUS_X50_EDID | I2C_BUS_SYSFS_KNOWN_RELIABLE
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Bus /dev/i2c-10 found:   true
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Bus /dev/i2c-10 probed:  true
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    errno for open:          OK(0): success
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    drm_connector_found_by:  DRM_CONNECTOR_NOT_CHECKED (0)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    last_checked_asleep:       false
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Display connectors reported by /sys:
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-HDMI-A-1
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    3
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 101
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       disconnected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-DP-1
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    11
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 108
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       disconnected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-DP-2
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    12
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 115
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       disconnected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-DP-3
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    13
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 121
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       disconnected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-DP-4
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    14
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 127
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       disconnected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-DP-5
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    15
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 133
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       disconnected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-DP-6
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    16
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 139
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       disconnected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-eDP-1
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    2
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 93
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      enabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       connected
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:         +0          +4          +8          +c            0   4   8   c
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0000   00 ff ff ff ff ff ff 00 30 ae 3a 40 00 00 00 00   ........0.:@....
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0010   00 1e 01 04 a5 1e 13 78 e3 78 bc a4 55 4c 9b 24   .......x.x..UL.$
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0020   0d 4f 54 00 00 00 01 01 01 01 01 01 01 01 01 01   .OT.............
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0030   01 01 01 01 01 01 35 3c 80 a0 70 b0 23 40 30 20   ......5<..p.#@0
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0040   36 00 2e bd 10 00 00 18 00 00 00 fd 00 28 3c 4a   6............(<J
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0050   4a 0f 01 0a 20 20 20 20 20 20 00 00 00 0f 00 d1   J...      ......
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0060   0a 28 d1 0a 28 28 0a 04 0e 6f 04 14 00 00 00 fe   .(..((...o......
lip 13 17:56:21 p14s org_kde_powerdevil[1553]: +0070   00 4d 4e 45 30 30 37 4a 41 31 2d 31 0a 20 00 4d   .MNE007JA1-1. .M
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:    Connector:    card1-Writeback-1
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       i2c_busno:    -1
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       connector_id: 145
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       name:         (null)
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       enabled:      disabled
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       status:       unknown
lip 13 17:56:21 p14s org_kde_powerdevil[1553]:       edid:         None

I know that --repair could be dangerous, but after backing up the data I decided to give it a chance.
Not sure where else I should look for important signs of system issues — any advice is welcome

Offline

#2 Yesterday 18:36:29

topcat01
Member
Registered: 2019-09-17
Posts: 195

Re: System failed to boot – Btrfs log tree error

In future, when you you have a log tree error, which indicates that the filesystem is fine, but there was a problem replaying the last few journal log entries (unclean shutdown?), instead of --repair you can just clear the log tree (safe) using:

btrfs-rescue zero-log /dev/mapper/cryptroot

since we know that the log is the problem. Otherwise things should be fine.

Offline

#3 Yesterday 19:47:51

860lacov
Member
Registered: 2020-05-02
Posts: 491

Re: System failed to boot – Btrfs log tree error

topcat01 wrote:

In future, when you you have a log tree error, which indicates that the filesystem is fine, but there was a problem replaying the last few journal log entries (unclean shutdown?), instead of --repair you can just clear the log tree (safe) using:

btrfs-rescue zero-log /dev/mapper/cryptroot

since we know that the log is the problem. Otherwise things should be fine.

I use btrfs for many years.
Never had a problem before so I knew about snapshots, send | receive functionality etc.

Lesson learned. Thank you.

Offline

Board footer

Powered by FluxBB