You are not logged in.

Pages: 1

#1 2024-08-28 18:36:21

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

BTRFS file corruption

Hi, I'm dealing with a complicated issue and because I'm pretty new I'm not sure what to do. Occasionally programs will start acting strange, and if I open the terminal I'll see the error in kitty: "read only filesystem". All related logs disappear on reboot because of the aforementioned read only filesystem, so it's been hard to troubleshoot. I bought a new drive a couple months, so it seemed fine until this started happening, but it's not my first time seeing a drive degrade within a half year.

Just now I've been able to reliably induce the issue by running yay -Syu and updating about 80 packages. Unfortunately, after doing so a couple of times to gather logs, on next reboot there was an error and arch could not start:

Loading Linux Linux ...
error: file '/@/boot/vmlinuz-linux' not found
Loading initial ramdisk ...
error: you need to load the kernel first.

I booted off of a snapshot to get a working system. I'm using grub-btrfs and timeshift with default settings and went back to my most recent backup. Now that I'm running off of a snapshot, and because my personal data and system is at risk, I want to double check with other people what is going on:

  1. What does it actually mean to be booted into a snapshot? Is this snapshot like a branching path, so all changes I make are associated with the snapshot, and would disappear if I go back to my main install?

  2. How do I recover the files of my filesystem so that I can boot back into it?

  3. What's the best way to troubleshoot what's going on with my drive - whether it's a one time corruption issue or a hardware issue

Aug 28 13:35:52 slut kernel: BTRFS: error (device nvme1n1p2 state A) in __btrfs_free_extent:3213: errno=-117 Filesystem corrupted
Aug 28 13:35:52 slut kernel: BTRFS info (device nvme1n1p2 state EA): forced readonly
Aug 28 13:35:52 slut kernel: BTRFS info (device nvme1n1p2 state EA): leaf 116866383872 gen 139053 total ptrs 203 free space 628 owner 2
Aug 28 13:35:52 slut kernel:         item 0 key (8100036608 168 4096) itemoff 16230 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 113549 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26733 offset 1695744 count 1
Aug 28 13:35:52 slut kernel:         item 1 key (8100040704 168 4096) itemoff 16177 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138844 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 27553 offset 393216 count 1
Aug 28 13:35:52 slut kernel:         item 2 key (8100044800 168 4096) itemoff 16124 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 96794 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26637 offset 4493312 count 1
Aug 28 13:35:52 slut kernel:         item 3 key (8100048896 168 4096) itemoff 16071 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 90938 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3476317 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 4 key (8100052992 168 4096) itemoff 16018 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 70432 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2807820 offset 192512 count 1
Aug 28 13:35:52 slut kernel:         item 5 key (8100057088 168 4096) itemoff 15965 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71808 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2925115 offset 294912 count 1
Aug 28 13:35:52 slut kernel:         item 6 key (8100061184 168 4096) itemoff 15928 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 68780032 count 1
Aug 28 13:35:52 slut kernel:         item 7 key (8100065280 168 20480) itemoff 15875 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137077 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4766299 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 8 key (8100085760 168 4096) itemoff 15822 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138835 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4807350 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 9 key (8100089856 168 4096) itemoff 15769 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139053 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 945689 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 10 key (8100093952 168 4096) itemoff 15716 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 132103 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4639747 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 11 key (8100098048 168 4096) itemoff 15663 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 111676 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3999616 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 12 key (8100102144 168 4096) itemoff 15610 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116539 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4241817 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 13 key (8100106240 168 4096) itemoff 15557 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 69608 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2747863 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 14 key (8100110336 168 4096) itemoff 15504 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138826 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4769755 offset 8192 count 1
Aug 28 13:35:52 slut kernel:         item 15 key (8100114432 168 12288) itemoff 15451 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71432 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2894005 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 16 key (8100126720 168 4096) itemoff 15414 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 68780032 count 1
Aug 28 13:35:52 slut kernel:         item 17 key (8100130816 168 8192) itemoff 15361 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139050 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4812078 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 18 key (8100139008 168 4096) itemoff 15324 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 81842 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 204062720 count 1
Aug 28 13:35:52 slut kernel:         item 19 key (8100143104 168 4096) itemoff 15271 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 74579 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 462961 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 20 key (8100147200 168 8192) itemoff 15218 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119311 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4329736 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 21 key (8100155392 168 8192) itemoff 15165 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 63511 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2470221 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 22 key (8100163584 168 4096) itemoff 15112 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139053 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 939871 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 23 key (8100167680 168 4096) itemoff 15059 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 65718 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1994661 offset 2187264 count 1
Aug 28 13:35:52 slut kernel:         item 24 key (8100171776 168 4096) itemoff 15006 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 65579 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1994661 offset 2162688 count 1
Aug 28 13:35:52 slut kernel:         item 25 key (8100175872 168 4096) itemoff 14953 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 66268 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2601745 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 26 key (8100179968 168 8192) itemoff 14900 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137827 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4782818 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 27 key (8100188160 168 4096) itemoff 14847 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138835 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4807352 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 28 key (8100192256 168 4096) itemoff 14794 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 108104 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2169141 offset 860160 count 1
Aug 28 13:35:52 slut kernel:         item 29 key (8100196352 168 4096) itemoff 14741 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116713 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4248334 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 30 key (8100200448 168 4096) itemoff 14688 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 82506 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 15515 offset 20480 count 1
Aug 28 13:35:52 slut kernel:         item 31 key (8100204544 168 4096) itemoff 14635 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76037 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3129605 offset 1499136 count 1
Aug 28 13:35:52 slut kernel:         item 32 key (8100208640 168 4096) itemoff 14582 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 74900 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3060168 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 33 key (8100212736 168 4096) itemoff 14545 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 68861952 count 1
Aug 28 13:35:52 slut kernel:         item 34 key (8100216832 168 8192) itemoff 14492 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138324 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4790979 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 35 key (8100225024 168 4096) itemoff 14439 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 54775 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26733 offset 806912 count 1
Aug 28 13:35:52 slut kernel:         item 36 key (8100229120 168 4096) itemoff 14386 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 128209 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4577620 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 37 key (8100233216 168 4096) itemoff 14333 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 131148 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4636524 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 38 key (8100237312 168 4096) itemoff 14280 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 130146 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4612246 offset 241664 count 1
Aug 28 13:35:52 slut kernel:         item 39 key (8100241408 168 4096) itemoff 14227 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 136241 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26637 offset 8142848 count 1
Aug 28 13:35:52 slut kernel:         item 40 key (8100245504 168 4096) itemoff 14174 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 134944 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3762045 offset 1839104 count 1
Aug 28 13:35:52 slut kernel:         item 41 key (8100249600 168 4096) itemoff 14121 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76871 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3166041 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 42 key (8100253696 168 8192) itemoff 14068 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138848 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4808313 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 43 key (8100261888 168 4096) itemoff 14015 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119771 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4370493 offset 774144 count 1
Aug 28 13:35:52 slut kernel:         item 44 key (8100265984 168 4096) itemoff 13962 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119524 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 833678 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 45 key (8100270080 168 8192) itemoff 13909 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137937 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4783274 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 46 key (8100278272 168 4096) itemoff 13856 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 57922 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26752 offset 1163264 count 1
Aug 28 13:35:52 slut kernel:         item 47 key (8100282368 168 4096) itemoff 13803 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 109654 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26637 offset 5054464 count 1
Aug 28 13:35:52 slut kernel:         item 48 key (8100286464 168 4096) itemoff 13750 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 109656 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3938304 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 49 key (8100290560 168 4096) itemoff 13697 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 115811 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26637 offset 5984256 count 1
Aug 28 13:35:52 slut kernel:         item 50 key (8100294656 168 4096) itemoff 13644 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 109612 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3936706 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 51 key (8100298752 168 4096) itemoff 13591 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 66268 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2601747 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 52 key (8100302848 168 4096) itemoff 13538 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138835 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4807354 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 53 key (8100306944 168 4096) itemoff 13485 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71412 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2892869 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 54 key (8100311040 168 4096) itemoff 13448 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 81842 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 204062720 count 1
Aug 28 13:35:52 slut kernel:         item 55 key (8100315136 168 4096) itemoff 13395 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 74701 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26733 offset 1126400 count 1
Aug 28 13:35:52 slut kernel:         item 56 key (8100319232 168 4096) itemoff 13342 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139027 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 602532 offset 24576 count 1
Aug 28 13:35:52 slut kernel:         item 57 key (8100323328 168 4096) itemoff 13289 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 130184 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3221231 offset 1929216 count 1
Aug 28 13:35:52 slut kernel:         item 58 key (8100327424 168 4096) itemoff 13236 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139027 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 259 objectid 424 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 59 key (8100331520 168 4096) itemoff 13183 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119293 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4328717 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 60 key (8100335616 168 4096) itemoff 13130 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 109998 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3942072 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 61 key (8100339712 168 4096) itemoff 13093 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 69959680 count 1
Aug 28 13:35:52 slut kernel:         item 62 key (8100343808 168 4096) itemoff 13040 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 74701 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26637 offset 4018176 count 1
Aug 28 13:35:52 slut kernel:         item 63 key (8100347904 168 4096) itemoff 12987 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 123119 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26637 offset 7774208 count 1
Aug 28 13:35:52 slut kernel:         item 64 key (8100352000 168 4096) itemoff 12934 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 130738 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4627064 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 65 key (8100356096 168 4096) itemoff 12881 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139046 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4811661 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 66 key (8100360192 168 4096) itemoff 12828 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 117261 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4266010 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 67 key (8100364288 168 4096) itemoff 12775 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 114416 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4140254 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 68 key (8100368384 168 4096) itemoff 12722 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 90520 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3472426 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 69 key (8100372480 168 4096) itemoff 12669 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 69608 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2747863 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 70 key (8100376576 168 4096) itemoff 12616 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 63517 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2470438 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 71 key (8100380672 168 4096) itemoff 12563 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 60059 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 15506 offset 2691072 count 1
Aug 28 13:35:52 slut kernel:         item 72 key (8100384768 168 4096) itemoff 12526 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 70172672 count 1
Aug 28 13:35:52 slut kernel:         item 73 key (8100388864 168 16384) itemoff 12473 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 78910 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3204450 offset 3801088 count 1
Aug 28 13:35:52 slut kernel:         item 74 key (8100405248 168 4096) itemoff 12420 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 111676 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3999617 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 75 key (8100409344 168 4096) itemoff 12383 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 115994 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 846184448 count 1
Aug 28 13:35:52 slut kernel:         item 76 key (8100413440 168 8192) itemoff 12330 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 118561 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4301246 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 77 key (8100421632 168 8192) itemoff 12277 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 135272 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4728334 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 78 key (8100429824 168 4096) itemoff 12224 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138835 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4807394 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 79 key (8100433920 168 8192) itemoff 12171 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 94882 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3528893 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 80 key (8100442112 168 4096) itemoff 12118 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 66268 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2601747 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 81 key (8100446208 168 8192) itemoff 12065 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138644 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4792595 offset 1957888 count 1
Aug 28 13:35:52 slut kernel:         item 82 key (8100454400 168 4096) itemoff 12012 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 128780 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4597518 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 83 key (8100458496 168 8192) itemoff 11959 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76818 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3161742 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 84 key (8100466688 168 4096) itemoff 11906 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71412 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2892870 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 85 key (8100470784 168 4096) itemoff 11853 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138835 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4807395 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 86 key (8100474880 168 4096) itemoff 11800 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 82833 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 472400 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 87 key (8100478976 168 4096) itemoff 11747 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 82205 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3353029 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 88 key (8100483072 168 4096) itemoff 11694 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139046 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 259 objectid 439 offset 36864 count 1
Aug 28 13:35:52 slut kernel:         item 89 key (8100487168 168 4096) itemoff 11641 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 126650 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3762045 offset 1462272 count 1
Aug 28 13:35:52 slut kernel:         item 90 key (8100491264 168 4096) itemoff 11604 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 70205440 count 1
Aug 28 13:35:52 slut kernel:         item 91 key (8100495360 168 4096) itemoff 11551 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 82833 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 472402 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 92 key (8100499456 168 4096) itemoff 11498 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 85393 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3129605 offset 3235840 count 1
Aug 28 13:35:52 slut kernel:         item 93 key (8100503552 168 4096) itemoff 11445 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119311 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4329737 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 94 key (8100507648 168 4096) itemoff 11392 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119311 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4329742 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 95 key (8100511744 168 4096) itemoff 11339 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76856 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3164346 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 96 key (8100515840 168 4096) itemoff 11286 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76857 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3164636 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 97 key (8100519936 168 4096) itemoff 11249 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 73252864 count 1
Aug 28 13:35:52 slut kernel:         item 98 key (8100524032 168 4096) itemoff 11196 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 80601 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26649 offset 241664 count 1
Aug 28 13:35:52 slut kernel:         item 99 key (8100528128 168 16384) itemoff 11143 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 78910 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3204450 offset 3932160 count 1
Aug 28 13:35:52 slut kernel:         item 100 key (8100544512 168 8192) itemoff 11090 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71476 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2898025 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 101 key (8100552704 168 4096) itemoff 11024 itemsize 66
Aug 28 13:35:52 slut kernel:                 extent refs 2 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 524544 objectid 783717 offset 0 count 1
Aug 28 13:35:52 slut kernel:                 ref#1: shared data backref parent 77266944 count 1
Aug 28 13:35:52 slut kernel:         item 102 key (8100556800 168 4096) itemoff 10971 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138955 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4807296 offset 61440 count 1
Aug 28 13:35:52 slut kernel:         item 103 key (8100560896 168 4096) itemoff 10918 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 105627 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1994661 offset 3932160 count 1
Aug 28 13:35:52 slut kernel:         item 104 key (8100564992 168 4096) itemoff 10865 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 117385 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 832752 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 105 key (8100569088 168 4096) itemoff 10812 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116588 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 740501 offset 26607616 count 1
Aug 28 13:35:52 slut kernel:         item 106 key (8100573184 168 4096) itemoff 10759 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 69608 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2747870 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 107 key (8100577280 168 4096) itemoff 10706 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 783736 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 108 key (8100581376 168 4096) itemoff 10653 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 50834 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2007599 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 109 key (8100585472 168 8192) itemoff 10600 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137229 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4767766 offset 8192 count 1
Aug 28 13:35:52 slut kernel:         item 110 key (8100593664 168 4096) itemoff 10547 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 110071 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3762045 offset 192512 count 1
Aug 28 13:35:52 slut kernel:         item 111 key (8100597760 168 4096) itemoff 10494 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 98439 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 656062 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 112 key (8100601856 168 4096) itemoff 10441 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 783769 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 113 key (8100605952 168 8192) itemoff 10388 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138007 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4783517 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 114 key (8100614144 168 4096) itemoff 10335 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 117261 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4266010 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 115 key (8100618240 168 4096) itemoff 10282 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116588 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 740500 offset 20840448 count 1
Aug 28 13:35:52 slut kernel:         item 116 key (8100622336 168 4096) itemoff 10229 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 113653 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3129603 offset 155648 count 1
Aug 28 13:35:52 slut kernel:         item 117 key (8100626432 168 4096) itemoff 10176 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 81842 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 469947 offset 22544384 count 1
Aug 28 13:35:52 slut kernel:         item 118 key (8100630528 168 4096) itemoff 10123 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76481 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3131746 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 119 key (8100634624 168 4096) itemoff 10070 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76108 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3134946 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 120 key (8100638720 168 8192) itemoff 10017 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76036 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3131085 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 121 key (8100646912 168 4096) itemoff 9964 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119311 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4329742 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 122 key (8100651008 168 4096) itemoff 9911 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119311 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4104117 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 123 key (8100655104 168 4096) itemoff 9858 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 69608 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2747870 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 124 key (8100659200 168 4096) itemoff 9805 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 783790 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 125 key (8100663296 168 8192) itemoff 9752 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137170 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4767454 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 126 key (8100671488 168 4096) itemoff 9699 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138835 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4391026 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 127 key (8100675584 168 4096) itemoff 9646 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119311 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4329743 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 128 key (8100679680 168 8192) itemoff 9593 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138324 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4790980 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 129 key (8100687872 168 8192) itemoff 9540 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71543 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2905952 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 130 key (8100696064 168 4096) itemoff 9487 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71495 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2899890 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 131 key (8100700160 168 8192) itemoff 9434 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71566 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2909285 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 132 key (8100708352 168 4096) itemoff 9381 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71543 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2905970 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 133 key (8100712448 168 8192) itemoff 9328 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 74512 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3027586 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 134 key (8100720640 168 4096) itemoff 9291 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 115994 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 846184448 count 1
Aug 28 13:35:52 slut kernel:         item 135 key (8100724736 168 4096) itemoff 9238 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 105744 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1994661 offset 4030464 count 1
Aug 28 13:35:52 slut kernel:         item 136 key (8100728832 168 12288) itemoff 9185 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76035 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3131203 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 137 key (8100741120 168 8192) itemoff 9132 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 66699 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1994661 offset 2703360 count 1
Aug 28 13:35:52 slut kernel:         item 138 key (8100749312 168 4096) itemoff 9079 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76081 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3134143 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 139 key (8100753408 168 4096) itemoff 9026 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 126086 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4542290 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 140 key (8100757504 168 4096) itemoff 8973 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 66704 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 15506 offset 3088384 count 1
Aug 28 13:35:52 slut kernel:         item 141 key (8100761600 168 4096) itemoff 8920 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 66268 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2601750 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 142 key (8100765696 168 4096) itemoff 8867 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 61130 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1994661 offset 1024000 count 1
Aug 28 13:35:52 slut kernel:         item 143 key (8100769792 168 8192) itemoff 8814 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 135892 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4738712 offset 3391488 count 1
Aug 28 13:35:52 slut kernel:         item 144 key (8100777984 168 4096) itemoff 8761 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 60059 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2287202 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 145 key (8100782080 168 8192) itemoff 8708 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76891 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3168137 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 146 key (8100790272 168 4096) itemoff 8655 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 129959 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4606364 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 147 key (8100794368 168 4096) itemoff 8602 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 128494 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4582992 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 148 key (8100798464 168 4096) itemoff 8549 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 79626 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3251645 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 149 key (8100802560 168 4096) itemoff 8496 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76894 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3168344 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 150 key (8100806656 168 8192) itemoff 8443 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138024 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4783572 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 151 key (8100814848 168 4096) itemoff 8390 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76071 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3134088 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 152 key (8100818944 168 4096) itemoff 8337 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76071 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3134089 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 153 key (8100823040 168 4096) itemoff 8284 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139047 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1833870 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 154 key (8100827136 168 4096) itemoff 8231 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76071 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3134089 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 155 key (8100831232 168 4096) itemoff 8178 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76108 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3134947 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 156 key (8100835328 168 4096) itemoff 8125 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 67296 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 941 offset 356352 count 1
Aug 28 13:35:52 slut kernel:         item 157 key (8100839424 168 4096) itemoff 8072 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76595 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3148428 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 158 key (8100843520 168 4096) itemoff 8019 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76869 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3165801 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 159 key (8100847616 168 4096) itemoff 7966 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116588 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 740511 offset 9437184 count 1
Aug 28 13:35:52 slut kernel:         item 160 key (8100851712 168 4096) itemoff 7913 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 111692 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4003487 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 161 key (8100855808 168 4096) itemoff 7860 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71412 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2892871 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 162 key (8100859904 168 4096) itemoff 7807 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 81842 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 469947 offset 22675456 count 1
Aug 28 13:35:52 slut kernel:         item 163 key (8100864000 168 4096) itemoff 7754 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 116589 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 783811 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 164 key (8100868096 168 4096) itemoff 7701 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 66268 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2601753 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 165 key (8100872192 168 4096) itemoff 7648 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76071 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3129603 offset 94208 count 1
Aug 28 13:35:52 slut kernel:         item 166 key (8100876288 168 4096) itemoff 7595 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76081 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3134143 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 167 key (8100880384 168 8192) itemoff 7542 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76595 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3148429 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 168 key (8100888576 168 4096) itemoff 7489 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71476 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2898027 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 169 key (8100892672 168 4096) itemoff 7436 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71476 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2898028 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 170 key (8100896768 168 8192) itemoff 7383 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137304 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4736394 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 171 key (8100904960 168 4096) itemoff 7346 itemsize 37
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 115995 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: shared data backref parent 943849472 count 1
Aug 28 13:35:52 slut kernel:         item 172 key (8100909056 168 4096) itemoff 7293 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 62887 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2440967 offset 172032 count 1
Aug 28 13:35:52 slut kernel:         item 173 key (8100913152 168 4096) itemoff 7240 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119601 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 890965 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 174 key (8100917248 168 4096) itemoff 7187 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139016 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26673 offset 8192 count 1
Aug 28 13:35:52 slut kernel:         item 175 key (8100921344 168 4096) itemoff 7134 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 132103 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4639749 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 176 key (8100925440 168 8192) itemoff 7081 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 95768 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3537276 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 177 key (8100933632 168 4096) itemoff 7028 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 63488 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2468104 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 178 key (8100937728 168 4096) itemoff 6975 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 53715 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 26637 offset 3141632 count 1
Aug 28 13:35:52 slut kernel:         item 179 key (8100941824 168 4096) itemoff 6922 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138988 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4810156 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 180 key (8100945920 168 4096) itemoff 6869 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119580 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4348733 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 181 key (8100950016 168 8192) itemoff 6816 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137965 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4783368 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 182 key (8100958208 168 8192) itemoff 6763 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137233 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4736002 offset 4096 count 1
Aug 28 13:35:52 slut kernel:         item 183 key (8100970496 168 8192) itemoff 6710 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 119311 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4329745 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 184 key (8100978688 168 8192) itemoff 6657 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137916 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4783198 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 185 key (8100986880 168 12288) itemoff 6604 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 63511 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2470217 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 186 key (8100999168 168 8192) itemoff 6551 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137754 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4782462 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 187 key (8101007360 168 8192) itemoff 6498 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 137754 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4782464 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 188 key (8101015552 168 4096) itemoff 6445 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 79235 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3233323 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 189 key (8101019648 168 8192) itemoff 6392 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138600 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4791989 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 190 key (8101027840 168 4096) itemoff 6339 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138878 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3762045 offset 2027520 count 1
Aug 28 13:35:52 slut kernel:         item 191 key (8101031936 168 4096) itemoff 6286 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 56549 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 1136128 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 192 key (8101036032 168 4096) itemoff 6233 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71412 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2892872 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 193 key (8101040128 168 4096) itemoff 6180 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 64097 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2502963 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 194 key (8101044224 168 12288) itemoff 6127 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 69592 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2744913 offset 741376 count 1
Aug 28 13:35:52 slut kernel:         item 195 key (8101056512 168 4096) itemoff 6074 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71412 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2892873 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 196 key (8101060608 168 4096) itemoff 6021 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 81842 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 469947 offset 22806528 count 1
Aug 28 13:35:52 slut kernel:         item 197 key (8101064704 168 4096) itemoff 5968 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 139008 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4807277 offset 7245824 count 1
Aug 28 13:35:52 slut kernel:         item 198 key (8101068800 168 8192) itemoff 5915 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 76600 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 3149061 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 199 key (8101076992 168 8192) itemoff 5862 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 138638 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 4792832 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 200 key (8101085184 168 4096) itemoff 5809 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 82833 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 256 objectid 472403 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 201 key (8101089280 168 4096) itemoff 5756 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 71558 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2907623 offset 0 count 1
Aug 28 13:35:52 slut kernel:         item 202 key (8101093376 168 4096) itemoff 5703 itemsize 53
Aug 28 13:35:52 slut kernel:                 extent refs 1 gen 69608 flags 1
Aug 28 13:35:52 slut kernel:                 ref#0: extent data backref root 257 objectid 2747872 offset 0 count 1
Aug 28 13:35:52 slut kernel: BTRFS critical (device nvme1n1p2 state EA): unable to find ref byte nr 8100552704 parent 0 root 256 owner 783717 offset 0 slot 102
Aug 28 13:35:52 slut kernel: BTRFS error (device nvme1n1p2 state EA): failed to run delayed ref for logical 8100552704 num_bytes 4096 type 178 action 2 ref_mod 1: -2
Aug 28 13:35:52 slut kernel: BTRFS: error (device nvme1n1p2 state EA) in btrfs_run_delayed_refs:2207: errno=-2 No such entry

Last edited by witchwrenna (2024-08-28 18:40:36)

Offline

#2 2024-08-28 18:52:31

Head_on_a_Stick
Member
From: The Wirral
Registered: 2014-02-20
Posts: 9,003
Website

Re: BTRFS file corruption

witchwrenna wrote:

What does it actually mean to be booted into a snapshot? Is this snapshot like a branching path, so all changes I make are associated with the snapshot, and would disappear if I go back to my main install?

That's right, yes.

witchwrenna wrote:

How do I recover the files of my filesystem so that I can boot back into it?

Your files should also be present in the snapshot. Be sure to back them up now, if you haven't already.

witchwrenna wrote:

What's the best way to troubleshoot what's going on with my drive - whether it's a one time corruption issue or a hardware issue

How long ago did you install this system? Have you been using btrfs-scrub@.timer?

Scrub the partition now then share the `btrfs scrub status` output here. Thanks.

EDIT: also check the drive health with smartctl or similar.

Last edited by Head_on_a_Stick (2024-08-28 18:53:15)

Jin, Jîyan, Azadî

Offline

#3 2024-08-28 19:31:12

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

Re: BTRFS file corruption

Head_on_a_Stick wrote:

How long ago did you install this system? Have you been using btrfs-scrub@.timer?

It's been 2 months. I went through a manual installation and didn't personally configure that, so I'm going to go with no.

Head_on_a_Stick wrote:

Scrub the partition now then share the `btrfs scrub status` output here. Thanks.

EDIT: also check the drive health with smartctl or similar.

I ran it on the root filesystem while still in my backup snapshot since I can't boot into my current install. Will this scan just the snapshot, or the whole disk? I see 1 uncorrectable listed.

sudo btrfs scrub status /           
UUID:             b80c51b8-58db-4670-a40d-0b10e1a2b757
Scrub started:    Wed Aug 28 15:08:31 2024
Status:           finished
Duration:         0:00:38
Total to scrub:   112.21GiB
Rate:             2.95GiB/s
Error summary:    csum=1
  Corrected:      0
  Uncorrectable:  1
  Unverified:     0

Smartctl says everything passes:

sudo smartctl -a /dev/nvme1n1
smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.10.5-arch1-1] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Samsung SSD 990 PRO with Heatsink 4TB
Serial Number:                      S7DSNJ0X112044A
Firmware Version:                   4B2QJXD7
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 4,000,787,030,016 [4.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      1
NVMe Version:                       2.0
Number of Namespaces:               1
Namespace 1 Size/Capacity:          4,000,787,030,016 [4.00 TB]
Namespace 1 Utilization:            127,183,155,200 [127 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 414143a0a5
Local Time is:                      Wed Aug 28 15:25:51 2024 EDT
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0055):     Comp DS_Mngmt Sav/Sel_Feat Timestmp
Log Page Attributes (0x2f):         S/H_per_NS Cmd_Eff_Lg Ext_Get_Lg Telmtry_Lg Log0_FISE_MI
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     82 Celsius
Critical Comp. Temp. Threshold:     85 Celsius

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     9.39W       -        -    0  0  0  0        0       0
 1 +     9.39W       -        -    1  1  1  1        0       0
 2 +     9.39W       -        -    2  2  2  2        0       0
 3 -   0.0400W       -        -    3  3  3  3     4200    2700
 4 -   0.0050W       -        -    4  4  4  4      500   21800

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        53 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    13,838,730 [7.08 TB]
Data Units Written:                 7,062,972 [3.61 TB]
Host Read Commands:                 84,063,377
Host Write Commands:                107,366,498
Controller Busy Time:               426
Power Cycles:                       107
Power On Hours:                     1,078
Unsafe Shutdowns:                   57
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               53 Celsius
Temperature Sensor 2:               56 Celsius

Error Information (NVMe Log 0x01, 16 of 64 entries)
No Errors Logged

Read Self-test Log failed: Invalid Field in Command (0x002)

Last edited by witchwrenna (2024-08-28 19:31:28)

Offline

#4 2024-08-28 19:44:58

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

Re: BTRFS file corruption

Out of 107 power cycles, smart says there are 57 unsafe shutdowns. Is typing reboot into the terminal considered an unsafe shutdown? Would that corrupt the filesystem?

Offline

#5 2024-08-28 20:06:58

Head_on_a_Stick
Member
From: The Wirral
Registered: 2014-02-20
Posts: 9,003
Website

Re: BTRFS file corruption

witchwrenna wrote:

I ran it on the root filesystem while still in my backup snapshot since I can't boot into my current install. Will this scan just the snapshot, or the whole disk?

I think it just scrubs the mountpoint.

I would use this (presuming you only have a single btrfs partition):

# mount /dev/nvme1n1p2 /mnt
# btrfs scrub start /mnt
# # wait
# btrfs scrub status /mnt
# umount /mnt
witchwrenna wrote:

I see 1 uncorrectable listed

Try the methods listed here and see if you can track down the bad files.

Having said that I would just re-format the partition and copy the backup, er, back. Why take the risk?

It is important to enable the scrub timer though, details on the relevant ArchWiki page.

witchwrenna wrote:

Is typing reboot into the terminal considered an unsafe shutdown? Would that corrupt the filesystem?

Not at all.

Btrfs should be immune to corruption from sudden power loss. I've run a laptop with no battery from btrfs and it took literally hundreds of hard resets with no data loss. It was scrubbed regularly though.

Jin, Jîyan, Azadî

Offline

#6 2024-08-28 22:36:01

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

Re: BTRFS file corruption

Alright, so followed your advice and tried this:

mount /dev/nvme1n1p2 /mnt   

ls /mnt             
-> @   @home   @snapshots  @var_cache   @var_log   timeshift-btrfs

btrfs scrub start /mnt

btrfs scrub status /mnt
UUID:             b80c51b8-58db-4670-a40d-0b10e1a2b757
Scrub started:    Wed Aug 28 17:19:43 2024
Status:           finished
Duration:         0:00:37
Total to scrub:   112.63GiB
Rate:             3.04GiB/s
Error summary:    csum=1
  Corrected:      0
  Uncorrectable:  1
  Unverified:     0

sudo dmesg | grep "checksum error at"                                         
[ 1890.966336] BTRFS warning (device nvme1n1p2): checksum error at logical 81734533120 on dev /dev/nvme1n1p2, physical 81742921728, root 288, inode 851883, offset 27525120, length 4096, links 1 (path: var/lib/flatpak/runtime/org.freedesktop.Platform.GL.nvidia-555-58-02/x86_64/1.4/b98fdca0728be9b1aa986b1569d4e15e79e26622fd3440e489feee9dda459afe/files/extra/libnvidia-rtcore.so.555.58.02)

So this file is corrupted with an unrecoverable sector, and it's setting my whole filesystem to read only because running pacman -Syu touches the nvidia packages and makes BTRFS panic? It certainly explains why I'm able to replicate this issue every time I run that command. I'm fine with removing just that package and maybe reinstalling if that's the issue.

What I don't understand is, what happened to my boot? I get the error: file '/@/boot/vmlinuz-linux' not found issue still. Maybe my startup configuration got messed up because the packages involved required recreating... I don't remember this part very well. Initrd, or mkinitcpio? So if the filesystem got set to read only in the middle of that part of the pacman process, it would explain that problem. Where should I look to check that?

Offline

#7 2024-08-29 00:20:58

ngoonee
Forum Fellow
From: Between Thailand and Singapore
Registered: 2009-03-17
Posts: 7,362

Re: BTRFS file corruption

witchwrenna wrote:

So this file is corrupted with an unrecoverable sector, and it's setting my whole filesystem to read only because running pacman -Syu touches the nvidia packages and makes BTRFS panic? It certainly explains why I'm able to replicate this issue every time I run that command. I'm fine with removing just that package and maybe reinstalling if that's the issue.

Is this on an SSD or a HDD?

Allan-Volunteer on the (topic being discussed) mailn lists. You never get the people who matters attention on the forums.
jasonwryan-Installing Arch is a measure of your literacy. Maintaining Arch is a measure of your diligence. Contributing to Arch is a measure of your competence.
Griemak-Bleeding edge, not bleeding flat. Edge denotes falls will occur from time to time. Bring your own parachute.

Offline

#8 2024-08-29 00:54:20

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

Re: BTRFS file corruption

It's an SSD.

Offline

#9 2024-08-29 17:21:31

Head_on_a_Stick
Member
From: The Wirral
Registered: 2014-02-20
Posts: 9,003
Website

Re: BTRFS file corruption

witchwrenna wrote:

what happened to my boot? I get the error: file '/@/boot/vmlinuz-linux' not found issue still. Maybe my startup configuration got messed up because the packages involved required recreating... I don't remember this part very well. Initrd, or mkinitcpio? So if the filesystem got set to read only in the middle of that part of the pacman process, it would explain that problem. Where should I look to check that?

I have no idea how Timeshift or grub-btrfs work, unfortunately.

Please share the content of /boot/grub/grub.cfg & /etc/fstab and also check /boot/ for the kernel, initramfs & µcode images.

Jin, Jîyan, Azadî

Offline

#10 2024-08-29 20:27:12

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

Re: BTRFS file corruption

Head_on_a_Stick wrote:

Please share the content of /boot/grub/grub.cfg & /etc/fstab and also check /boot/ for the kernel, initramfs & µcode images.

Seems a bit empty...

ls -la /mnt/@/boot    
drwxr-xr-x    - root 30 Jun 14:12  efi
drwxr-xr-x    - root 28 Aug 13:35  grub
.rw-r--r-- 8.1M root 15 Aug 09:05  intel-ucode.img

I did some reading and there's a possibility that a system crash in the middle of pacman update can cause the linux files themselves to go missing among other half installed packages. I booted up a live iso, mounted the drives and chrooted in, then tried to run pacman. Unfortunately I got the error that some library was too short, so somehow pacman itself or the libraries it uses also got hosed. I then followed the instructions listed in the wiki for when pacman crashes during an upgrade: https://wiki.archlinux.org/title/Pacman … an_upgrade

#hundreds of files appeared when I ran this command 
find /mnt/usr/lib -size 0

#Ran out of space in the default 256mb ramdisk
mount -o remount,size=1G /run/archiso/cowspace

#My 2 month old iso at this point had an expired key for one of the packages 
pacman-key --refresh [person's name]

#Set up per the wiki
mount -t proc proc /mnt/proc; mount --rbind /sys /mnt/sys; mount --rbind /dev /mnt/dev

#Many files seemed to be halfway installed, forced it with overwrite command
pacman --root=/mnt --cachedir=/mnt/var/cache/pacman/pkg -Syu --overwrite \*

#Upgrade failed here because of being sent back to read only mode

At this point, I had deleted the files listed in the scrub command and wondered if there was some deeper issues so I ran btrfs check --read only and here are the results:

[1/7] checking root items
[2/7] checking extents
data extent[8100552704, 4096] referencer count mismatch (root 256 owner 783717 offset 0) wanted 0 have 1
data extent[8100552704, 4096] bytenr mimsmatch, extent item bytenr 8100552704 file item bytenr 0
data extent[8100552704, 4096] referencer count mismatch (root 524544 owner 783717 offset 0) wanted 1 have 0
backpointer mismatch on [8100552704 4096]
ref mismatch on [17540980736 12288] extent item 576460752303423489, found 1
ERROR: errors found in extent allocation tree or chunk allocation
[3/7] checking free space tree
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups skipped (not enabled on this FS)
Opening filesystem to check...
Checking filesystem on /dev/disk/by-label/Root
UUID: b80c51b8-58db-4670-a40d-0b10e1a2b757
found 121083748352 bytes used, error(s) found
total csum bytes: 112625056
total tree bytes: 833961984
total fs tree bytes: 647987200
total extent tree bytes: 52494336
btree space waste bytes: 148806126
file data blocks allocated: 153469415424
 referenced 154757423104

The problem is that if there's something wrong with my filesystem, my snapshots are sitting on that same partition (on a different subvolume) in the filesystem. I'm worried if I revert to a snapshot that it won't actually fix the underlying problem. I think this is the point where I would run the btrfs check --repair command but there's lots of warnings on that. Should I do it or are there more things to check?

Last edited by witchwrenna (2024-08-29 20:33:03)

Offline

#11 2024-08-29 20:52:51

Head_on_a_Stick
Member
From: The Wirral
Registered: 2014-02-20
Posts: 9,003
Website

Re: BTRFS file corruption

witchwrenna wrote:

Seems a bit empty...

Probably best share the output of the `set` command from the GRUB command line then. Upload a picture to a hosting site and share the link here, thanks. That should show where grub.cfg is.

witchwrenna wrote:

I'm worried if I revert to a snapshot that it won't actually fix the underlying problem

That's why I suggested a re-format and restore approach instead. You do have a backup, right? Note that snapshots are not backups.

witchwrenna wrote:

I think this is the point where I would run the btrfs check --repair command but there's lots of warnings on that. Should I do it or are there more things to check?

Never run that command. Even if threatened.

Jin, Jîyan, Azadî

Offline

#12 2024-08-29 21:30:47

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

Re: BTRFS file corruption

Head_on_a_Stick wrote:

Probably best share the output of the `set` command from the GRUB command line then. Upload a picture to a hosting site and share the link here, thanks. That should show where grub.cfg is.

I think the empty folder shows that pacman getting interrupted caused the linux files to be removed and I remember the update included linux. Compare these two folders, one is my snapshot and the other is the non-snapshot version.

#snapshot boot
ls /boot          
 efi   grub   initramfs-linux-fallback.img   initramfs-linux.img   intel-ucode.img   vmlinuz-linux

#main boot
ls /mnt/@/boot/   
 efi   grub   intel-ucode.img

If you still want to see the set command, I'll get that for you.

Head_on_a_Stick wrote:

That's why I suggested a re-format and restore approach instead. You do have a backup, right? Note that snapshots are not backups.

Sorta! I'll figure something out between my github scripts and NAS files. I scripted out a lot of my install so I can bounce back pretty easily. If I wanted a quick solution I would have done that sooner, but I also want to learn here what went wrong and how to troubleshoot it in the future. I learnt a lot about pacman and btrfs from all this.

Head_on_a_Stick wrote:

Never run that command. Even if threatened.

Honestly, if I'm at the point where I'm reformatting anyway, I think I'm going to try it for fun.

Offline

#13 2024-09-01 00:48:39

witchwrenna
Member
Registered: 2024-07-05
Posts: 8

Re: BTRFS file corruption

Alright, here's my progress after doing a bunch of research. I copied all my data to a separate drive, booted into a live ISO again, and ran the following. I don't have logs because it was part of a live iso, so this is from memory.

#Fix filesystem and face potential data lose
btrfs check --repair /dev/nvme1n1p2

#See where any problems are without having to mount the filesystem
btrfs check --check-data-csum /dev/nvme1n1p2

#I had to mount all my subvolumes and recreate my drive structure for the next command to be able to work
mount -o subvolid=256 /dev/disk/by-label/Root /mnt
mount -o subvolid=257 /dev/disk/by-label/Root /mnt/home
mount -o subvolid=258 /dev/disk/by-label/Root /mnt/.snapshots
mount -o subvolid=259 /dev/disk/by-label/Root /mnt/var/log
mount -o subvolid=260 /dev/disk/by-label/Root /mnt/var/cache
mount -o subvolid=262 /dev/disk/by-label/Root /mnt/var/lib/portables
mount -o subvolid=263 /dev/disk/by-label/Root /mnt/var/lib/machines

btrfs inspect-internal logical-resolve [numbers from --check-data-csum]

rm /path/to/listed/file

Then and only then was I able to follow the instructions I linked in a previous post of mine: https://wiki.archlinux.org/title/Pacman … an_upgrade

I don't know if my system is actually fully repaired from these actions, but I was able to boot into my drive without using any snapshots and things seem to be working!

Last edited by witchwrenna (2024-09-01 01:34:36)

Offline

Pages: 1

Board footer

Powered by FluxBB