You are not logged in.

#1 2021-05-14 13:26:17

LoneWolfEthan
Member
Registered: 2018-06-19
Posts: 2

btrfs multidisk volume unmountable after 2nd HDD gets click of death

So there I was, happily using partitionless btrfs on my 2 TB drive. I remembered I had a 1 TB drive lying around and decided to add it to the volume for some extra capacity, as I was running out of storage. So, I ran 'btrfs device add /dev/sdc /srv/nfs && btrfs balance start /srv/nfs' and waited for it to complete...

A month later, the 1 TB HDD I added started clicking, wouldn't show up in the system. Regrettably, I don't have a backup. 'btrfs balance' seems to have moved the tree root to the new drive, if I'm understanding the errors I'm getting...

# btrfs filesystem show /dev/sdb
bad tree block 2106233978880, bytenr mismatch, want=2106233978880, have=0
Couldn't read tree root
Label: 'Storage'  uuid: 6d044e2a-c410-468e-97b1-e3bc86966538
        Total devices 2 FS bytes used 1.75TiB
        devid    1 size 1.82TiB used 1.33TiB path /dev/sdb
        *** Some devices missing

# btrfs check /dev/sdb
Opening filesystem to check...
bad tree block 2106233978880, bytenr mismatch, want=2106233978880, have=0
Couldn't read tree root
ERROR: cannot open file system

Is there anything I can do to recover whatever's still on my 2 TB drive? I can't use check, rescue, or recover; nothing works.

Offline

#2 2021-05-14 20:07:03

xerxes_
Member
Registered: 2018-04-29
Posts: 1,064

Re: btrfs multidisk volume unmountable after 2nd HDD gets click of death

Can you mount it, even read only, from archiso to copy files from it? And you can always do raw image of it (if you have place for it on other drive(s)), so you can then experiment and recover if something goes wrong.

Offline

#3 2021-05-14 22:24:04

LoneWolfEthan
Member
Registered: 2018-06-19
Posts: 2

Re: btrfs multidisk volume unmountable after 2nd HDD gets click of death

Sadly, I don't have a way of mounting it, even when mounting with any combination of the rescue mount options.

# mount /dev/sdb /mnt -t btrfs -o ro,degraded,rescue=usebackuproot,rescue=nologreplay,notreelog
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/sdb, missing codepage or helper program, or other error.

dmesg output for the mount:

[   38.802851] BTRFS info (device sdb): allowing degraded mounts
[   38.802859] BTRFS info (device sdb): trying to use backup root at mount time
[   38.802863] BTRFS info (device sdb): disabling log replay at mount time
[   38.802866] BTRFS info (device sdb): disabling tree log
[   38.802868] BTRFS info (device sdb): using free space tree
[   38.802869] BTRFS info (device sdb): has skinny extents
[   38.825208] BTRFS warning (device sdb): devid 2 uuid 03763036-ab65-4627-ae8f-c4ed8b8935d2 is missing
[   38.826584] BTRFS warning (device sdb): couldn't read tree root
[   38.845083] BTRFS warning (device sdb): failed to read root (objectid=4): -5
[   38.859399] BTRFS warning (device sdb): failed to read root (objectid=4): -5
[   38.868029] BTRFS warning (device sdb): failed to read root (objectid=4): -5
[   38.870658] BTRFS error (device sdb): open_ctree failed

Additional debugging info:

# btrfs-find-root /dev/sdb
Couldn't read tree root
Superblock thinks the generation is 17798
Superblock thinks the level is 0
Well block 2956098732032(gen: 17797 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956098715648(gen: 17796 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956098322432(gen: 17795 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956042452992(gen: 17794 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956399460352(gen: 17790 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956399443968(gen: 17789 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956398690304(gen: 17788 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956314411008(gen: 17787 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956294504448(gen: 17786 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956173295616(gen: 17785 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956171444224(gen: 17784 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956171034624(gen: 17783 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956170625024(gen: 17782 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956162203648(gen: 17781 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956161466368(gen: 17780 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956153749504(gen: 17779 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956088836096(gen: 17777 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956389302272(gen: 17764 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956387532800(gen: 17763 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956385878016(gen: 17762 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956353044480(gen: 17760 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956325044224(gen: 17759 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956324667392(gen: 17758 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956324257792(gen: 17757 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956322783232(gen: 17756 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956321226752(gen: 17755 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956320620544(gen: 17754 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956320096256(gen: 17753 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956319522816(gen: 17752 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956318834688(gen: 17751 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956318228480(gen: 17750 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956316557312(gen: 17749 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956312854528(gen: 17745 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956349718528(gen: 17726 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957082836992(gen: 15919 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957079101440(gen: 15918 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957075398656(gen: 15917 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957070417920(gen: 15915 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957059538944(gen: 15909 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957056884736(gen: 15908 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957052116992(gen: 15906 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957047988224(gen: 15901 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957042696192(gen: 15900 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957039583232(gen: 15899 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957035864064(gen: 15897 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957034176512(gen: 15893 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957031931904(gen: 15891 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957024051200(gen: 15890 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957020151808(gen: 15888 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957017497600(gen: 15884 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957016875008(gen: 15883 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957008125952(gen: 15882 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957004881920(gen: 15881 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2957004849152(gen: 15880 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956991152128(gen: 15874 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956959956992(gen: 15872 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956978454528(gen: 15871 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956956024832(gen: 15870 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956953993216(gen: 15867 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956953452544(gen: 15866 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956948652032(gen: 15865 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956946014208(gen: 15864 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956942508032(gen: 15863 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956939444224(gen: 15858 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956932759552(gen: 15856 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956929040384(gen: 15855 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956923027456(gen: 15854 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956920635392(gen: 15852 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956919586816(gen: 15848 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956914655232(gen: 15847 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956911771648(gen: 15846 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956906201088(gen: 15845 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956902809600(gen: 15840 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956901826560(gen: 15838 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956895141888(gen: 15837 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956887195648(gen: 15836 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956892553216(gen: 15835 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956876070912(gen: 15829 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956874104832(gen: 15828 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956870647808(gen: 15827 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956864569344(gen: 15821 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956861915136(gen: 15820 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956858359808(gen: 15819 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956853395456(gen: 15818 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956848709632(gen: 15813 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956813942784(gen: 15811 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956802277376(gen: 15807 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956801097728(gen: 15804 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956800868352(gen: 15803 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956799164416(gen: 15801 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956784631808(gen: 15800 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956790464512(gen: 15799 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956784123904(gen: 15798 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956773867520(gen: 15793 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956771524608(gen: 15792 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956765577216(gen: 15791 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956762726400(gen: 15786 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956762202112(gen: 15785 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956761104384(gen: 15784 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956757073920(gen: 15783 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956751732736(gen: 15782 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956740132864(gen: 15774 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956737429504(gen: 15773 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956685688832(gen: 15756 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956669517824(gen: 15740 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956661997568(gen: 15729 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956656967680(gen: 15728 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956641583104(gen: 15722 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956611158016(gen: 15708 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956541591552(gen: 15685 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956515082240(gen: 15616 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956517507072(gen: 15613 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956514295808(gen: 15612 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956494733312(gen: 15609 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956493275136(gen: 15605 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956489555968(gen: 15603 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956481806336(gen: 15602 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956480577536(gen: 15600 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956472647680(gen: 15593 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956458229760(gen: 15585 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956449382400(gen: 15584 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956446941184(gen: 15579 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956414582784(gen: 15542 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956400410624(gen: 13710 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956400394240(gen: 13709 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956399984640(gen: 13708 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956403834880(gen: 13494 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956403818496(gen: 13493 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
Well block 2956410339328(gen: 11535 level: 0) seems good, but generation/level doesn't match, want gen: 17798 level: 0
# btrfs inspect-internal dump-super /dev/sdb
superblock: bytenr=65536, device=/dev/sdb
---------------------------------------------------------
csum_type               0 (crc32c)
csum_size               4
csum                    0x19476619 [match]
bytenr                  65536
flags                   0x1
                        ( WRITTEN )
magic                   _BHRfS_M [match]
fsid                    6d044e2a-c410-468e-97b1-e3bc86966538
metadata_uuid           6d044e2a-c410-468e-97b1-e3bc86966538
label                   Storage
generation              17798
root                    2106233978880
sys_array_size          129
chunk_root_generation   17794
root_level              0
chunk_root              3827920011264
chunk_root_level        1
log_root                0
log_root_transid        0
log_root_level          0
total_bytes             3000603820032
bytes_used              1919486611456
sectorsize              4096
nodesize                16384
leafsize (deprecated)   16384
stripesize              4096
root_dir                6
num_devices             2
compat_flags            0x0
compat_ro_flags         0x3
                        ( FREE_SPACE_TREE |
                          FREE_SPACE_TREE_VALID )
incompat_flags          0x171
                        ( MIXED_BACKREF |
                          COMPRESS_ZSTD |
                          BIG_METADATA |
                          EXTENDED_IREF |
                          SKINNY_METADATA )
cache_generation        0
uuid_tree_generation    17798
dev_item.uuid           1a6f85b6-09c5-4f29-9008-9c6376c65416
dev_item.fsid           6d044e2a-c410-468e-97b1-e3bc86966538 [match]
dev_item.type           0
dev_item.total_bytes    2000398934016
dev_item.bytes_used     1461429731328
dev_item.io_align       4096
dev_item.io_width       4096
dev_item.sector_size    4096
dev_item.devid          1
dev_item.dev_group      0
dev_item.seek_speed     0
dev_item.bandwidth      0
dev_item.generation     0

Offline

Board footer

Powered by FluxBB