You are not logged in.

#1 2023-02-18 22:28:25

vestingz
Member
Registered: 2020-12-03
Posts: 9

[SOLVED] I/O error on root partition; corrupt superblock

After booting, I end up in a maintenance shell, stating that due to I/O errors on my root partition I need to manually run `fsck` on it. This however fails due to a corrupt superblock. It is suggested to find and alternate superblock to use for `fsck`. As `dumpe2fs` and `mke2fs` however are both not available in the maintenance shell to locate backup superblocks, I am stuck there and have not proceeded forward yet.

It might be noteworthy that this partition/SSD does not hold any absolutely relevant data; important data is stored in a ZFS storage pool consisting of three HDDs and another HDD not being part of that pool. I would be very happy to rescue some config files from that borked root filesystem.

My questions are:

1. What is the advisable procedure to try to rescue that data from that partition? As of yet, I would boot a live system and try to clone the disk to an image file on another external hard drive for further procedures, like so:

dd if=/dev/sda of=/mnt/sda-image.img bs=64K conv=noerror,sync status=progress

How do I work on this image then? Do I just mount it with `mount`?

2. Assuming that the SSD is probably broken, is it probable to be able to fix the filesystem enough to get the system booted and at least export the ZFS pool so I can import it on a new install?

3. If fixing is not possible/not feasible, is it safe to import a ZFS pool on a new install without it being exported first?

4. If importing is not safe, is there an advisable way to export that ZFS pool from within a live system?

Last edited by vestingz (2023-03-04 20:10:56)

Offline

#2 2023-02-18 23:03:08

seth
Member
Registered: 2012-09-03
Posts: 49,941

Re: [SOLVED] I/O error on root partition; corrupt superblock

1. you might need dd_rescue; "mount -o loop …"

2. "the SSD is probably broken, is it probable to be able to fix the filesystem" …
You'll need a sane superblock and then hope that not too many sectors are compromised
Did you check the https://wiki.archlinux.org/title/SMART data of the drive?

3/4. https://wiki.archlinux.org/title/ZFS#Ex … orage_pool

If a storage pool is to be used on another system, it will first need to be exported. It is also necessary to export a pool if it has been imported from the archiso as the hostid is different in the archiso as it is in the booted system. The zpool command will refuse to import any storage pools that have not been exported. It is possible to force the import with the -f argument, but this is considered bad form.

Any attempts made to import an un-exported storage pool will result in an error stating the storage pool is in use by another system. This error can be produced at boot time abruptly abandoning the system in the busybox console and requiring an archiso to do an emergency repair by either exporting the pool, or adding the zfs_force=1 to the kernel boot parameters (which is not ideal).

Do you have a sample of the IO errors?
Because if those are (all) bus errors, drive and filesystem might actually be fine.

Online

#3 2023-02-18 23:04:05

frostschutz
Member
Registered: 2013-11-15
Posts: 1,409

Re: [SOLVED] I/O error on root partition; corrupt superblock

don't use conv=noerror,sync, it corrupts data

use ddrescue

if nothing can be done about the filesystem, you can grab plaintext files with photorec, or even `strings -w | grep -C 1000 someuniquestring`

Online

#4 2023-03-04 20:10:15

vestingz
Member
Registered: 2020-12-03
Posts: 9

Re: [SOLVED] I/O error on root partition; corrupt superblock

Thank you both for valuable info and sorry for my late reply.

I solved the problem like so:

1. Boot Arch live USB.

2. Create image of the failing disk:

 ddrescue --no-trim --no-scrape /dev/sdb /mnt/backup/image.img rescue.map

3. On another system, mount that image to the next free loopback device:

 losetup -f -P /path/to/image.img 

4. Run dumpe2fs on that partition to identify next superblock:

 sudo dumpe2fs /dev/loop0p3 | grep super 

5. Fix filesystem with alternate superblock:

 sudo e2fsck -b /dev/loop0p3 

6. Mount partition:

 sudo mount /dev/loop0p3 /mnt/point 

Now the filesystem should be available under /mnt/point with all files in place. Note that I was lucky as I could skip trimming and scraping phase with ddrescue.

Offline

#5 2023-03-17 09:29:35

d.ALT
Member
Registered: 2019-05-10
Posts: 914

Re: [SOLVED] I/O error on root partition; corrupt superblock

vestingz wrote:

Note that I was lucky as I could skip trimming and scraping phase with ddrescue.

Interesting. How could you check image.img is OK?[1] Do you feel confortable you made a good dump from the orginal? This could help a lot future users saving time during rescues!

[1] it seems you skipped important steps!

ddrescue wrote:

3) (Second phase; Trimming) Trimming retries sector by sector the edges of the large block reads failed during the copying phase. Trimming is done in one pass as follows. For each non-trimmed block, read forwards one sector at a time from the leading edge of the block until a bad sector is found. Then read backwards one sector at a time from the trailing edge of the block until a bad sector is found. Then mark the bad sectors found (if any) as bad-sector, and mark the rest of the block as non-scraped without trying to read it. If any edge is already adjacent to a bad sector, it is considered as already trimmed and is not trimmed again.

4) (Third phase; Scraping) Scrape together, sector by sector, the data not recovered by the copying or trimming phases. Scraping is done in one pass. Each non-scraped block is read forwards, one sector at a time. Any bad sectors found are marked as bad-sector.


(https://www.gnu.org/software/ddrescue/m … anual.html)

Last edited by d.ALT (2023-03-17 09:30:11)


<49,17,III,I>    Fama di loro il mondo esser non lassa;
<50,17,III,I>    misericordia e giustizia li sdegna:
<51,17,III,I>    non ragioniam di lor, ma guarda e passa.

Offline

Board footer

Powered by FluxBB