Arch Linux

uroter

Member

Registered: 2021-01-01

Posts: 12

[SOLVED] Rescue deleted partition with LVM

While installing another distro I accidentally deleted a partiton with an LVM logical volume mounted as '/home' for my Arch system
That is, before I had something like this in my 1TB drive:

NAME             FSTYPE      FSVER    LABEL            UUID                                  
sda                                                                                          
├─sda1           vfat        FAT32            
├─sda2                                                                                       
├─sda3
├─sda4            
├─sda5              
├─sda6           LVM2_member LVM2 001                  qcB4jM-tCmm-aVEw-GSkc-ay42-hBqr-zP5trI  # ←390.6G
│ └─mivol-lvroot ext4        1.0                       56ec0169-9067-48cb-b9aa-ea1b7f3bea63
│ └─mivol-lvhome ext4        1.0                       ???  
├─sda7
└─sda8           LVM2_member?
  └─mivol-lvhome ext4 

I have a physical volume in /dev/sda6, where I made 2 logical volumes mounted as '/' and '/home'. Then I expanded the volume group to /dev/sda8, and used this space for expanding the lvhome logical volume.
After the deletion I have this:

NAME             FSTYPE      FSVER    LABEL            UUID                                  
sda                                                                                          
├─sda1           vfat        FAT32                     D09D-8A36                             
├─sda2                                                                                       
├─sda3 
├─sda4
├─sda5
├─sda6           LVM2_member LVM2 001                  qcB4jM-tCmm-aVEw-GSkc-ay42-hBqr-zP5trI
│ └─mivol-lvroot ext4        1.0                       56ec0169-9067-48cb-b9aa-ea1b7f3bea63  
├─sda7
└─sda8           ext4        1.0                       2283c4bf-3fb2-4d8d-9907-c993f86afa8b   # ← New distro's '/', only 75GB

I deleted the original /dev/sda8 with the physical volume, and installed the Linux Mint distro in a new sda8 partition (BUT starting from the end of the drive and only 75GB, so I KNOW it didn't overwrite the original data since there was much more free space beforehand).
Now, of course, I can't boot into my Arch, since after GRUB, there is a message that a partition's uuid for LVM can't be found and drops me a rescue shell. Running 'lvs':

  WARNING: Couldn't find device with uuid CqeNKr-vguf-1RTX-JHGB-qUIH-iOTX-xdQKoX.
  WARNING: VG mivol is missing PV CqeNKr-vguf-1RTX-JHGB-qUIH-iOTX-xdQKoX (last written to /dev/sda8).
  LV     VG    Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  lvhome mivol -wi-----p- 230.31g                                                    
  lvroot mivol -wi-ao---- 185.00g 

My /etc/fstab in lvroot:

# /dev/mapper/mivol-lvroot
UUID=56ec0169-9067-48cb-b9aa-ea1b7f3bea63	/         	ext4      	rw,relatime	0 1

# /dev/sda1
UUID=D09D-8A36      	/boot/efi 	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro	0 2

# /dev/mapper/mivol-lvhome
UUID=92adb380-e7e9-4baf-a035-8846e846cd17	/home     	ext4      	rw,relatime	0 2

I'm trying to rescue this partition, I already backed the drive to a partition in another drive:

ddrescue --force -n /dev/sda /dev/sdc8 mapfile1  # sda partitions weren't mounted when doing this

I'm currently running an analysis with testdisk on the backup:

testdisk /dev/sdc8 

It's gonna take an extra hour I think.

It's my first time rescuing a partition, and firstly I'm wondering if I'm following the right steps. Then if testdisk finds the missing LVM partition, how exactly should I proceed? My guess is booting from a live environment (I'm using the Rescatux distro), running testdisk on /dev/sda, analysing, selecting the missing partition and selecting 'Write', but, would this cause a conflict since there's already a sda8 partition?, and would I need to do any extra rescue, setup or uuid renaming afterwards for LVM to recognize it?
I'd really appreciate some advice, thanks for reading.

Other notes:
- There is no encryption in the LVM volumes
- EFI/GPT system
- There was a hibernation setup in the LV, and after GRUB, the error message mentions something about hibernation (I don't remember if when installing Mint, my Arch was hibernating but I doubt it)
- I'm currently running testdisk from Linux Mint
- ddrescue's mapfile didn't report any bad blocks

Last edited by uroter (2022-08-18 00:30:34)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,647

Re: [SOLVED] Rescue deleted partition with LVM

in order to undelete a LVM volume, you have to check LVM metadata backups / archives, normally stored under /etc/lvm/

it's also stored in each PV, for a cursory check try `head -c 1M /dev/PV | strings -n 32 -w`, it should have stuff like `# Generated by LVM2 version 2.03.16 <time date>` - you need the latest one that still had the definiton of your home LV

if this is SSD and you have issue_discards=1 in your lvm.conf, then data will already be discarded after lvremove and there is no way to recover

EDIT:

sorry, if it's just about the partition table, can you show `parted /dev/disk unit s print free`? you'd have to re-create the missing partition at the correct offset.

Last edited by frostschutz (2022-08-17 19:44:51)

uroter

Member

Registered: 2021-01-01

Posts: 12

Re: [SOLVED] Rescue deleted partition with LVM

Hello frostschutz, it is a hard drive, not SSD.

$ sudo parted /dev/sda unit s print free
Model: ATA ST1000LM035-1RK1 (scsi)
Disk /dev/sda: 1953525168s
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start        End          Size        File system  Name                          Flags
        34s          2047s        2014s       Free Space
 1      2048s        534527s      532480s     fat32        EFI System Partition          boot, esp
 2      534528s      567295s      32768s                   Microsoft reserved partition  msftres
 3      567296s      323196927s   322629632s  ntfs         Basic data partition          msftdata
 7      323196928s   324016127s   819200s
        324016128s   941203455s   617187328s  Free Space
 8      941203456s   1098981375s  157777920s  ext4
 6      1098981376s  1918181375s  819200000s                                             lvm
 4      1918181376s  1920188415s  2007040s    ntfs         Basic data partition          hidden, diag
 5      1920188416s  1953511423s  33323008s   ntfs         Basic data partition          hidden, msftdata
        1953511424s  1953525134s  13711s      Free Space

testdisk report (at 72%):

TestDisk 7.1, Data Recovery Utility, July 2019
Christophe GRENIER <grenier@cgsecurity.org>
https://www.cgsecurity.org

Disk /dev/sdb8 - 1529 GB / 1424 GiB - CHS 1458176 64 32
     Partition               Start        End    Size in sectors
 P EFI System                  2048     534527     532480 [EFI System Partition] [NO NAME]
 P MS Data                   567296  323196927  322629632 [Disco local]
 P Linux filesys. data    323196928  323201023       4096
 P Linux LVM              324016128  533731327  209715200
 P Linux filesys. data    941203456 1098981375  157777920
 D Linux LVM             1098981376 1918181375  819200000
 D MS Data               1886865409 1920188416   33323008
 D MS Data               1916174337 1918181376    2007040
 D MS Data               1918181376 1920188415    2007040 [Windows RE Tools]
>D MS Data               1920188416 1953511423   33323008 [Disco local]
 D MS Data               1920188416 1953523711   33335296 [Disco local]
 D MS Data               1953523711 1986859006   33335296

Last edited by uroter (2022-08-17 20:02:52)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,647

Re: [SOLVED] Rescue deleted partition with LVM

well, with any luck that `Start 324016128s End 941203455s Size  617187328s  Free Space` will be your missing PV. testdisk also found a LVM header at this offset, so maybe... one way to find out, so good luck

uroter

Member

Registered: 2021-01-01

Posts: 12

Re: [SOLVED] Rescue deleted partition with LVM

I do suspect it is there, and thanks, but do you think the steps I mentioned are fine?

My guess is booting from a live environment (I'm using the Rescatux distro), running testdisk on /dev/sda, analysing, selecting the missing partition and selecting 'Write', but, would this cause a conflict since there's already a sda8 partition?, and would I need to do any extra rescue, setup or uuid renaming afterwards for LVM to recognize it?

frostschutz

Member

Registered: 2013-11-15

Posts: 1,647

Re: [SOLVED] Rescue deleted partition with LVM

I'd just go with parted here... `mkpart lvm 324016128s 941203455s` and hope for the best...

the partition number will be different, but as long as the LVM metadata is still there, it won't matter

if you're dualbooting with windows, also set the lvm flag for the partition ( set <number> lvm on ) or such or windows might try to format it

Last edited by frostschutz (2022-08-17 20:29:51)

uroter

Member

Registered: 2021-01-01

Posts: 12

Re: [SOLVED] Rescue deleted partition with LVM

I ended up going with my original plan and it worked!! (although a few things got misconfigured). testdisk found the deleted partition from 324016129 to 533731327 (100GB) and was able to write the new partition table, when I booted GRUB (the one of my Arch system) my lvhome LV was automatically recognized, and everything in my /home was there.

However, for some reason testdisk didn't find the 'Microsoft reserved partition' that was already there, and it marked the EFI partition and the Linux Mint partition as 'MS Data', not 'EFI System' and 'Linux filesys. data'.
Of course as a result, it didn't boot correctly, and Mint's GRUB can't boot Mint right now. I had to enter my firmware's bootloader and select my Arch's grubx64.efi. But those are easy fixes I think, just have to change the partition types, and I haven't used Windows in years, and wanted to get rid of Mint's GRUB anyway (instead I write a new entry to Arch's GRUB).

Thanks frostschutz for the LVM flag advice, I don't think I had that flag all this time, it could have been terrible. And yeah I think `mkpart lvm 324016128s 533731327s` would have been enough and better (I forgot that "deleting" a partition just erases the entry from the partition table and didn't want to risk anything). I'll mark this solved.