One of LVM VG disks damaged? Couldn't find device with uuid.

knovoselic · 2020-12-07 19:59:58

Hi all. Unfortunately today I couldn't boot my system. I put it into hibernation last night and today I was greeted first with a message that resume hibernation file couldn't be found, and then that my root LVM can't be found/mounted.

I've booted live linux distro to look into it and I'm not sure what happened.
It looks like something happened to one of my physical disks in the LVM and it got corrupted. The LVM shows the following message:

Couldn't find device with uuid CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM.

This UUID does not exist, I've checked everything that I can. After examining my system, it looks like my corrupted disk is /dev/sda. However, I do not remember how I have set up this LVM. Can a whole disk be used as a LVM PV without having it being partitioned? Currently the disk has a GPT layout but no partitions. Is disk without any partitions expected in a LVM setup?
If not, then I guess that something has happened to the partition table and now it's gone. I'll try to recover it using TestDisk (or something else if recommended?), but before doing anything I wanted to ask for help, so that I do not mess it up more.
The first thing I did after noticing the problem was to create a backup of the disk using dd, so I think I can experiment freely, without losing the data. dd cloning was successful, without any I/O errors and the image looks like it contains some data (I have examined first few KB and it looks like data, not just one value or zeros).

Here's some info which I think should be helpful with this problem:

# pvscan
  WARNING: Device for PV CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM not found or rejected by a filter.
  Couldn't find device with uuid CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM.
  PV /dev/sdb3   VG vgMain          lvm2 [<465.03 GiB / 0    free]
  PV [unknown]   VG vgMain          lvm2 [238.47 GiB / 0    free]
  Total: 2 [703.50 GiB] / in use: 2 [703.50 GiB] / in no VG: 0 [0   ]

# pvdisplay
  WARNING: Device for PV CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM not found or rejected by a filter.
  Couldn't find device with uuid CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM.
  --- Physical volume ---
  PV Name               /dev/sdb3
  VG Name               vgMain
  PV Size               <465.03 GiB / not usable <1.01 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              119047
  Free PE               0
  Allocated PE          119047
  PV UUID               z7FIwA-X2Y2-S4zA-DNsg-XHoP-zCjm-I8jQCZ
   
  --- Physical volume ---
  PV Name               [unknown]
  VG Name               vgMain
  PV Size               238.47 GiB / not usable <1.32 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              61049
  Free PE               0
  Allocated PE          61049
  PV UUID               CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM

# lvscan
  WARNING: Device for PV CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM not found or rejected by a filter.
  Couldn't find device with uuid CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM.
  inactive          '/dev/vgMain/lvRoot' [50.00 GiB] inherit
  inactive          '/dev/vgMain/lvHome' [621.50 GiB] inherit
  inactive          '/dev/vgMain/lvSwap' [32.00 GiB] inherit

# lvdisplay
  WARNING: Device for PV CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM not found or rejected by a filter.
  Couldn't find device with uuid CtMQDK-T6Oj-ymff-2VWP-s2DW-VZtM-O08RGM.
  --- Logical volume ---
  LV Path                /dev/vgMain/lvRoot
  LV Name                lvRoot
  VG Name                vgMain
  LV UUID                2s9Cyh-LqlF-HK9q-EaUz-L3iF-AaGd-I7rexj
  LV Write Access        read/write
  LV Creation host, time ant-18.7, 2018-08-08 04:37:46 +0000
  LV Status              NOT available
  LV Size                50.00 GiB
  Current LE             12800
  Segments               2
  Allocation             inherit
  Read ahead sectors     auto
   
  --- Logical volume ---
  LV Path                /dev/vgMain/lvHome
  LV Name                lvHome
  VG Name                vgMain
  LV UUID                a9hdDY-ihSa-wdj3-FCJy-1uCf-jBmD-2uvCHS
  LV Write Access        read/write
  LV Creation host, time ant-18.7, 2018-08-08 04:37:55 +0000
  LV Status              NOT available
  LV Size                621.50 GiB
  Current LE             159104
  Segments               3
  Allocation             inherit
  Read ahead sectors     auto
   
  --- Logical volume ---
  LV Path                /dev/vgMain/lvSwap
  LV Name                lvSwap
  VG Name                vgMain
  LV UUID                VJXY6b-aZTR-HsXa-z42C-pDlr-Ijof-JTt8AN
  LV Write Access        read/write
  LV Creation host, time ferdo, 2019-10-29 09:42:49 +0000
  LV Status              NOT available
  LV Size                32.00 GiB
  Current LE             8192
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto

# blkid
/dev/sdb1: LABEL_FATBOOT="EFI" LABEL="EFI" UUID="4DAA-2AC3" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI System" PARTUUID="d060ff79-a442-4d2f-aabe-ad5a71908f62"
/dev/sdb2: UUID="4DAA-7DFD" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="Linux filesystem" PARTUUID="6012644d-c130-43fc-8b70-33c7243eb9bd"
/dev/sdb3: UUID="z7FIwA-X2Y2-S4zA-DNsg-XHoP-zCjm-I8jQCZ" TYPE="LVM2_member" PARTLABEL="Linux LVM" PARTUUID="3a4e5dd4-b78b-42d0-afd7-962282203acb"
/dev/sdc2: LABEL="Delta" BLOCK_SIZE="512" UUID="0A941FED941FDA51" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="7fc344cd-266c-4880-8bd8-80648a8b38a5"
/dev/sdd1: LABEL="Recovery" BLOCK_SIZE="512" UUID="22FE1D3EFE1D0BA3" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="c581db24-e56b-44b2-800b-726ca87415cb"
/dev/sdd2: UUID="E21D-96B4" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="6a872a4a-9a02-4c73-9fc4-f338ba460fa7"
/dev/sdd4: BLOCK_SIZE="512" UUID="48461E91461E7FBC" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="1b10401a-0953-40ac-95e1-19b7d2f6bf60"
/dev/sdd5: BLOCK_SIZE="512" UUID="7C0AC6F40AC6AB0A" TYPE="ntfs" PARTUUID="b098af6d-ab58-4821-8098-5d12c9fbdee8"
/dev/loop0: TYPE="squashfs"
/dev/loop1: TYPE="squashfs"
/dev/loop2: TYPE="squashfs"
/dev/loop3: TYPE="squashfs"
/dev/sde1: LABEL="Golf" BLOCK_SIZE="512" UUID="3850F8C250F88840" TYPE="ntfs" PARTUUID="7e020d83-01"
/dev/sr0: BLOCK_SIZE="2048" UUID="2020-12-03-10-06-11-00" LABEL="MANJARO_GNOME_202" TYPE="iso9660" PTTYPE="dos"
/dev/sda: PTUUID="92cf67dc-7b50-42af-bf5c-51d330e5cd35" PTTYPE="gpt"
/dev/sdc1: PARTLABEL="Microsoft reserved partition" PARTUUID="3c21de75-5941-4b4e-aaac-bde6ddd84a8b"
/dev/sdd3: PARTLABEL="Microsoft reserved partition" PARTUUID="23e92227-2588-4614-bd9e-e780c3242965"

# fdisk -l
Disk /dev/sda: 238.47 GiB, 256060514304 bytes, 500118192 sectors
Disk model: Samsung SSD 850 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 92CF67DC-7B50-42AF-BF5C-51D330E5CD35


Disk /dev/sdb: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 850 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 45EFCB37-9861-49AE-94A2-05F204BEA4C9

Device       Start       End   Sectors  Size Type
/dev/sdb1     2048   1126400   1124353  549M EFI System
/dev/sdb2  1128448   1538047    409600  200M EFI System
/dev/sdb3  1538048 976773134 975235087  465G Linux LVM


Disk /dev/sdc: 1.82 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: MB2000EAZNL     
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 27900C37-FD1D-406D-A8A3-D79478D2EAC6

Device     Start        End    Sectors  Size Type
/dev/sdc1     34      32767      32734   16M Microsoft reserved
/dev/sdc2  32768 3907026943 3906994176  1.8T Microsoft basic data


Disk /dev/sdd: 232.89 GiB, 250059350016 bytes, 488397168 sectors
Disk model: Samsung SSD 850 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7A557B4B-2B18-4A89-9282-BBD6DFF42FA3

Device         Start       End   Sectors   Size Type
/dev/sdd1       2048   1023999   1021952   499M Windows recovery environment
/dev/sdd2    1024000   1228799    204800   100M EFI System
/dev/sdd3    1228800   1261567     32768    16M Microsoft reserved
/dev/sdd4    1261568 487229164 485967597 231.7G Microsoft basic data
/dev/sdd5  487229440 488394751   1165312   569M Windows recovery environment


Disk /dev/loop0: 72.27 MiB, 75784192 bytes, 148016 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop1: 288.29 MiB, 302297088 bytes, 590424 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop2: 1.56 GiB, 1678585856 bytes, 3278488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop3: 656.31 MiB, 688193536 bytes, 1344128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sde: 111.79 GiB, 120034123776 bytes, 234441648 sectors
Disk model:  External HDD   
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x7e020d83

Device     Boot Start       End   Sectors   Size Id Type
/dev/sde1        2048 234436607 234434560 111.8G  7 HPFS/NTFS/exFAT

Arch Linux

#1 2020-12-07 19:59:58

One of LVM VG disks damaged? Couldn't find device with uuid.

Board footer