Arch Linux

rhapsodite

Member

Registered: 2015-07-17

Posts: 2

[SOLVED] Why does such a large "not usable" area exist after vgcreate?

I created LVM2 partition on iSCSI drive as the following (gdisk -l /dev/sdb):

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF00  EFI System
   2         1050624         2099199   512.0 MiB   8300  Linux filesystem
   3         2099200       201326558   95.0 GiB    8E00  Linux LVM

But after pvcreate and vgcreate, pvdisplay shows large "not usable" area:
  --- Physical volume ---
  PV Name               /dev/sdb3
  VG Name               vg1
  PV Size               95.00 GiB / not usable 4.00 GiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              23295
  Free PE               23295
  Allocated PE          0
  PV UUID               ijVBrk-UkIe-3cBJ-27ru-uEEu-UYEI-FBNhLR

In spite of 4MiB PE size, why does large "not usable" area exist?

==== 24 Jul. addtional info.

I tried to compare the cases between iSCSI drive and normal HDD.
On VMware Player 7.1.2, I created a 20GB SCSI virtual disk, allocated a partion of Linux LVM type, pvcreate and vgcreate.

Then the output of pvdisplay was
# pvdisplay /dev/sda1
  --- Physical volume ---
  PV Name               /dev/sda1
  VG Name               vg1
  PV Size               20.00 GiB / not usable 2.98 MiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              5119
  Free PE               5119
  Allocated PE          0
  PV UUID               LeccKh-Wy8J-k2tn-VH5n-nN88-CcmP-Q9DdO1

and the output of pvck was
# pvck /dev/sda1
  Found label on /dev/sda1, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=1044480

Next, I connected the same vritual machine to a 20GB iSCSI drive, and allocated a partion of Linux LVM type, pvcreate and vgcreate.
The result was
# pvdisplay /dev/sdc2
  --- Physical volume ---
  PV Name               /dev/sdc2
  VG Name               vg1
  PV Size               20.00 GiB / not usable 4.00 GiB
  Allocatable           yes
  PE Size               4.00 MiB
  Total PE              4096
  Free PE               4096
  Allocated PE          0
  PV UUID               gtRT6X-VQu7-SQGn-5I1O-ZBN7-2bHO-c1FV31

# pvck /dev/sdc2
  Found label on /dev/sdc2, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=4294962688

Apparently metadata size of iSCSI drive is too large.
What is the reason of this size difference?
Are there any problem to use LVM2 on iSCSI drive?

Last edited by rhapsodite (2015-07-26 08:46:09)

rhapsodite

Member

Registered: 2015-07-17

Posts: 2

Re: [SOLVED] Why does such a large "not usable" area exist after vgcreate?

This was due to large value of optimal_io_size which was one of block device parameters.
By setting data_alignment_detection to 0 in /etc/lvm/lvm.conf, we can turn off  to refer this value.
The result is:

# pvdisplay
  --- Physical volume ---
  PV Name               /dev/sdb2
  VG Name               vg1
  PV Size               47.50 GiB / not usable 2.98 MiB
  Allocatable           yes (but full)
  PE Size               4.00 MiB
  Total PE              12159
  Free PE               0
  Allocated PE          12159
  PV UUID               rXC7Qg-frSj-nzL8-iI6i-0aZn-Z4bn-OPFPAR

# pvck /dev/sdb2
  Found label on /dev/sdb2, sector 1, type=LVM2 001
  Found text metadata area: offset=4096, size=1044480
(After some trials, drive name, size and UUID are different from those of the above post.)

This value is used to determine metadata size of PV (even when we give metadatasize option to pvcreate command).
optimal_io_size parameter is reported from block device, and in my case  4294966784(=0xFFFFFE00) was set from iSCSI drive created on RAID in QNAP NAS.
We can directly see this value by "cat /sys/block/sdX/queue/optimal_io_size".

note:
linux kernel package version 4.1.2-2
lvm2 package version 2.02.125-1

Last edited by rhapsodite (2015-07-26 09:10:55)