You are not logged in.
Pages: 1
I have a zfs pool of 4 disks in a raidz-2 configuration in a media server. (The Arch system is on a different drive, the zfs pool is used for data storage only).
The pool was originally created with 4x2Tb drives, but I have been replacing them with 4Tb as they failed. The pool how has 4xTb drive, but it refuses to grow to accommodate the new available storage.
I used the standard procedure:
1. Replace drives one by one with "zpool replace pool old_drive new_drive", then waiting for the resilvering to complete.
2. Once all four were replaced, I turn autoexpand off and on again with "zpool set autoexpand=off[on] pool"
However, the pool size did not grow:
[stefano@polus ~]$ zpool status -c size
pool: zfsdatapool
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: resilvered 1.69T in 20:53:28 with 0 errors on Mon Aug 23 20:20:16 2021
config:
NAME STATE READ WRITE CKSUM size
zfsdatapool ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
sda ONLINE 0 0 0 3.6T
wwn-0x5000c500b2d7861e ONLINE 0 0 0 3.6T
wwn-0x5000c50079420af9 ONLINE 0 0 0 3.6T
wwn-0x50014ee2bbdb66cb ONLINE 0 0 0 3.6T
errors: No known data errors
[stefano@polus ~]$ zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
zfsdatapool 7.25T 6.97T 286G - 7.25T - 96% 1.00x ONLINE -With 4x4Tb (nominal) disks in a radiz2 configuration I would expect to see approximately 11 to to 12 Tb of available space, not 7.25Tb.
The four drives have no particular partitioning scheme---they were formatted by zfs from there original (new) condition. Here is the layout (sdb is the ssd drive holding the Arch system):
[stefano@polus ~]$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 3.6T 0 disk
├─sda1 8:1 0 3.6T 0 part
└─sda9 8:9 0 8M 0 part
sdb 8:16 0 111.8G 0 disk
├─sdb1 8:17 0 1M 0 part
└─sdb2 8:18 0 111.8G 0 part /
sdc 8:32 0 3.6T 0 disk
├─sdc1 8:33 0 3.6T 0 part
└─sdc9 8:41 0 8M 0 part
sdd 8:48 0 3.6T 0 disk
├─sdd1 8:49 0 3.6T 0 part
└─sdd9 8:57 0 8M 0 part
sde 8:64 0 3.6T 0 disk
├─sde1 8:65 0 3.6T 0 part What am I doing wrong?
Last edited by stefano (2021-08-31 16:07:53)
Offline
If the vdev was configured as raidz (raidz1): One disk redundancy, you get three disk worth of data out of four disks total.
If the vdev was configured as raidz2: Two disk redundancy, you get two disk worth of data out of four disks total.
7.25TB data space from 4x4TB in raidz2 seems to be correct.
Offline
My bad, I got confused after several attempts and reading countless zfs pages. But the problem is real, in spite of my clumsiness in describing it. Here is a second attempt:
Facts:
1. The zfs pool was initially created with 4x2TB disks in a raidz-2 configuration, i.e. with two extra disks.
2. The total capacity was around 3.6T, as it should have been (about 50% of total)
3. Disks were progressively replaced to 4x4TB
4. Autoexpand feature was set to on
Problem:
4. Capacity should be about 50%, or between 7TB and 8TB, but it is still stuck at 3.6TB
zpool output:
[stefano@polus ~]$ zpool status -c size
pool: zfsdatapool
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(5) for details.
scan: resilvered 1.69T in 20:53:28 with 0 errors on Mon Aug 23 20:20:16 2021
config:
NAME STATE READ WRITE CKSUM size
zfsdatapool ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
wwn-0x5000c50079420edf ONLINE 0 0 0 3.6T
wwn-0x5000c500b2d7861e ONLINE 0 0 0 3.6T
wwn-0x5000c50079420af9 ONLINE 0 0 0 3.6T
wwn-0x50014ee2bbdb66cb ONLINE 0 0 0 3.6T
errors: No known data errorsSo, 4x3.6 TB disks in raidz-2 should give me ~7.5 TB total, as per zpool list
[stefano@polus ~]$ zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
zfsdatapool 7.25T 6.95T 306G - 7.25T - 95% 1.00x ONLINE -
so far so good, but unfortunately df disagrees:
[stefano@polus ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
dev 3.9G 0 3.9G 0% /dev
run 3.9G 896K 3.9G 1% /run
/dev/sdb2 110G 22G 83G 21% /
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 0 3.9G 0% /tmp
zfsdatapool 3.5T 3.4T 36G 99% /data
tmpfs 793M 0 793M 0% /run/user/1001and zfs list concurs:
[stefano@polus ~]$ zfs list
NAME USED AVAIL REFER MOUNTPOINT
zfsdatapool 3.37T 36.0G 3.37T /dataAt this point I'm utterly confused. Why am I not seeing the whole capacity "zpool list" reports in both df's and zfs's outputs? More substantially, how much capacity do I have? I have tried unmounting/exporting / reimporting the pool to no avail. Neither did rebooting. I found an old github report (https://github.com/openzfs/zfs/issues/3953) on a similar issue that attributed the problem to a stale file handle that a regular reboot should solve. Not so in my case.
Offline
I don't know when you turned "on" autoexpand. But my guess is that it has to be done before you start replacing the disks.
You should probably read up on the following command (man zpool-online): zpool online [-e] pool device...
If the currently stored data is crucial, make sure you have a backup
Also unless you have reasons not to, you should attend to the zpool status message about upgrading feature sets.
Offline
I don't know when you turned "on" autoexpand. But my guess is that it has to be done before you start replacing the disks.
You should probably read up on the following command (man zpool-online): zpool online [-e] pool device...
If the currently stored data is crucial, make sure you have a backup
Also unless you have reasons not to, you should attend to the zpool status message about upgrading feature sets.
Ah, offlining/onlining all the devices in order did the trick, thanks a lot. I am not sure why autoexpand did not work---the online docs I consulted instructed to set it to on *after* the resilvering. I guess they were wrong. In general, it seems growing zraid pools is not a very common occurrence. Oracle has a very single line on their page on this topic:
"To expand a LUN, use the zpool online –e command. By default, a LUN that is added to a pool is not expanded to its full size unless the autoexpand pool property is enabled. If the property is disabled, use the command to expand the LUN automatically. You can run the command regardless of whether the LUN is offline or already online."
which seems to suggest the autoexpand option method should have worked and that offlining the devices was actually redundant (but it didn't hurt, thankfully).
Offline
Pages: 1