Arch Linux

Dreaming in Code

Member

Registered: 2014-08-16

Posts: 22

[Solved] Ext4 partition becomes partially read only

One month ago I did a resize (shrinking) of a ext4 partition on my removable hard drive, utilizing some commands on the wiki which I cannot remember now, and rebuilding the partition table with parted. Things seemed to be ok, until today I tried to write something into the ext4 partition.

The partition is mounted rw, and I can create multiple empty files on it. However I cannot write any data into the files, it says no space left on device. Old files can be read, and I can delete them.

I saw a df of 10+GiB, and I deleted another ~800MiB but still I cannot write files.

SMART data is good, running e2fsck -fv also claims good.

Do you have any idea what is going on with my partition?

Last edited by Dreaming in Code (2016-10-02 01:05:45)

djgera

Developer

From: Buenos Aires - Argentina

Registered: 2008-12-24

Posts: 723

Website

Re: [Solved] Ext4 partition becomes partially read only

is there any kernel error?. Dump info about the filesystem (tune2fs -l /dev/sdXY) and about the partition (fdisk -l /dev/sdX)

Last edited by djgera (2016-09-30 18:36:17)

Dreaming in Code

Member

Registered: 2014-08-16

Posts: 22

Re: [Solved] Ext4 partition becomes partially read only

dmesg seems alright:

[31199.289597] usb 2-3: new high-speed USB device number 5 using xhci_hcd
[31199.463817] usb-storage 2-3:1.0: USB Mass Storage device detected
[31199.464221] scsi host3: usb-storage 2-3:1.0
[31202.048452] scsi 3:0:0:0: Direct-Access     NORELSYS 106X             2E   PQ: 0 ANSI: 6
[31202.049595] sd 3:0:0:0: [sdc] 976773168 512-byte logical blocks: (500 GB/466 GiB)
[31202.050911] sd 3:0:0:0: [sdc] Write Protect is off
[31202.050915] sd 3:0:0:0: [sdc] Mode Sense: 43 00 00 00
[31202.051101] sd 3:0:0:0: [sdc] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[31202.057700]  sdc: sdc1 sdc2
[31202.058765] sd 3:0:0:0: [sdc] Attached SCSI disk
[31202.573585] EXT4-fs (sdc1): mounted filesystem with ordered data mode. Opts: (null)

But last night among a lot of plugging in and out and trying to write files, I accidentally got a message like 'Unable to expand inode 6434. Delete some EAs or run e2fsck', though e2fsck reported that all is good and I never got it again.

tune2fs:

$ sudo tune2fs -l /dev/sdc1
tune2fs 1.43.3 (04-Sep-2016)
Filesystem volume name:   <none>
Last mounted on:          /run/media/zh/e10cc7b3-7f73-4d10-90b2-be901c8bf63d
Filesystem UUID:          e10cc7b3-7f73-4d10-90b2-be901c8bf63d
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              19660800
Block count:              78643200
Reserved block count:     3932160
Free blocks:              2411972
Free inodes:              19450688
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1005
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Flex block group size:    16
Filesystem created:       Fri Oct 30 14:56:06 2015
Last mount time:          Sat Oct  1 08:49:59 2016
Last write time:          Sat Oct  1 08:53:45 2016
Mount count:              1
Maximum mount count:      -1
Last checked:             Sat Oct  1 01:06:21 2016
Check interval:           0 (<none>)
Lifetime writes:          343 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      a4c369d2-9a71-4ecc-bfcd-13347d247ca1
Journal backup:           inode blocks

fdisk:

$ sudo fdisk -l /dev/sdc
Disk /dev/sdc: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x4b1eb7f8

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sdc1            2048 629147648 629145601   300G 83 Linux
/dev/sdc2       629149696 976771071 347621376 165.8G  7 HPFS/NTFS/exFAT

djgera

Developer

From: Buenos Aires - Argentina

Registered: 2008-12-24

Posts: 723

Website

Re: [Solved] Ext4 partition becomes partially read only

There are some bad things here:

(1) Partition size is 512 bytes smaller than filesystem size 629145601*512 < 78643200*4096
(2) Partition size is not a multiple of 4096 bytes.

Solutions:
(1) With fdisk delete partition 1, then create again, do not worry, no data is destroyed. Default values will be OK, since first sector will be 2048 and last sector will be 629149695.
(2) Run resize2fs, on such filesystem, it will be expand to the maximum value by default.
(3) run e2fsck again.

This is al least to start and keep things in the right size,

Dreaming in Code

Member

Registered: 2014-08-16

Posts: 22

Re: [Solved] Ext4 partition becomes partially read only

@djgera

Thank you! I did what you said and the parition looks good now, although resize2fs did not change block count to 78643456 though it said it will do so at the beginning, it is still 78643200, probably due to some safety checks?

And I found the problem is that ext4 has reserved blocks only for root ( http://askubuntu.com/questions/249387/d … ze-of-home ), so even though I have gigabytes of free spaces I cannot write blocks (Reserved block count: 3932160 > Free blocks: 2411972 from tune2fs report), and normally this won't happen unless a partition is shrinked or something. After tune2fs -m 0 things worked well. Thank you for your help!

djgera

Developer

From: Buenos Aires - Argentina

Registered: 2008-12-24

Posts: 723

Website

Re: [Solved] Ext4 partition becomes partially read only

Nice catch. I did not taken in the account at the first time.

PD: maybe, because the disk is in use, and kernel did not re-read the partition table?

djgera

Developer

From: Buenos Aires - Argentina

Registered: 2008-12-24

Posts: 723

Website

Re: [Solved] Ext4 partition becomes partially read only

The "unused space" is due something related to number of blocks in the last block group (128MiB each bg), the same thing will happens on creation time, looks like there is a minimum...

$ truncate -s $((78643456*2**12)) pepe ; mkfs.ext4 -v -m 0 -F pepe
...
warning: 256 blocks unused.
...
19699200 inodes, 78643200 blocks
...
2400 block groups
...

$ truncate -s $((78643763*2**12)) pepe ; mkfs.ext4 -v -m 0 -F pepe
...
warning: 563 blocks unused.
...
19699200 inodes, 78643200 blocks
...
2400 block groups
...

$ truncate -s $((78643764*2**12)) pepe ; mkfs.ext4 -v -m 0 -F pepe
...
19668992 inodes, 78643764 blocks
...
2401 block groups
...

$ truncate -s $((78643765*2**12)) pepe ; mkfs.ext4 -v -m 0 -F pepe
...
19668992 inodes, 78643765 blocks
...
2401 block groups
...

edit: 78643200 % 32768 = 0 so, it just fit a complete block group, after that a new bg is created, in this case if at least 564 blocks available.
edit2: there are much other variables in the game, on the same total size, if number of inodes changes...unused space change...

Last edited by djgera (2016-10-04 16:22:26)