[Solved]After dd my 300GB into 1Tb drive I can't use remaining space

SkyTod · 2018-05-23 13:41:48

I had an old 300 GB hard drive and wanted to replace with a new 1TB disk. I just ran:

dd if=/dev/sdy1 of=/dev/sda bs=64K conv=noerror,sync status=progress

I got confused when my space wasn't actually expanded.
lsblk:

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 931,5G 0 disk /home
sdb      8:16   0 167,7G  0 disk 
├─sdb1   8:17   0   255M  0 part /boot
├─sdb2   8:18   0 159,7G  0 part /
├─sdb3   8:19   0     1K  0 part 
└─sdb5   8:21   0   7,8G  0 part [SWAP]

df -h:

Filesystem      Size  Used Avail Use% Mounted on
dev             3,9G     0  3,9G   0% /dev
run             3,9G  1,7M  3,9G   1% /run
/dev/sdb2       158G   43G  107G  29% /
tmpfs           3,9G  180M  3,8G   5% /dev/shm
tmpfs           3,9G     0  3,9G   0% /sys/fs/cgroup
tmpfs           3,9G  224K  3,9G   1% /tmp
/dev/sdb1       247M   55M  181M  24% /boot
/dev/sda        255G  237G  4,6G  99% /home
tmpfs           797M   16K  797M   1% /run/user/120
tmpfs           797M   60K  797M   1% /run/user/1000

There is only 4,6G free space, however, It should be over 500GB.

Where is it? How can I make them available?

Last edited by SkyTod (2018-05-25 02:27:58)

Trilby · 2018-05-23 14:10:00

You copied the partition, filesystem and all, to the new device. This is very different from copying the contents. Dd would not resize the partition or filesystem, you'd have to do this separately.

But more importantly, if that was the actual command, you didn't even copy the existing partition to a partition on the new disk, you wrote it to the block device itself.

I'd suggest starting again. Partition the new drive, create filesystem(s) on it, then copy the data from the old drive over using rsync.

SkyTod · 2018-05-23 14:26:41

I didn't want to copy the contents only. I copied the filesystem.

So, is there no way to sort it out without dd it again? I had to create a separate partition with the exact size and after that expand it with remaining unallocated space in order to get my old hard drive + extra space, right? Because I'm not sure.

Last edited by SkyTod (2018-05-23 14:36:58)

Trilby · 2018-05-23 14:37:53

What? None of the above makes any sense.

SkyTod wrote:

I didn't want to copy the contents only. I copied the filesystem.

Yes, that's what you did, but it's not what you should have done.

SkyTod wrote:

So, is there no way to sort it out without dd it again?

Yes, that's precisely what I suggested, you should not have used dd at all, it is the wrong tool for this task.

SkyTod wrote:

I had to create a separate partition with the exact size and after that expand it with remaining unallocated space in order to get my hard + extra space, right?

What? Are you saying you previously did this? If so, it was completely irrelevant: you wrote the old filesystem directly to the block device overwriting any previous partition table on the 1TB drive.

There should be no resizing necessary:
1) create a partition table on the 1TB drive, you may only want a single partition that takes the full disk
2) create a filesystem on that partition
3) copy the contents of the old disk to the new one with rsync which will be *much* faster than dd as it's copying files, not empty space.

You *might* be able to use a tool like resizefs to resize the filesystem currently on the 1TB drive. I say might as you can't actually expand the partition because there isn't one: you overwrote the partition table so there is no partition to expand. And using resizefs on a filesystem directly on a device node just sounds like a bad idea to me (as does having the filesystem there in the first place).

Last edited by Trilby (2018-05-23 14:40:08)

SkyTod · 2018-05-23 15:14:58

Well. I didn't want to copy my files. Filesystem properties like permissions, mountpoint id, etc are important as well. After dd it worked like before.

Okay, maybe my assumption was wrong. Are you saying that it should be okay using simply rsync and no extra magic required??

ua4000 · 2018-05-23 15:35:00

Since you didn't tell, I will guess you want to copy your entire OS ?
Yes, that's possibel with rsync, see for the required options: https://wiki.archlinux.org/index.php/Rs … tem_backup

Trilby · 2018-05-23 15:51:41

SkyTod wrote:

Well. I didn't want to copy my files. Filesystem properties like permissions, mountpoint id, etc are important as well.

rsync maintains those. Those are file attributes not properties of the filesystem.

edit: this is assuming you use a compatible filesystem. If you rsync from ext4 to fat32 you'll have problems as fat just doesn't have file attributes (or at least not those that ext2/3/4 does). But as long as you have any linux filesystem on the 1TB drive, the file attributes can be preserved (even `cp` can preserve them).

Last edited by Trilby (2018-05-23 15:53:48)

SkyTod · 2018-05-23 17:08:29

ua4000 wrote:

Since you didn't tell, I will guess you want to copy your entire OS ?
Yes, that's possibel with rsync, see for the required options: https://wiki.archlinux.org/index.php/Rs … tem_backup

Not entire, just /home. Thanks for the link, I'll try that way.

Trilby wrote:

SkyTod wrote:
Well. I didn't want to copy my files. Filesystem properties like permissions, mountpoint id, etc are important as well.
edit: this is assuming you use a compatible filesystem. If you rsync from ext4 to fat32 you'll have problems as fat just doesn't have file attributes (or at least not those that ext2/3/4 does). But as long as you have any linux filesystem on the 1TB drive, the file attributes can be preserved (even `cp` can preserve them).

Thanks, I'll give a try. I used to consider dd as the right tools for this kind of tasks.

Scimmia · 2018-05-23 17:13:10

dd can work but not as you used it. Copying a partition to the disk instead of to a partition was mistake #1. Not resizing the filesystem after you copied it was mistake #2.

Trilby · 2018-05-23 17:17:45

Agreed, it can be made to work, but I'd say it's still just the wrong tool for the present goals: by it's nature dd is horribly inefficient (as it copies everything including unused space) and then it requires addition filesystem resizing steps which can be easily avoided.

SkyTod · 2018-05-25 02:26:59

Yeah, rsync did it well. Thanks everebody.

Arch Linux

#1 2018-05-23 13:41:48

[Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#2 2018-05-23 14:10:00

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#3 2018-05-23 14:26:41

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#4 2018-05-23 14:37:53

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#5 2018-05-23 15:14:58

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#6 2018-05-23 15:35:00

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#7 2018-05-23 15:51:41

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#8 2018-05-23 17:08:29

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#9 2018-05-23 17:13:10

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#10 2018-05-23 17:17:45

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

#11 2018-05-25 02:26:59

Re: [Solved]After dd my 300GB into 1Tb drive I can't use remaining space

Board footer