You are not logged in.

#1 2013-07-11 02:36:26

w201
Member
Registered: 2012-10-04
Posts: 289

[SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

Hey everyone - so I purchased a cheap hard drive today to backup my system. I cloned the entire drive to the new drive, which has 80Gbs more space. I booted into the new drive and noticed that the additional 80 gigs is not allocated to any of the partitions. This is my first time cloning so I probably F'd something up, even though everything works fine. If you look below, sbd does show 149Gs. How can I regain that space and assign it to my home partition? Thanks!

***OLD***

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda      8:0    0 75.8G  0 disk 
├─sda1   8:1    0   500M  0 part /boot/efi
├─sda2   8:2    0   500M  0 part /boot
├─sda3   8:3    0    20G  0 part /
├─sda4   8:4    0    20G  0 part /var
└─sda5   8:5    0  33.6G  0 part /home

***NEW***

NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sdb      8:0    0 149.1G  0 disk 
├─sdb1   8:1    0   500M  0 part /boot/efi
├─sdb2   8:2    0   500M  0 part /boot
├─sdb3   8:3    0    20G  0 part /
├─sdb4   8:4    0    20G  0 part /var
└─sdb5   8:5    0  33.6G  0 part /home

Last edited by w201 (2013-07-12 09:14:50)

Offline

#2 2013-07-11 12:01:28

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

What commands / program did you use to clone the drive ?

you should be able to use tools like parted / gparted to add the free space to the home partition on the new disk.
(boot from a live usb/cd to use those tools, if you're a bit lucky you will not even suffer data loss.)

an alternative might be to create an sdb6 partition and use lvm2 to combine sdb5 and sdb6 into 1 home.


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#3 2013-07-11 12:08:21

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

If you used a tool like dd, or most other cloning tools, then this is to be expected.  Partitions have sizes, and when you copy partitions, you copy their size.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#4 2013-07-11 12:17:55

w201
Member
Registered: 2012-10-04
Posts: 289

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

Hi Lone_Wolf,

I used...

dd if=/dev/sda of=/dev/sdb bs=4096 conv=notrunc,noerror

I'll try using parted as you suggested. For future reference, using the dd command, is it possible to clone a drive and instruct the computer to assign the additional space to a specific partition so that it's not lost? I didn't see any options like that on the man page.

Last edited by w201 (2013-07-11 12:18:27)

Offline

#5 2013-07-11 12:21:57

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

No.  dd does not create partitions, it doesn't see partitions, it doesn't know anything about partitions or even need the disk to be partitioned.  dd just copies bits from one device to another.

If you use dd to copy a block device, you'll always have to adjust the partition size.

You can partition the target device appropriately first, then dd each partition instead of the whole device in one go.  `dd if=/dev/sda1 of /dev/sdb1 && dd if=/dev/sda2 of=/dev/sdb2` instead of `dd if=/dev/sda of=/dev/sdb` as an example.

Or more likely you'll want to use something like rsync instead of dd.  This has the added bonus of not wasting time copying unused space.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#6 2013-07-11 12:29:26

w201
Member
Registered: 2012-10-04
Posts: 289

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

Trilby wrote:

No.  dd does not create partitions, it doesn't see partitions, it doesn't know anything about partitions or even need the disk to be partitioned.  dd just copies bits from one device to another.

If you use dd to copy a block device, you'll always have to adjust the partition size.

You can partition the target device appropriately first, then dd each partition instead of the whole device in one go.  `dd if=/dev/sda1 of /dev/sdb1 && dd if=/dev/sda2 of=/dev/sdb2` instead of `dd if=/dev/sda of=/dev/sdb` as an example.

Or more likely you'll want to use something like rsync instead of dd.  This has the added bonus of not wasting time copying unused space.

Trilby,

Thanks, that's exactly what I needed to know. I'm heading over to the rsync wiki as I haven't used it before, but I'll give that a try.

Offline

#7 2013-07-11 13:05:41

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

I've done this many times, and I always just use cp -a .

Offline

#8 2013-07-11 13:29:43

w201
Member
Registered: 2012-10-04
Posts: 289

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

Based on what I've read so far on the rsync wiki, if you're backing up regularly, rsync has the advantage of backing up only the files that changed since the last backup...so the first backup takes a while, and subsequent backups are much faster.

**EDIT**

oops, I guess cp can do that too :-)

Last edited by w201 (2013-07-11 13:34:32)

Offline

#9 2013-07-11 13:37:26

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

That's true, but it's relative.  In either case rsync (or cp -a) will be faster than dd.

rysnc on the first backup, or cp -a, will copy every file on the device.  dd will copy ever binary digit, whether it is in a file or not.

I'm not sure if rsync's incremental backup ability would be useful in your case.  Generally this is done with a separate target path for each subsequent backup, so if a file hasn't changed, it just creates a symlink to the existing backup.  If you are overwriting the same target there is nothing to link to, I don't know if rsyn can leave unchanged files alone (perhaps it can, but I've never done this).

Last edited by Trilby (2013-07-11 13:37:49)


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#10 2013-07-11 14:52:22

w201
Member
Registered: 2012-10-04
Posts: 289

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

Basically, I want a a second drive which I can back up to on a weekly or even monthly basis in case my drive fails, as my main drive is pushing 6 years. I know Western Digitals are rock solid, but I have had a few fail in the past. Regardless, I've already recovered the lost space, partitioned my drive and created the file system extensions, so it's ready for backup. I guess I have a few options, just not sure if I should go with cp or rsync at this point.

Offline

#11 2013-07-11 15:26:22

dag
Member
From: US
Registered: 2013-01-20
Posts: 216

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

if interupted rsync can resume from large files

Last edited by dag (2013-07-11 15:29:31)


--------------------------------------
alcoves wonder creates the wonder unto the ages; never lose that.

Offline

#12 2013-07-11 18:28:48

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

I'd use 'rsync -aAHXP' otherwise some things are lost and will not work as expected.


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#13 2013-07-11 20:54:36

w201
Member
Registered: 2012-10-04
Posts: 289

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

I'm going with rsync. There's one thing that's not very clear to me and the wiki doesn't go into much detail. The wiki says for a full system backup use:

# rsync -aAXv /* /path/to/backup/folder --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found}

My drive is already partitioned. I'm confused on how to set the destination path because I have all 5 partitions mounted at /mnt/1, /mnt/2, etc.. That make any sense? ALso, on the first backup, do I leave out the exclude string? Sorry guys, I've had a long day and I'm probably over thinking this too much, but the destination path is confusing to me. How will rsync know exactly where to place every partition unless I tell it exactly where to go?

Last edited by w201 (2013-07-11 20:54:56)

Offline

#14 2013-07-11 22:41:46

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 29,442
Website

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

You have two choices for that.  One, use several rsync commands, one for each partition; or two, mount the partitions under their appropriate names (eg root as /mnt,  boot as /mnt/boot, home /mnt/home, etc) then use /mnt as the path/to/backup.

Those excluded directories should always be excluded - they are not 'on-disk' directories and are recreated at boot.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Offline

#15 2013-07-12 00:36:23

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

tomk wrote:

I've done this many times, and I always just use cp -a .

As mentioned above, this will lose some attributes and break some things. Not much but it will lose information which rsync can conserve (if you use the flags recommended in the wiki on full backup with rsync).

EDIT: https://bbs.archlinux.org/viewtopic.php … 3#p1299083 was the post I had in mind.

Last edited by cfr (2013-07-12 02:31:17)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#16 2013-07-12 01:31:31

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

cfr, nobody mentioned anything of the sort concerning cp -a. Please re-read the relevant posts, and do not spread FUD.

Offline

#17 2013-07-12 02:19:00

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

I interpreted the rsync comment that way. However, perhaps that was not intended. It is, nonetheless, true. I can't currently find the thread where this caused a failure. I think it was ping but I may be wrong about that.

$ getfacl /tmp/a/1
getfacl: Removing leading '/' from absolute path names
# file: tmp/a/1
# owner: <me>
# group: <me>
user::rw-
group::r--
group:between:rwx
mask::rwx
other::r--

$ rsync -aAXv /tmp/a/ /tmp/b

$ getfacl /tmp/b/1 
getfacl: Removing leading '/' from absolute path names
# file: tmp/b/1
# owner: <me>
# group: <me>
user::rw-
group::r--
group:between:rwx
mask::rwx
other::r--

$ cp -a /tmp/a /tmp/c

$ getfacl /tmp/c
getfacl: Removing leading '/' from absolute path names
# file: tmp/c
# owner: <me>
# group: <me>
user::rwx
group::r-x
other::r-x

As you can see, cp -a loses information. In most cases, this will not matter but in some cases, it will. I think I must be wrong about it being ping, though, because it doesn't seem to apply to ping any more. Nonetheless, it is certainly not FUD. Believe me, I come from OS X and this stuff really can matter. The fact that most stuff in Linux does not depend on it does not mean that nothing does even if I cannot now find the thread to prove it can actually break stuff. Moreover, on any system where ACLs are used, cp -a risks breaking permissions configured by the admin (though perhaps in this case the admin can be expected to know that cp -a will do this).

EDIT: The thread I was trying to find includes a comment towards the end where somebody mentions updating the wiki page on full system backup with rsync to include the necessary flags. The original incantation did not preserve all attributes either. The current recommendation preserves everything I've tested for but this is not, obviously, exhaustive. There used to be a standard test for OS X backup applications but I'm not aware of one for Linux. Maybe this is because Linux has not traditionally depended on this stuff whereas not only has OS X so depended, at one point no backup programme preserved even most of the information and many preserved almost none. Also, OS X relies much, much more extensively on extended attributes which are a minefield.

EDIT: That's wrong. So either cp -a does now preserve the information or I'm wrong about what it fails to preserve.

$ getfacl /tmp/c/1 
getfacl: Removing leading '/' from absolute path names
# file: tmp/c/1
# owner: <me>
# group: <me>
user::rw-
group::r--
group:between:rwx
mask::rwx
other::r--

Currently, I'm not sure which...

EDIT: It is capabilities I was thinking of and ping does depend on them but cp -a now seems to preserve capabilities just as rsync -aAX does. Sorry about that.

Last edited by cfr (2013-07-12 02:45:35)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#18 2013-07-12 09:08:18

w201
Member
Registered: 2012-10-04
Posts: 289

Re: [SOLVED] Cloning From an 80G Drive to a 160G Drive - Lost Drive Space

Trilby wrote:

You have two choices for that.  One, use several rsync commands, one for each partition; or two, mount the partitions under their appropriate names (eg root as /mnt,  boot as /mnt/boot, home /mnt/home, etc) then use /mnt as the path/to/backup.

Those excluded directories should always be excluded - they are not 'on-disk' directories and are recreated at boot.

Trilby - That did the trick. This was a good learning experience! Thanks to everyone else who chimed in as well.

Offline

Board footer

Powered by FluxBB