Ways to create a filesystem image

cafe · 2017-01-11 20:50:02

Hello arch users,

I'm trying to create a single file disk image so that I can replicate a fully configured system to 16 other computers on the lab. My goal is to have something similar to the arch installer image, but instead of using it to install archlinux I'll rather copy the image to 16 other hard drives using dd (or any similar tool).

The reason I've opened this topic is to discuss ways to replicate a system to other computers. I can imagine that this is a fairly common admin task in small to medium size labs. The challenges I'm facing are basically ways to copy the bootloader and the disk partition table.

Here's what I'm testing :

I start by creating a single file with approximately the same size of the source system (partition table + bootloader + boot partition + swap + system partitions like. root, home, usr, ..)

cafe@arch ~ % dd if=/dev/zero of=sys.img bs=1M count=16000 status=progress # supose 16GB of used space (from df -h)

Create a loop device to manipulate the image file like a block device

cafe@arch ~ % sudo losetup /dev/loop0 sys.img

Then partition the image similarly to the source system (supose a UEFI system, but could by a BIOS one)

cafe@arch ~ % sudo parted /dev/loop0
(parted) mklabel gpt
(parted) mkpart primary fat32 1MB 100MB
(parted) mkpart primary linux-swap 100MB 2GB
(parted) mkpart primary btrfs 2GB 100%
(parted) name 1 boot
(parted) name 2 swap
(parted) name 3 root
(parted) set 1 boot on

Make filesystems and mount somewhere

cafe@arch ~ % sudo mkfs.fat -F32 /dev/loop0p1
cafe@arch ~ % sudo mkswap /dev/loop0p2
cafe@arch ~ % sudo mkfs.btrfs -L arch /dev/loop0p3
cafe@arch ~ % mkdir targetfs
cafe@arch ~ % sudo mount /dev/loop0p3 
cafe@arch ~ % sudo mkdir targetfs/boot
cafe@arch ~ % sudo mount /dev/loop0p1 targetfs/boot

Use rsync to move everything to targetfs

cafe@arch ~ % sudo rsync -aAX --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found","/home/cafe/sys.img","/home/cafe/targetfs"} / /home/cafe/targetfs

With this, I'll have a file (sys.img) that I can replicate on other computers, but it lacks the bootloader (if it was a BIOS based system). Also, the partition table is not exactly the same.

For each computer, I'll later regenerate UUIDs, hostname and expand the filesystem to use the full target drive (such as a 500GB HD, for instance).

In my opinion, there has to be simpler ways to do this. I would like the system image to be as small as possible, carrying only necessary files and not the full disk dd img.

Does anyone knows how the archiso installation image is created?

teateawhy · 2017-01-11 21:40:03

Your problem is not about filesystem images, it is about administrating computers.
You'd better reasearch the whole configuration management/immutable infrastructure/devops topic and then consider if filesystem images are really what you want.

Alber · 2017-01-11 21:41:47

Just several ideas. Think of them as a brainstorming.

You can do a dd just for the bootloader, this would be the MBR and maybe an extra space until the first partition. A few MBytes.

rsync for the rest.

But you will have the fstab problem. But, perhaps, using labels would avoid the problem. This way, all computers would must be partitioned with the same labels.

cafe · 2017-01-11 22:46:11

teateawhy wrote:

Your problem is not about filesystem images, it is about administrating computers.
You'd better reasearch the whole configuration management/immutable infrastructure/devops topic and then consider if filesystem images are really what you want.

Wow, that's a lot of stuff to process. Any clues on where to start? If you know any management software or web framework, I would be pleased to know your recommendations.

Alber wrote:

But you will have the fstab problem.

That's true. I'll consider using labels on my source system. Thank you!

Last edited by cafe (2017-01-11 22:46:33)

cafe · 2017-01-13 11:56:17

Where exactly is grub in a BIOS system? I mean.. in "dd" terms..

WorMzy · 2017-01-13 12:26:29

See https://wiki.archlinux.org/index.php/GR … ll_to_disk

cafe · 2017-01-13 19:37:21

WorMzy wrote:

See https://wiki.archlinux.org/index.php/GR … ll_to_disk

Thank you. I've tried in two different ways and both worked :

Installing grub on the loop device

# Using /mnt as a mount base point
cafe@arch ~ % sudo grub-install --target=i386-pc --debug --boot-directory=/mnt/boot /dev/loop0

Or by copying the first 512 bytes of the hardrive. In my case it was a BIOS system.

cafe@arch ~ % sudo dd if=/dev/sda of=/dev/loop0 bs=512 count=1

I think the dd way is more efficient but more error-prone. Anyway.. Now I'm interested in keeping the client systems in sync, but I guess that's a discussion for another forum topic.

Arch Linux

#1 2017-01-11 20:50:02

Ways to create a filesystem image

#2 2017-01-11 21:40:03

Re: Ways to create a filesystem image

#3 2017-01-11 21:41:47

Re: Ways to create a filesystem image

#4 2017-01-11 22:46:11

Re: Ways to create a filesystem image

#5 2017-01-13 11:56:17

Re: Ways to create a filesystem image

#6 2017-01-13 12:26:29

Re: Ways to create a filesystem image

#7 2017-01-13 19:37:21

Re: Ways to create a filesystem image

Board footer