Install 2nd instance of Arch Linux without internet connection

ccy · 2013-05-22 02:22:07

I have successfully install a first instance of Arch Linux using Arch Linux ISO + pacstrap + pacman with internet connection.

I have updated to latest package database list (/var/lib/pacman/sync) and package cache files (/var/cache/pacman/pkg) in first instance of Arch Linux with "pacman -Swyu".

Next, I would like to install 2nd instance of Arch Linux without using internet connection.

I share the "/var/lib/pacman/sync" and "/var/cache/pacman/pkg" via NFS from first Arch Linux instance.

I boot the 2nd instance with Arch Linux ISO and attempt to install without internet connection. I mount the two NFS shares from 1st instance to "/var/cache/pacman/pkg" and "/mnt/var/lib/pacman/sync" respectively. I execute

# pacstrap -i -c /mnt"

to start installation.

I found it is almost impossible with current implementation of pacstrap script. The pacstrap script always attempt to update package database list with -Sy option.

I think some amendments on pacstrap script is needed to install Arch Linux without internet connection.

dodo3773 · 2013-05-22 03:13:53

Are the installations going to be the same? Same setup, architecture, etc..? If so why not just use an rsync snapshot, partition the drive, copy the rsync snapshot to the new disk/partition whatever, and update grub and /etc/fstab? Wouldn't that be the quicker easier route? I really don't think the current arch iso is intentionally built with the ability to install offline. Could always use the archiso program though and create a custom one. It's pretty straightforward if you follow the wiki entry on it. I still think rsync snapshot is easiest though.

ccy · 2013-05-22 03:34:29

No, the new instance will not be same. I will always keep the package cache in NFS to the latest updates.

In my usage, I always want the few new instance of up to date Arch Linux in the shortest time, so I think the offline installation should be the way to go.

WonderWoofy · 2013-05-22 03:50:30

Well it won't be the shortest amount of time if you spend a significant portion of it doing things like debugging this

Seriously though, while I understand what you want to do any why, if you are dead set on using the arch-install-scripts for this, then you are pretty much on your own I think (in regard to dev help). This is because the pacstrap command is meant to be used with an internets connection just like you have noticed. But if you want to use a shared package cache, you should also be savvy enough to install the base system without the arch-install-scripts.

Honestly, it is not that hard. Just read through the pacman man page, and see how to have it point to a difference pacman cache and root and you should be set. The genfstab is just the lazy way out (though I use it to reduce the amount i need to type), and arch-chroot is simply a shortcut in chrooting. So I think that since they are simply shortcuts around what is not that hard to being with, they are also meant to be as simple as possible.

Granted I could be totally wrong and the devs could welcome more complexity to the arch-install-scripts. But this is what I gather from having read over them.

ccy · 2013-05-22 04:12:28

pacstrap help me create the necessary directories in the file system which I am very appreciated. It would be great if they could add an option in pacstrap to allow using local *.db without internet connection.

WonderWoofy · 2013-05-22 04:48:04

That is whatthe actual pacman is for. Pactrap is really just a wrapper around pacman you know. It is meant to be simple and serve that single purpose, so IMO you are asking for what was intended to be a very simple tool to gain complexity.

Just make the directory structure yourself, I think all you need is the two /var paths, then mount those shares, and away you go. Pacman itself offers all you need.

ccy · 2013-05-22 05:33:55

Don't I have to create these directories with proper mode:
/mnt//{dev,run,etc}
/mnt//{tmp}
/mnt//{sys,proc}

These directories are created in pacstrap script.

WonderWoofy · 2013-05-22 05:40:59

You won't have to create those anyway because they are part of the filesystem package...

ccy · 2013-05-22 05:42:35

Thanks for the info. Never thought about that.

ccy · 2013-05-22 10:28:59

I try the alternate way of installation using pacman only (I didn't use pacstrap this time). I am able to install base packages to the newly mounted /mnt partition. I then install grub-bios and perform necessary grub configuration.

After reboot my machine, the grub menu is showing but select the menu option will immediately hang the system with "kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)"

After some trial and errors, I notice that the /boot/grub/grub.cfg is missing a line:

initrd /boot/initramfs-linux.img

And the file /boot/initramfs-linux.img isn't exist in /boot too. After copy /boot/initramfs-linux.img from other Arch Linux instant and amend grub.cfg, the system boot properly now.

Is there any steps I missed if using pacman only to perform installation?

tomk · 2013-05-22 10:43:53

Well yeah - you missed the step where you generate your initramfs image, obviously.

You should still follow the official Install Guide, even if you're not using everything in it - in particular, section 2.8 "Configure the system".

Last edited by tomk (2013-05-22 10:51:27)

progandy · 2013-05-22 11:58:11

ccy wrote:

pacstrap help me create the necessary directories in the file system which I am very appreciated. It would be great if they could add an option in pacstrap to allow using local *.db without internet connection.

Hey, pacstrap is a bash-script. The installation iso contains some useful tools like nano or vi to edit it and remove the sync-option.

ccy · 2013-05-22 13:40:55

Thanks. I miss the "mkinitcpio -p linux'. It works after execute the command.

tomk wrote:

Well yeah - you missed the step where you generate your initramfs image, obviously.
You should still follow the official Install Guide, even if you're not using everything in it - in particular, section 2.8 "Configure the system".

ccy · 2013-05-22 13:41:50

Is there possible to include my suggestion into the future release of pacstrap?

progandy wrote:

ccy wrote:
pacstrap help me create the necessary directories in the file system which I am very appreciated. It would be great if they could add an option in pacstrap to allow using local *.db without internet connection.
Hey, pacstrap is a bash-script. The installation iso contains some useful tools like nano or vi to edit it and remove the sync-option.

tomk · 2013-05-22 17:31:22

Unlikely, as already explained by WonderWoofy above. But feel free to post a feature request if it means that much to you.

teateawhy · 2013-05-22 18:12:02

I made the edit by adding the option -o to do what you want.
I am too lazy for a feature request right now...

#!/bin/bash

#
# Assumptions:
#  1) User has partitioned, formatted, and mounted partitions on /mnt
#  2) Network is functional
#  3) Arguments passed to the script are valid pacman targets
#  4) A valid mirror appears in /etc/pacman.d/mirrorlist
#

shopt -s extglob

out() { printf "$1 $2\n" "${@:3}"; }
error() { out "==> ERROR:" "$@"; } >&2
msg() { out "==>" "$@"; }
msg2() { out "  ->" "$@";}
die() { error "$@"; exit 1; }

in_array() {
  local i
  for i in "${@:2}"; do
    [[ $1 = "$i" ]] && return
  done
}

track_mount() {
  mount "$@" && CHROOT_ACTIVE_MOUNTS=("$2" "${CHROOT_ACTIVE_MOUNTS[@]}")
}

api_fs_mount() {
  CHROOT_ACTIVE_MOUNTS=()
  { mountpoint -q "$1" || track_mount "$1" "$1" --bind; } &&
  track_mount proc "$1/proc" -t proc -o nosuid,noexec,nodev &&
  track_mount sys "$1/sys" -t sysfs -o nosuid,noexec,nodev &&
  track_mount udev "$1/dev" -t devtmpfs -o mode=0755,nosuid &&
  track_mount devpts "$1/dev/pts" -t devpts -o mode=0620,gid=5,nosuid,noexec &&
  track_mount shm "$1/dev/shm" -t tmpfs -o mode=1777,nosuid,nodev &&
  track_mount run "$1/run" -t tmpfs -o nosuid,nodev,mode=0755 &&
  track_mount tmp "$1/tmp" -t tmpfs -o mode=1777,strictatime,nodev,nosuid
}

api_fs_umount() {
  umount "${CHROOT_ACTIVE_MOUNTS[@]}"
}

valid_number_of_base() {
  local base=$1 len=${#2} i=

  for (( i = 0; i < len; i++ )); do
    (( (${2:i:1} & ~(base - 1)) == 0 )) || return
  done
}

mangle() {
  local i= chr= out=

  unset {a..f} {A..F}

  for (( i = 0; i < ${#1}; i++ )); do
    chr=${1:i:1}
    case $chr in
      [[:space:]\\])
        printf -v chr '%03o' "'$chr"
        out+=\\
        ;;&
        # fallthrough
      *)
        out+=$chr
        ;;
    esac
  done

  printf '%s' "$out"
}

unmangle() {
  local i= chr= out= len=$(( ${#1} - 4 ))

  unset {a..f} {A..F}

  for (( i = 0; i < len; i++ )); do
    chr=${1:i:1}
    case $chr in
      \\)
        if valid_number_of_base 8 "${1:i+1:3}" ||
            valid_number_of_base 16 "${1:i+1:3}"; then
          printf -v chr '%b' "${1:i:4}"
          (( i += 3 ))
        fi
        ;;&
        # fallthrough
      *)
        out+=$chr
    esac
  done

  printf '%s' "$out${1:i}"
}

dm_name_for_devnode() {
  read dm_name <"/sys/class/block/${1#/dev/}/dm/name"
  if [[ $dm_name ]]; then
    printf '/dev/mapper/%s' "$dm_name"
  else
    # don't leave the caller hanging, just print the original name
    # along with the failure.
    print '%s' "$1"
    error 'Failed to resolve device mapper name for: %s' "$1"
  fi
}

fstype_is_pseudofs() {
  # list taken from util-linux source: libmount/src/utils.c
  local -A pseudofs_types=([anon_inodefs]=1
                           [autofs]=1
                           [bdev]=1
                           [binfmt_misc]=1
                           [cgroup]=1
                           [configfs]=1
                           [cpuset]=1
                           [debugfs]=1
                           [devfs]=1
                           [devpts]=1
                           [devtmpfs]=1
                           [dlmfs]=1
                           [fuse.gvfs-fuse-daemon]=1
                           [fusectl]=1
                           [hugetlbfs]=1
                           [mqueue]=1
                           [nfsd]=1
                           [none]=1
                           [pipefs]=1
                           [proc]=1
                           [pstore]=1
                           [ramfs]=1
                           [rootfs]=1
                           [rpc_pipefs]=1
                           [securityfs]=1
                           [sockfs]=1
                           [spufs]=1
                           [sysfs]=1
                           [tmpfs]=1)
  (( pseudofs_types["$1"] ))
}



newroot=/mnt
hostcache=0
copykeyring=1
copymirrorlist=1
offline=0

usage() {
  cat <<EOF
usage: ${0##*/} [options] root [packages...]

  Options:
    -C config      Use an alternate config file for pacman
    -c             Use the package cache on the host, rather than the target
    -d             Allow installation to a non-mountpoint directory
    -G             Avoid copying the host's pacman keyring to the target
    -i             Avoid auto-confirmation of package selections
    -M             Avoid copying the host's mirrorlist to the target
    -o             Do not update the pacman cache for offline installation
    -h             Print this help message

pacstrap installs packages to the specified new root directory. If no packages
are given, pacstrap defaults to the "base" group.

EOF
}

if [[ -z $1 || $1 = @(-h|--help) ]]; then
  usage
  exit $(( $# ? 0 : 1 ))
fi

(( EUID == 0 )) || die 'This script must be run with root privileges'

while getopts ':C:cdGiMo' flag; do
  case $flag in
    C)
      pacman_config=$OPTARG
      ;;
    d)
      directory=1
      ;;
    c)
      hostcache=1
      ;;
    i)
      interactive=1
      ;;
    G)
      copykeyring=0
      ;;
    M)
      copymirrorlist=0
      ;;
    o)
      offline=1
      ;;
    :)
      die '%s: option requires an argument -- '\''%s'\' "${0##*/}" "$OPTARG"
      ;;
    ?)
      die '%s: invalid option -- '\''%s'\' "${0##*/}" "$OPTARG"
      ;;
  esac
done
shift $(( OPTIND - 1 ))

(( $# )) || die "No root directory specified"
newroot=$1; shift
pacman_args=("${@:-base}")

if (( ! hostcache )); then
  pacman_args+=(--cachedir="$newroot/var/cache/pacman/pkg")
fi

if (( ! interactive )); then
  pacman_args+=(--noconfirm)
fi

if (( ! offline )); then
 pacman_args+=(-y)
fi

if [[ $pacman_config ]]; then
  pacman_args+=(--config="$pacman_config")
fi

[[ -d $newroot ]] || die "%s is not a directory" "$newroot"
if ! mountpoint -q "$newroot" && (( ! directory )); then
  die '%s is not a mountpoint!' "$newroot"
fi

# create obligatory directories
msg 'Creating install root at %s' "$newroot"
mkdir -m 0755 -p "$newroot"/var/{cache/pacman/pkg,lib/pacman,log} "$newroot"/{dev,run,etc}
mkdir -m 1777 -p "$newroot"/tmp
mkdir -m 0555 -p "$newroot"/{sys,proc}

# always call umount on quit after this point
trap 'api_fs_umount "$newroot" 2>/dev/null' EXIT

# mount API filesystems
api_fs_mount "$newroot" || die "failed to setup API filesystems in new root"

msg 'Installing packages to %s' "$newroot"
if ! pacman -r "$newroot" -S "${pacman_args[@]}"; then
  die 'Failed to install packages to new root'
fi

if (( copykeyring )); then
  # if there's a keyring on the host, copy it into the new root, unless it exists already
  if [[ -d /etc/pacman.d/gnupg && ! -d $newroot/etc/pacman.d/gnupg ]]; then
    cp -a /etc/pacman.d/gnupg "$newroot/etc/pacman.d/"
  fi
fi

if (( copymirrorlist )); then
  # install the host's mirrorlist onto the new root
  cp -a /etc/pacman.d/mirrorlist "$newroot/etc/pacman.d/"
fi

# vim: et ts=2 sw=2 ft=sh:

ccy · 2013-05-23 02:52:44

Thanks teateawhy. It works now. I also file a request: https://bugs.archlinux.org/task/35434

teateawhy wrote:

I made the edit by adding the option -o to do what you want.
I am too lazy for a feature request right now...

ccy · 2013-05-23 08:42:23

I found out the reason why using pacman to install base packages doesn't work. It is due to the installing of linux package at this stage:

( 72/113) installing linux [######################] 100%
>>> Updating module dependencies. Please wait ...
>>> Generating initial ramdisk, using mkinitcpio. Please wait...
==> ERROR: /proc must be mounted!

Generating initial ramdisk with mkinitcpio fail due to "/proc must be mounted". I think pacstrap do something with /mnt/proc before execute pacman.

tomk wrote:

Well yeah - you missed the step where you generate your initramfs image, obviously.
You should still follow the official Install Guide, even if you're not using everything in it - in particular, section 2.8 "Configure the system".

Awebb · 2013-05-23 11:20:41

If you can mount the nfs share, I assume you have a working network. Why don't you set up a local mirror/repository using the packages from the first install and sync from this mirror on the second box? It does not solve all offline install scenarios, yes. It should be no problem to remove the local repository later and sync with the regular servers. This is, by the way, what some people here do, they only download the packages once and then sync all computers from a local repository.

Arch Linux

#1 2013-05-22 02:22:07

Install 2nd instance of Arch Linux without internet connection

#2 2013-05-22 03:13:53

Re: Install 2nd instance of Arch Linux without internet connection

#3 2013-05-22 03:34:29

Re: Install 2nd instance of Arch Linux without internet connection

#4 2013-05-22 03:50:30

Re: Install 2nd instance of Arch Linux without internet connection

#5 2013-05-22 04:12:28

Re: Install 2nd instance of Arch Linux without internet connection

#6 2013-05-22 04:48:04

Re: Install 2nd instance of Arch Linux without internet connection

#7 2013-05-22 05:33:55

Re: Install 2nd instance of Arch Linux without internet connection

#8 2013-05-22 05:40:59

Re: Install 2nd instance of Arch Linux without internet connection

#9 2013-05-22 05:42:35

Re: Install 2nd instance of Arch Linux without internet connection

#10 2013-05-22 10:28:59

Re: Install 2nd instance of Arch Linux without internet connection

#11 2013-05-22 10:43:53

Re: Install 2nd instance of Arch Linux without internet connection

#12 2013-05-22 11:58:11

Re: Install 2nd instance of Arch Linux without internet connection

#13 2013-05-22 13:40:55

Re: Install 2nd instance of Arch Linux without internet connection

#14 2013-05-22 13:41:50

Re: Install 2nd instance of Arch Linux without internet connection

#15 2013-05-22 17:31:22

Re: Install 2nd instance of Arch Linux without internet connection

#16 2013-05-22 18:12:02

Re: Install 2nd instance of Arch Linux without internet connection

#17 2013-05-23 02:52:44

Re: Install 2nd instance of Arch Linux without internet connection

#18 2013-05-23 08:42:23

Re: Install 2nd instance of Arch Linux without internet connection

#19 2013-05-23 11:20:41

Re: Install 2nd instance of Arch Linux without internet connection

Board footer