SOLVED Pacman issue in bash script

lenhuppe · 2021-08-19 19:09:45

SOLUTION:

The lscpu command was failing and the proc_type variable was not being set. Therefore the case command was unable to run.

I replaced

proc_type=$(lscpu | awk '/Vendor ID:/ {print $3}')

with

proc_type=$(awk '/vendor_id/ {print $3}' /proc/cpuinfo | sort -u)

***

I have a custom installation/rescue script that I am developing. I have run it successfully many times but recently a section of code stopped working. The code in question detects the cpu type and installs the correct microcode package which is either amd-ucode or intel-ucode. When I run my installer the package amd-ucode is not being installed. As you might expect for an installer this script runs in chroot from a booted ISO. Once again I have run this installer successfully many times in the past.

When troubleshooting I found that I can run 'pacman -Syy' followed by 'pacman -Si amd-ucode' from the booted ISO console and I get package information. Also there are no error messages when the installer runs. When I boot for the first time I can run 'sudo pacman -S amd-ucode' with no issues.

I am stumped

Broken code segment

#
# determine processor type and install microcode
# 
proc_type=$(lscpu | awk '/Vendor ID:/ {print $3}')
case "$proc_type" in
	GenuineIntel)
		print "Installing Intel microcode"
		pacman -S --noconfirm intel-ucode
		proc_ucode=intel-ucode.img
		;;
	AuthenticAMD)
		print "Installing AMD microcode"
		pacman -S --noconfirm amd-ucode
		proc_ucode=amd-ucode.img
		;;
esac

Last edited by lenhuppe (2021-08-20 00:03:32)

Trilby · 2021-08-19 19:32:30

Please post or link to the full script. What is "print" and where does it print to? Does the "Installing AMD microcode" message arrive at that destination? (i.e., is that branch of the case statement reached?)

Last edited by Trilby (2021-08-19 19:32:45)

lenhuppe · 2021-08-19 19:47:17

Trilby wrote:

Please post or link to the full script. What is "print" and where does it print to? Does the "Installing AMD microcode" message arrive at that destination? (i.e., is that branch of the case statement reached?)

Script 1 is launched from the console and it calls Script 2 in chroot. The code in question is near the end of Script 2.

My custom print command works by printing to the console. I provide plenty of feedback as the script runs for troubleshooting purposes.

Script 1 = archzfs.sh

#!/usr/bin/env bash

set -e

#
# language
#
# language=en
export LANG=C

#
# system info
#
host_name="${1}"
kernel=linux-lts

#
# locale info
#
timezone=America/New_York
countrycode=US
country=us

#
# zpool options
#
# see https://wiki.archlinux.org/title/Install_Arch_Linux_on_ZFS#Create_the_root_zpool
# see https://openzfs.github.io/openzfs-docs/man/8/zpool-create.8.html
#
# when 2.1.0 ships
# -o compatibility=off
#
zpool_options=(
-f -d
-m none
-R /mnt
-o ashift=12
-o autotrim=on
-o feature@lz4_compress=enabled
-o feature@zpool_checkpoint=enabled
-o compatibility=off
-O compression=lz4
-O relatime=on
-O atime=off
-O dnodesize=legacy
-O normalization=formD
-O devices=off
-O xattr=sa
)

#
# base packages
#
base_pkgs=(
base
"$kernel"
"$kernel"-headers
linux-firmware
)

#
# print messages
#
function print {
  echo -e '\n\t\033[0;32m' "$1" '\033[0m\n'
}

#### ----- MAIN ---- ####

#
# locate disks
#
if [ "$(ls /dev/nvme?)" ] ; then
	mapfile -t < <(nvme list | awk '/nvme?n?/ { print $1 }')
else
	mapfile -t < <(ls /dev/sd?)
fi
#spare="${MAPFILE[?]}"
disk1="${MAPFILE[0]}"
disk2="${MAPFILE[1]}"

#
# gain approval
#
clear
print "Drives found:"
print "ZFS disk1 = $disk1"
print "ZFS disk2 = $disk2"
print "USB boot= /dev/$(lsblk -f | grep iso | awk '/^sd?/ {print $1}')"
print "host = $host_name"
read -r -p "Continue? yes/no : " prompt  
until [ "$prompt" == "yes" ] || [ "$prompt" == "no" ] ; do
	read -r -p "Continue? yes/no : " prompt  
done
case "$prompt" in
	yes) print "Welcome to the Arch Linux on OpenZFS installation program" ;;
	no) print "quitting" ; exit ;;
esac

#
# set locale
#
print "Syncronizng with the network time server"
timedatectl set-ntp true && sleep 30
timedatectl set-local-rtc 0
timedatectl set-timezone "$timezone"
echo -e '\n'
timedatectl
localectl set-locale en_"$countrycode".UTF-8
localectl set-keymap "$country"
echo -e '\n'
localectl

#
# wipe drives
#
# lba: 512 bytes	zpool ashift=12		
# lba: 4096 bytes	zpool ashift=9
#
print "Wiping drives"
case "$disk1:$disk2" in
	/dev/nvme?n1:/dev/nvme?n1)
		#
		# nvme id-ns /dev/nvme?n? | grep lba
		#	nlbaf	label of lba format used 	0 = first
		#	lbaf 	qty of lba formats 			0 = qty one
		#	lbads	lba size  					9=512 bytes, 12=4096 bytes
		#
		wipefs -a "$disk1"
		sgdisk -Z "$disk1"
		blkdiscard -f "$disk1"
		nvme format -f "$disk1"
		parted "$disk1" mklabel gpt
		#
		wipefs -a "$disk2"
		sgdisk -Z "$disk2"
		blkdiscard -f "$disk2"
		nvme format -f "$disk2"
		parted "$disk2" mklabel gpt
		;;
	/dev/sd?:/dev/sd?)
		# 
		# hdparm -I /dev/sd? | grep size
		#	Logical  Sector size:	512 bytes
		#	Physical Sector size:	512 bytes
		#
		wipefs -a "$disk1"
		sgdisk -Z "$disk1"
		blkdiscard -f "$disk1"
		parted "$disk1" mklabel gpt
		#
		wipefs -a "$disk2"
		sgdisk -Z "$disk2"
		blkdiscard -f "$disk2"
		parted "$disk2" mklabel gpt
		;;
esac

#
# UEFI Partitions
#
print "Partitioning drives for UEFI system"
sgdisk -n 0:1M:+256M -t 0:EF00 "$disk1"
sgdisk -n 0:0:-0M -t 0:BF00 "$disk1"
partprobe -s "$disk1"
sgdisk -n 0:1M:+256M -t 0:EF00 "$disk2"
sgdisk -n 0:0:-0M -t 0:BF00 "$disk2"
partprobe -s "$disk2"

#
#	there are two possible partition scenarios:
#
#	SSD	/dev/sd?		NVMe	/dev/nvme?n1			
#							
#	<disk1>1 = boot		<disk1>p1 = boot
#	<disk1>2 = root1	<disk1>p2 = root1 
#	<disk2>1 = unused	<disk2>p1 = unused
#	<disk2>2 = root2	<disk2>p2 = root2 
#
#	boot is mounted at /boot and used as an EFI partition
#	root1 and root2 form the zpool mirror where the root filesystem lies
#	* what to do with unused partition on disk2 ?
#
case "$disk1:$disk2" in
	/dev/sd?:/dev/sd?)
		boot="$disk1"1	
		root1="$(blkid -s PARTUUID -o value "$disk1"2)"
		root2="$(blkid -s PARTUUID -o value "$disk2"2)"
		;;
	/dev/nvme?n1:/dev/nvme?n1)
		boot="$disk1"p1	
		root1="$(blkid -s PARTUUID -o value "$disk1"p2)"
		root2="$(blkid -s PARTUUID -o value "$disk2"p2)"
		;;
esac

#
# UEFI boot partition
#
print "Creating UEFI boot partition"
mkfs.fat -F 32 -n EFI "$boot"

#
# Mirrored zpool
#
print "Creating ZFS pool"
zpool create "${zpool_options[@]}" zroot mirror "$root1" "$root2"
zpool status

#
# Datasets
#
print "Creating datasets"
#
# ROOT dataset
#
zfs create -o canmount=off -o mountpoint=none zroot/ROOT
zfs create -o canmount=noauto -o mountpoint=/ zroot/ROOT/default
zpool set bootfs=zroot/ROOT/default zroot
#
# DATA dataset	
#
zfs create -o canmount=off -o mountpoint=none zroot/DATA
zfs create -o mountpoint=/home zroot/DATA/home
zfs create -o mountpoint=/root zroot/DATA/root
zfs create -o mountpoint=/archive zroot/DATA/archive
zfs list

#
# Mount everything 
#
print "Mounting everything" 
zfs umount -a
zpool export zroot
zpool import -d /dev/disk/by-partuuid/ -R /mnt zroot -N
zfs mount zroot/ROOT/default
zfs mount -a
mkdir /mnt/boot
mount "$boot" /mnt/boot

#
# Base installation
#
print "Installing Arch Linux"
pacman -Sy
pacstrap /mnt "${base_pkgs[@]}"
genfstab -U -p /mnt >> /mnt/etc/fstab
sed -i 's/^zroot/#zroot/' /mnt/etc/fstab
mkdir /mnt/archive/archzfs
cp /usr/local/bin/*.sh /mnt/archive/archzfs
arch-chroot /mnt /archive/archzfs/chroot_config.sh \
	"$host_name" "$kernel" "$timezone" "$countrycode" "$country"

#
# Finish before rebooting
#
print "Unmounting everything"
umount /mnt/boot
umount -R /mnt
zfs umount -a && zpool export zroot && print "Arch Linux on OpenZFS installation complete"

Part 2 = chroot_config.sh

#!/usr/bin/env bash

set -e

#
# capture args
#
host_name="${1}"
kernel="${2}"
timezone="${3}"
countrycode="${4}"
country="${5}"

#
# language
#
# language=en
export LANG=C

#
# user info
#
user_name=lenny
user_passwd=sysop
root_passwd=sysr00t

#
# utility list
#
utility_pkgs=(
sudo
openssh
irqbalance
vim
vim-ale
shellcheck
reflector
dosfstools
exfatprogs
e2fsprogs
)

#
# print messages
#
function print {
  echo -e '\n\t\033[0;32m' "$1" '\033[0m\n'
}

print "Configuring the new system"

#
# DNS setup
#
# linking stubresolv.conf to resolv.conf does not work in chroot - link in booted system
#

#
# user accounts
#
sed -i '/^PS1=/d' /etc/skel/.bashrc && echo -e "PS1='\u@\h \W \$ '" >> /etc/skel/.bashrc
useradd -m -g users -G wheel -s /bin/bash "$user_name"
sh -c "echo $user_name:$user_passwd | chpasswd"
sh -c "echo root:$root_passwd | chpasswd"

#
# ArchZFS archive permissions
#
chown -R "$user_name":users /archive
chmod 700 /archive/archzfs/*

#
# host
#
echo "$host_name" > /etc/hostname
tee /etc/hosts << eof
#
# localhost config
#
::1         localhost
127.0.0.1   localhost
127.0.1.1   $host_name.localdomain $host_name
eof

#
# locale
#
# localectl cannot be run in chroot - creating config files manually
sed -i "s/#en_$countrycode.UTF-8/en_$countrycode.UTF-8/" /etc/locale.gen
locale-gen
echo -e "KEYMAP=$country" > /etc/vconsole.conf
echo -e "LANG=en_$countrycode.UTF-8" > /etc/locale.conf

#
# date & time setup
#
# timedatectl cannot be run in chroot - linking to timezone data manually
ln -sf /usr/share/zoneinfo/"$timezone" /etc/localtime

#
# networking setup 
#
network=/etc/systemd/network
nic0=$(networkctl | awk '/ether/ {print $2}')

#
# ethernet config
#
tee "$network"/wired.network << eof
[Match]
Name=$nic0
[Network]
MulticastDNS=yes
IPv6PrivacyExtensions=kernel
DHCP=yes
[DHCP]
RouteMetric=10
UseDomains=yes
eof

#
# tethering config
#
tee "$network"/tethered.network << eof
[Match]
Name=eth0
[Network]
MulticastDNS=yes
IPv6PrivacyExtensions=kernel
DHCP=yes
[DHCP]
RouteMetric=30
UseDomains=yes
eof

#
# sysctl
#
sysctl=/etc/sysctl.d

#
# swappiness
#
tee "$sysctl"/sysctl.conf << eof
vm.swappiness=10
eof

#
# IPv6 privacy extensions
#
tee "$sysctl"/40-ipv6.conf << eof
# Enable IPv6 Privacy Extensions
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.use_tempaddr = 2
net.ipv6.conf.$nic0.use_tempaddr = 2
net.ipv6.conf.eth0.use_tempaddr = 2
eof

#
# enable the archzfs repo
# see https://openzfs.github.io/openzfs-docs/Getting%20Started/Arch%20Linux/0-archzfs-repo.html
#
print "Enabling the archzfs repo"
pacman-key --init
pacman-key --populate archlinux
curl -L https://archzfs.com/archzfs.gpg | pacman-key -a -
key_id=$(curl -L https://git.io/JsfVS)
pacman-key --finger "$key_id" && pacman-key --lsign-key "$key_id"
curl -L https://git.io/Jsfw2 | grep ^Server > /etc/pacman.d/archzfs-mirrorlist
tee -a /etc/pacman.conf << eof
[archzfs]
Include = /etc/pacman.d/archzfs-mirrorlist
eof
pacman -Syy

#
# utility config
#
print "Installing system utilities"
pacman -S --noconfirm "${utility_pkgs[@]}"
sed -i "s/# %wheel ALL=(ALL) ALL/%wheel ALL=(ALL) ALL/" /etc/sudoers
sed -i "s/#   Port 22/Port 22/" /etc/ssh/ssh_config
sed -i "s/#Port 22/Port 22/" /etc/ssh/sshd_config
tee /etc/xdg/reflector/reflector.conf << eof
--country $country
--protocol https
--fastest 6
--sort rate
--save /etc/pacman.d/mirrorlist
eof
systemctl enable sshd
systemctl enable irqbalance

#
# install openzfs
#
print "Installing OpenZFS"
pacman -S --noconfirm zfs-dkms
zgenhostid "$(hostid)"
zpool set cachefile=/etc/zfs/zpool.cache zroot
cp /etc/mkinitcpio.conf /etc/mkinitcpio.conf.bak
sed -i "s/HOOKS=(base udev autodetect modconf block filesystems keyboard fsck)/HOOKS=(base udev autodetect modconf block keyboard zfs filesystems fsck)/" /etc/mkinitcpio.conf
sed -i "s/BINARIES=()/BINARIES=(fsck)/"  /etc/mkinitcpio.conf
mkinitcpio -p "$kernel"

#
# systemd and zfs services to be enabled
#
print "Enabling systemd services"
sysd_svcs=(
systemd-networkd
systemd-resolved
systemd-timesyncd	
zfs.target
zfs-import-cache
zfs-import.target
zfs-mount
)
for s in "${sysd_svcs[@]}"; do
	systemctl enable "$s"
done

#
# determine processor type and install microcode
# 
proc_type=$(lscpu | awk '/Vendor ID:/ {print $3}')
case "$proc_type" in
	GenuineIntel)
		print "Installing Intel microcode"
		pacman -S --noconfirm intel-ucode
		proc_ucode=intel-ucode.img
		;;
	AuthenticAMD)
		print "Installing AMD microcode"
		pacman -S --noconfirm amd-ucode
		proc_ucode=amd-ucode.img
		;;
esac	

#
# bootloader setup
#
print "Installing systemd-boot"
bootctl install --esp-path=/boot
cp /boot/loader/loader.conf /boot/loader/loader.conf.bak
print "Creating EFI loader config"
tee /boot/loader/loader.conf << eof
default		arch
timeout		3
editor		no
eof
print "Creating Arch config"
tee /boot/loader/entries/arch.conf << eof
title		Arch Linux on OpenZFS
linux		vmlinuz-$kernel
initrd		$proc_ucode
initrd		initramfs-$kernel.img
options		zfs=zroot/ROOT/default rw
eof
print "Configuring systemd-boot"
bootctl update
bootctl set-default arch.conf 
bootctl list

Last edited by lenhuppe (2021-08-19 19:57:03)

Trilby · 2021-08-19 20:13:07

lenhuppe wrote:

I provide plenty of feedback as the script runs for troubleshooting purposes.

That's good ... because we could use that feedback here. This is why I asked about it ... soo ....

lenhuppe · 2021-08-19 22:59:14

Trilby wrote:

lenhuppe wrote:
I provide plenty of feedback as the script runs for troubleshooting purposes.
That's good ... because we could use that feedback here. This is why I asked about it ... soo ....

The lscpu command was failing and the proc_type variable was not being set. Therefore the case command was unable to run.

Last edited by lenhuppe (2021-08-20 00:06:12)

qinohe · 2021-08-19 23:42:47

I'm curious why you think the line in your script s wrong or wont work.
Haven't tried the script, though...

In your script:

lscpu | awk '/Vendor ID:/ {print $3}'

You even replaced it in your edit with

lscpu | awk '/vendor_id/ {print $3}'

Which obviously is wrong, vendor_id should be Vendor ID: , which is correct in the script...??

Loose the 'voila' line:-)

lenhuppe · 2021-08-20 00:03:15

qinohe wrote:

I'm curious why you think the line in your script s wrong or wont work.
Haven't tried the script, though...
In your script:
lscpu | awk '/Vendor ID:/ {print $3}'
You even replaced it in your edit with
lscpu | awk '/vendor_id/ {print $3}'
Which obviously is wrong, vendor_id should be Vendor ID: , which is correct in the script...??
Loose the 'voila' line:-)

Thank you for the heads up on my typo. I have made corrections.

What I can tell you is that by parsing the /proc/cpuinfo file instead of using lscpu everything is working again. Also the util-linux package where lscpu comes from has been recently updated.

The following code is working:

proc_type=$(awk '/vendor_id/ {print $3}' /proc/cpuinfo | sort -u)

qinohe · 2021-08-20 00:12:59

lenhuppe wrote:

Thank you for the heads up on my typo. I have made corrections.
What I can tell you is that by parsing the /proc/cpuinfo file instead of using lscpu everything is working again. Also the util-linux package where lscpu comes from has been recently updated.
The following code is working:
proc_type=$(awk '/vendor_id/ {print $3}' /proc/cpuinfo | sort -u)

Hey Lenny, Your welcome, but still, that lscpu line should work fine it surely does here!
I did see the 'first post' edit you made and you had an 'unbound variable' which is easaly solved:

proc_type=''

...
script
'''
proc_type=$(lscpu | awk '/Vendor ID:/ {print $3}')
case
....

lenhuppe · 2021-08-20 00:49:50

qinohe wrote:

lenhuppe wrote:
Thank you for the heads up on my typo. I have made corrections.
What I can tell you is that by parsing the /proc/cpuinfo file instead of using lscpu everything is working again. Also the util-linux package where lscpu comes from has been recently updated.
The following code is working:
proc_type=$(awk '/vendor_id/ {print $3}' /proc/cpuinfo | sort -u)
Hey Lenny, Your welcome, but still, that lscpu line should work fine it surely does here!
I did see the 'first post' edit you made and you had an 'unbound variable' which is easaly solved:
proc_type=''

...
script
'''
proc_type=$(lscpu | awk '/Vendor ID:/ {print $3}')
case
....

The original code worked fine for months as I tweaked the script on my test machine. However in the time since I created that code there have been multiple updates to the packages I use. I am not sure why the code stopped working but I ran it several times today and it failed every time. When I learned that lscpu looks to the /proc/cpuinfo file I bypassed it to see if that would fix the problem, and it did.

Arch Linux

#1 2021-08-19 19:09:45

SOLVED Pacman issue in bash script

#2 2021-08-19 19:32:30

Re: SOLVED Pacman issue in bash script

#3 2021-08-19 19:47:17

Re: SOLVED Pacman issue in bash script

#4 2021-08-19 20:13:07

Re: SOLVED Pacman issue in bash script

#5 2021-08-19 22:59:14

Re: SOLVED Pacman issue in bash script

#6 2021-08-19 23:42:47

Re: SOLVED Pacman issue in bash script

#7 2021-08-20 00:03:15

Re: SOLVED Pacman issue in bash script

#8 2021-08-20 00:12:59

Re: SOLVED Pacman issue in bash script

#9 2021-08-20 00:49:50

Re: SOLVED Pacman issue in bash script

Board footer