libxml2 upgrade has borked system. Best strategy to remedy?

kevdog · 2025-05-11 13:40:53

Currently running arch linux in vm.

Upgraded today which seemed like a simple upgrade:

$ sudo pacman -Syyu
:: Synchronizing package databases...
 archzfs                     1960.0   B  6.51 KiB/s 00:00 [###############################] 100%
 core                         117.7 KiB  1308 KiB/s 00:00 [###############################] 100%
 extra                          7.8 MiB  43.9 MiB/s 00:00 [###############################] 100%
 multilib                     134.4 KiB  1494 KiB/s 00:00 [###############################] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (10) bash-5.2.037-5  filesystem-2025.05.03-1  gcc-libs-15.1.1+r7+gf36ec88aa85a-1
              glibc-2.41+r48+g5cb575ca9a3d-1  iana-etc-20250502-1  libxml2-2.14.2-2
              meson-1.8.0-1  ncurses-6.5-4  postgresql-libs-17.4-4  sqlite-3.49.2-1

Total Installed Size:  256.64 MiB
Net Upgrade Size:        5.04 MiB

:: Proceed with installation? [Y/n] y
(10/10) checking keys in keyring                          [###############################] 100%
(10/10) checking package integrity                        [###############################] 100%
(10/10) loading package files                             [###############################] 100%
(10/10) checking for file conflicts                       [###############################] 100%
(10/10) checking available disk space                     [###############################] 100%
:: Processing package changes...
( 1/10) upgrading iana-etc                                [###############################] 100%
( 2/10) upgrading filesystem                              [###############################] 100%
warning: /etc/hosts installed as /etc/hosts.pacnew
( 3/10) upgrading glibc                                   [###############################] 100%
Generating locales...
  en_US.UTF-8... done
Generation complete.
( 4/10) upgrading gcc-libs                                [###############################] 100%
( 5/10) upgrading ncurses                                 [###############################] 100%
( 6/10) upgrading bash                                    [###############################] 100%
( 7/10) upgrading libxml2                                 [###############################] 100%
( 8/10) upgrading sqlite                                  [###############################] 100%
( 9/10) upgrading meson                                   [###############################] 100%
(10/10) upgrading postgresql-libs                         [###############################] 100%
:: Running post-transaction hooks...
(1/5) Creating system user accounts...
(2/5) Applying kernel sysctl settings...
(3/5) Creating temporary files...
(4/5) Arming ConditionNeedsUpdate...
(5/5) Updating the info directory file...
↱ (ssh)  kevdog@arch-TM             
↳ yay -Syyu
yay: error while loading shared libraries: libxml2.so.2: cannot open shared object file: No such file or directory
↱ (ssh)  kevdog@arch-TM                  
↳ ssh kevdog
ssh: Could not resolve hostname kevdog: Name or service not known
↱ (ssh)  kevdog@arch-TM                                
↳ sudo pacman -S libxml2
pacman: error while loading shared libraries: libxml2.so.2: cannot open shared object file: No such file or directory

Rebooted system, however I'm now stuck at grub commandl line with error of unknown file system.

When using an arch installation cd (which I've done before many times) - what am I looking for to rescue this system? Seems strange it would bring down entire system.

Scimmia · 2025-05-11 14:04:17

If you hadn't updated your system in a week and a half, there should have been a lot more updates than that. What mirror is at the top of your mirrorlist?

seth · 2025-05-11 14:14:08

Stop using the second "y", read up what that actually does.

however I'm now stuck at grub commandl line with error of unknown file system.

That's not reflected by the posted output, but you're probably booting a dated kernel that lacks the /lib/modules/$(uname -r) path - usually because you forgot to mount the /boot partition before the kernel update or mounted some partition into /boot that you're not actually booting from (check the BOOT_IMAGE path in journals of the installed system. NOT the install iso)
libxml2 was last updated 4/30 - pacman uses it through libarchive which isn't in your update list - a list that's suspiciously short for covering at least 12 days ?
=> Bad mirror?

Edit: great.

Last edited by seth (2025-05-11 14:14:26)

kevdog · 2025-05-11 15:23:33

Hey thanks for reply.

So my VM is provisioned with zfs on root filesystem. I've booted an arch install CD with the necessary zfs modules. I imported the zfs pool and mounted it at /mnt. I also bin mounted /run so resolv.conf would work. My entire process (yes I'm aware this info probably irrelevant to my problems was the following):

root@archiso / # lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
loop0     7:0    0 962.3M  1 loop /run/archiso/airootfs
sr0      11:0    1   1.1G  0 rom  /run/archiso/bootmnt
xvda    202:0    0   200G  0 disk
├─xvda1 202:1    0     2M  0 part
└─xvda2 202:2    0   200G  0 part
root@archiso / # mount /dev/xvda2 /mnt
mount: /mnt: unknown filesystem type 'zfs_member'.
       dmesg(1) may have more information after failed mount system call.
32 root@archiso / # zpool import -R /mnt zroot
root@archiso / # for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done

root@archiso / # chroot /mnt

With the chroot in place, I updated the mirrorlist with reflector:

# reflector --country US --number 30 --sort age --protocol https --save /etc/pacman.d/mirrorlist

Here is my mirrorlist:

################# Arch Linux mirrorlist generated by Reflector #################
################################################################################

# With:       reflector --country US --number 30 --sort age --protocol https --save /etc/pacman.d/mirrorlist
# When:       2025-05-11 15:21:56 UTC
# From:       [url]https://archlinux.org/mirrors/status/json/[/url]
# Retrieved:  2025-05-11 15:21:56 UTC
# Last Check: 2025-05-11 15:10:41 UTC

Server = [url]https://losangeles.mirror.pkgbuild.com/$repo/os/$arch[/url]
Server = [url]https://us.arch.niranjan.co/$repo/os/$arch[/url]
Server = [url]https://mirror.theash.xyz/arch/$repo/os/$arch[/url]
Server = [url]https://america.mirror.pkgbuild.com/$repo/os/$arch[/url]
Server = [url]https://arch.mirror.constant.com/$repo/os/$arch[/url]
Server = [url]https://mirrors.rit.edu/archlinux/$repo/os/$arch[/url]
Server = [url]https://arlm.tyzoid.com/$repo/os/$arch[/url]
Server = [url]https://mirror.givebytes.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://mirrors.kernel.org/archlinux/$repo/os/$arch[/url]
Server = [url]https://us-mnz.soulharsh007.dev/archlinux/$repo/os/$arch[/url]
Server = [url]https://mirrors.vectair.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://iad.mirrors.misaka.one/archlinux/$repo/os/$arch[/url]
Server = [url]https://us.mirrors.cicku.me/archlinux/$repo/os/$arch[/url]
Server = [url]https://mirrors.lug.mtu.edu/archlinux/$repo/os/$arch[/url]
Server = [url]https://arch-mirror.brightlight.today/$repo/os/$arch[/url]
Server = [url]https://yonderly.org/mirrors/archlinux/$repo/os/$arch[/url]
Server = [url]https://repo.ialab.dsu.edu/archlinux/$repo/os/$arch[/url]
Server = [url]https://arch.hu.fo/archlinux/$repo/os/$arch[/url]
Server = [url]https://mirror.arizona.edu/archlinux/$repo/os/$arch[/url]
Server = [url]https://mirror.zackmyers.io/archlinux/$repo/os/$arch[/url]
Server = [url]https://arch-mirror.marcusspencer.xyz:4443/archlinux/$repo/os/$arch[/url]
Server = [url]https://archmirror1.octyl.net/$repo/os/$arch[/url]
Server = [url]https://mirrors.ocf.berkeley.edu/archlinux/$repo/os/$arch[/url]
Server = [url]https://codingflyboy.mm.fcix.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://coresite.mm.fcix.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://forksystems.mm.fcix.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://irltoolkit.mm.fcix.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://mirror.fcix.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://mnvoip.mm.fcix.net/archlinux/$repo/os/$arch[/url]
Server = [url]https://nnenix.mm.fcix.net/archlinux/$repo/os/$arch[/url]

With this list - here is what I'm prompted to upgrade:

# pacman -Syu
:: Synchronizing package databases...
 archzfs is up to date
 core is up to date
 extra is up to date
 multilib is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Packages (10) bash-5.2.037-5  filesystem-2025.05.03-1  gcc-libs-15.1.1+r7+gf36ec88aa85a-1
              glibc-2.41+r48+g5cb575ca9a3d-1  iana-etc-20250502-1  libxml2-2.14.2-2
              meson-1.8.0-1  ncurses-6.5-4  postgresql-libs-17.4-4  sqlite-3.49.2-1

Total Download Size:    56.04 MiB
Total Installed Size:  256.64 MiB
Net Upgrade Size:        5.04 MiB

Once I upgrade - pacman, yay is borked due to libxml2 conflict. Because I'm running within a VM, I need to rollback to prior snapshot so I could repeat the process.

Last edited by kevdog (2025-05-11 15:26:25)

Scimmia · 2025-05-11 15:26:50

We asked what mirror you were using, so you overwrote that information? Good luck.

kevdog · 2025-05-11 15:38:57

Scimmia wrote:

We asked what mirror you were using, so you overwrote that information? Good luck.

??? What. If you really need to know this information, then I could restore from snapshot, however whether using the old mirror or the updated mirror with reflector, the same libxml2 problem rears its ugly head. Clearly there seems to be a problem somewhere as other have posted a similar error recently: https://www.reddit.com/r/archlinux/comm … w_i_fixed/

I'm not sure just installing libxml2-legacy is the proper solution, however this is what I'm actually finding as the "workaround" for now. Is this the recommended solution?

Last edited by kevdog (2025-05-11 15:54:36)

seth · 2025-05-11 19:03:07

Is this the recommended solution?

No. libxml2-legacy is a kludgecrutch for software that's only available as precompiled sources from upstream.

pacman -Qi pacman libarchive

You can in doubt use the pacman from the install iso to query the installed system:

pacman --root=/mnt -Qi pacman libarchive

Your mirrror has https://losangeles.mirror.pkgbuild.com/ … kg.tar.zst so that's not it.

Edit:

Clearly there seems to be a problem somewhere as other have posted a similar error recently:

These problems are down to
* broken mirrors
* "I'm using not-arch archlinux"
* partial updates (did you put pacman into the ignore list to avoid the 7.0 move? Your update list is however too sparse for most setups in general)
* binary only packages

We'll get there, but better already brace yourself for this being an issue on your site

Last edited by seth (2025-05-11 19:06:07)

Scimmia · 2025-05-12 00:51:51

Because you didn't update your databases. Ironically, this is where you need to ignore seth's advice to not use the extra y.

Last edited by Scimmia (2025-05-12 02:34:16)

kevdog · 2025-05-12 04:29:40

In terms of holding packages - yes glibc and pacman were listed on the ignore list. I've removed these packages from the hold list and reinstalled pacman, glibc, and libarchive from within the chroot.

I installed pacman-static from repo that got me going which allowed me to update the system and reinstalled pacman.

One of the issues I'm getting however when trying to install packages are the constant errors similar to:

jansson: /usr/include/jansson.h exists in filesystem
jansson: /usr/include/jansson_config.h exists in filesystem
jansson: /usr/lib/libjansson.a exists in filesystem
jansson: /usr/lib/libjansson.so exists in filesystem
jansson: /usr/lib/libjansson.so.4 exists in filesystem
jansson: /usr/lib/libjansson.so.4.14.1 exists in filesystem
jansson: /usr/lib/pkgconfig/jansson.pc exists in filesystem
jansson: /usr/share/licenses/jansson/LICENSE exists in filesystem
binutils: /usr/bin/addr2line exists in filesystem
binutils: /usr/bin/ar exists in filesystem
binutils: /usr/bin/as exists in filesystem
binutils: /usr/bin/c++filt exists in filesystem
...

I'm noticing this with almost all the packages I'm trying to install -- pacman, etc. I'm aware of the overwrite option, but I'm just curious why this error is now showing up. I'm wondering if its recommended to just reinstall all installed packages (which is very strange since for example I had vim installed however it didn't show with pacman -Qqn). Expounding upon this:

# pacman -Q
acl 2.3.2-1
attr 2.5.2-1
audit 4.0.3-1
bash 5.2.037-5
brotli 1.1.0-3
bzip2 1.0.8-6
ca-certificates 20240618-1
ca-certificates-mozilla 3.111-1
ca-certificates-utils 20240618-1
coreutils 9.7-1
cryptsetup 2.7.5-2
curl 8.13.0-2
db5.3 5.3.28-5
dbus 1.16.2-1
dbus-broker 36-4
dbus-broker-units 36-4
dbus-units 36-4
device-mapper 2.03.32-1
e2fsprogs 1.47.2-2
expat 2.7.1-1
file 5.46-4
filesystem 2025.05.03-1
findutils 4.10.0-2
gawk 5.3.2-1
gcc-libs 15.1.1+r7+gf36ec88aa85a-1
gdbm 1.25-1
gettext 0.25-1
glib2 2.84.1-1
glibc 2.41+r48+g5cb575ca9a3d-1
gmp 6.3.0-2
gnulib-l10n 20241231-1
gnupg 2.4.7-1
gnutls 3.8.9-1
gpgme 1.24.2-1
gpm 1.20.7.r38.ge82d1a6-6
grep 3.12-2
gzip 1.14-2
hwdata 0.395-1
iana-etc 20250502-1
icu 76.1-1
json-c 0.18-1
kbd 2.7.1-2
keyutils 1.6.3-3
kmod 34.2-1
krb5 1.21.3-1
leancrypto 1.3.0-1
libarchive 3.7.9-2
libassuan 3.0.0-1
libcap 2.76-1
libcap-ng 0.8.5-3
libelf 0.193-2
libevent 2.1.12-4
libffi 3.4.8-1
libgcrypt 1.11.1-1
libgpg-error 1.55-1
libidn2 2.3.7-1
libksba 1.6.7-2
libldap 2.6.9-1
libnghttp2 1.65.0-1
libnghttp3 1.9.0-1
libnsl 2.0.1-1
libp11-kit 0.25.5-1
libpsl 0.21.5-2
libsasl 2.1.28-5
libseccomp 2.5.6-1
libsecret 0.21.7-1
libssh2 1.11.1-1
libsysprof-capture 48.0-5
libtasn1 4.20.0-1
libtirpc 1.3.6-1
libunistring 1.3-1
libusb 1.0.28-1
libverto 0.3.2-5
libxcrypt 4.4.38-1
libxml2 2.14.2-2
libyaml 0.2.5-3
linux-api-headers 6.14-1
lmdb 0.9.33-1
lz4 1:1.10.0-2
meson 1.8.0-1
mpdecimal 4.0.0-2
mpfr 4.2.2-1
ncurses 6.5-4
nettle 3.10.1-1
ninja 1.12.1-2
npth 1.8-1
openssl 3.5.0-1
p11-kit 0.25.5-1
pacman 7.0.0.r6.gc685ae6-2
pacman-mirrorlist 20250503-1
pacman-static 6.0.1-2
pam 1.7.0-2
pambase 20230918-2
pcre2 10.45-1
perl 5.40.2-1
pgbackrest 2.55.1-1
pinentry 1.3.1-5
popt 1.19-2
postgresql-libs 17.4-4
procps-ng 4.0.5-3
python 3.13.3-1
python-tqdm 4.67.1-2
readline 8.2.013-1
sed 4.9-3
shadow 4.17.4-1
sqlite 3.49.2-1
systemd 257.5-2
systemd-libs 257.5-2
tpm2-tss 4.1.3-1
tzdata 2025b-1
util-linux 2.41-4
util-linux-libs 2.41-4
vim 9.1.1376-1
vim-runtime 9.1.1376-1
xz 5.8.1-1
zlib 1:1.3.1-2
zstd 1.5.7-2

However clearly packages like mkinitcpio are installed on the system:

# which mkinitcpio
/sbin/mkinitcpio

However when trying to install mkinitcpio:

mkinitcpio: /usr/bin/lsinitcpio exists in filesystem
mkinitcpio: /usr/bin/mkinitcpio exists in filesystem
mkinitcpio: /usr/lib/initcpio/functions exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/consolefont exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/encrypt exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/keymap exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/memdisk exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/resume exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/shutdown exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/sleep exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/udev exists in filesystem
mkinitcpio: /usr/lib/initcpio/hooks/usr exists in filesystem
mkinitcpio: /usr/lib/initcpio/init exists in filesystem
mkinitcpio: /usr/lib/initcpio/init_functions exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/acpi_override exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/autodetect exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/base exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/block exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/consolefont exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/encrypt exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/filesystems exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/fsck exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/hostdata exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/keyboard exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/keymap exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/kms exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/lvm2 exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/mdadm_udev exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/memdisk exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/microcode exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/modconf exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/resume exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/sd-encrypt exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/sd-shutdown exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/sd-vconsole exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/shutdown exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/sleep exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/strip exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/systemd exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/udev exists in filesystem
mkinitcpio: /usr/lib/initcpio/install/usr exists in filesystem
mkinitcpio: /usr/lib/initcpio/shutdown exists in filesystem
mkinitcpio: /usr/lib/initcpio/udev/01-memdisk.rules exists in filesystem
mkinitcpio: /usr/lib/kernel/install.d/50-mkinitcpio.install exists in filesystem
mkinitcpio: /usr/lib/systemd/system/halt.target.wants/mkinitcpio-generate-shutdown-ramfs.service exists in filesystem
mkinitcpio: /usr/lib/systemd/system/kexec.target.wants/mkinitcpio-generate-shutdown-ramfs.service exists in filesystem
mkinitcpio: /usr/lib/systemd/system/mkinitcpio-generate-shutdown-ramfs.service exists in filesystem
mkinitcpio: /usr/lib/systemd/system/poweroff.target.wants/mkinitcpio-generate-shutdown-ramfs.service exists in filesystem
mkinitcpio: /usr/lib/systemd/system/reboot.target.wants/mkinitcpio-generate-shutdown-ramfs.service exists in filesystem
mkinitcpio: /usr/lib/tmpfiles.d/mkinitcpio.conf exists in filesystem
mkinitcpio: /usr/share/bash-completion/completions/lsinitcpio exists in filesystem
mkinitcpio: /usr/share/bash-completion/completions/mkinitcpio exists in filesystem
mkinitcpio: /usr/share/libalpm/hooks/60-mkinitcpio-remove.hook exists in filesystem
mkinitcpio: /usr/share/libalpm/hooks/90-mkinitcpio-install.hook exists in filesystem
mkinitcpio: /usr/share/libalpm/scripts/mkinitcpio exists in filesystem
mkinitcpio: /usr/share/man/man1/lsinitcpio.1.gz exists in filesystem
mkinitcpio: /usr/share/man/man5/mkinitcpio.conf.5.gz exists in filesystem
mkinitcpio: /usr/share/man/man8/mkinitcpio.8.gz exists in filesystem
mkinitcpio: /usr/share/mkinitcpio/example.preset exists in filesystem
mkinitcpio: /usr/share/mkinitcpio/hook.preset exists in filesystem
mkinitcpio: /usr/share/zsh/site-functions/_mkinitcpio exists in filesystem
Errors occurred, no packages were upgraded.

It's almost like pacman has lost track of the installed packages.

seth · 2025-05-12 04:54:02

Don't ignore random packages just because you can

pacman-conf  | curl -F 'file=@-' 0x0.st
sudo LC_ALL=C pacman -Qkk | grep -v ', 0 altered files' | grep -v backup > /tmp/howbadisit.txt 2>&1
cat /tmp/howbadisit.txt | curl -F 'file=@-' 0x0.st

About fixing it, just in case you indeed previously used a bogus mirror, enforce a database update.
Then re-install the packages w/ a broken mtree w/ "--dbonly" before actually re-installing them.

If your package database has been so severely damaaged that you completely "lost" packages, see https://wiki.archlinux.org/title/Pacman … l_database

I need to rollback to prior snapshot … I could restore from snapshot

This might be cause and pot. solution to this problem iff you can properly restore an undamaged snapshot, fix your pacman config and mirror list and then update from there.

Lone_Wolf · 2025-05-12 12:19:03

 archzfs                     1960.0   B  6.51 KiB/s 00:00 [###############################] 100%

Is your root on ZFS ?
Which kernel are you using ?

Arch Linux

#1 2025-05-11 13:40:53

libxml2 upgrade has borked system. Best strategy to remedy?

#2 2025-05-11 14:04:17

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#3 2025-05-11 14:14:08

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#4 2025-05-11 15:23:33

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#5 2025-05-11 15:26:50

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#6 2025-05-11 15:38:57

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#7 2025-05-11 19:03:07

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#8 2025-05-12 00:51:51

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#9 2025-05-12 04:29:40

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#10 2025-05-12 04:54:02

Re: libxml2 upgrade has borked system. Best strategy to remedy?

#11 2025-05-12 12:19:03

Re: libxml2 upgrade has borked system. Best strategy to remedy?

Board footer