Arch Linux

Duke_of_hazzard

Member

Registered: 2026-07-15

Posts: 1

System corruption after freeze during pacman upgrade: zero-byte files

I am posting this mostly as a recovery report in case somebody runs into a similar failure pattern.

I am mainly documenting this because the combination of:

Unrecognized archive format
no mtree file
many zero-byte files
Truncated tar archive detected while reading data

after a frozen pacman transaction was extremely confusing.

Hardware / system:
Arch Linux
Btrfs root filesystem
Timeshift / grub-btrfs snapshots
Intel Core i9-14900KF
ASUS ROG STRIX Z790-A GAMING WIFI D4
NVIDIA GeForce RTX 5070 Ti
nvidia-open-dkms
Kernel after recovery: 7.1.3-arch1-3

The motherboard was still running BIOS 0502 from 2022. It has since been updated to BIOS 3107, with BIOS defaults loaded and XMP/AI overclocking disabled.

What happened

The machine froze during a large package upgrade.

After rebooting, the installed system was badly broken. Plasma would not start correctly and I eventually used an Arch installation USB for recovery.

The FAT EFI filesystem also had the dirty bit set. I repaired it from the live system using fsck.fat.

During investigation I found several different kinds of damage.

Pacman's local database initially had broken package metadata. For example:

error while reading file /var/lib/pacman/local/linux package metadata. For example:

```text
error-7.1.3.arch1-3/mtree:
Unrecognized archive format

There were many more packages with unreadable mtree files.

pacman -Qkk also showed actual damaged files. At one point there were many empty files/libraries and, for example:

chromium: 276 total files, 242 altered files
filesystem: 127 total files, 3 altered files
java-runtime-common: 21 total files, 2 altered files

Several KF6 libraries were zero bytes and ldconfig repeatedly printed messages such as:

File /usr/lib/... is empty, not checked.

The local package database also had specific broken entries. Examples included:

linux: no mtree file
nvidia-open-dkms: no mtree file

and a damaged pipewire-audio local database entry with missing desc / file list metadata.

Initial recovery

I repaired individual critical packages first.

The kernel package was reinstalled from the pacman package cache. Afterwards:

pacman -Qkk linux
linux: 7675 total files, 0 altered files

The initramfs was regenerated successfully.

nvidia-open-dkms was also repaired, and the damaged PipeWire package database state was repaired using cached packages.

Eventually:

pacman -Dk
No database errors have been found!

However, a full pacman -Qkk still showed a large number of packages whose mtree metadata could not be read.

At that point I stopped repairing packages one by one.

Full reinstall attempt

I attempted to reinstall all native repository packages:

pacman -Qqn | pacman -S --overwrite \* -

There were 1169 packages in the transaction.

The first attempt failed because of existing filesystem conflicts.

With --overwrite '*', the transaction started, but then failed while extracting Python:

error: could not extract /usr/lib/python3.14/...pyc
(Truncated tar archive detected while reading data)
error: problem occurred while upgrading python
error: failed to commit transaction
Errors occurred, no packages were upgraded.

I deleted the cached Python package so pacman would download it again.

The next attempt failed on a completely different package:

error: could not extract /usr/share/nvidia/nvoptix.bin
(Truncated tar archive detected while reading data)
error: problem occurred while upgrading nvidia-utils

This made me suspect something beyond a single corrupt package archive.

Storage checks

The root filesystem is Btrfs.

Kernel logs did not show Btrfs checksum/corruption or NVMe I/O errors.

Also:

btrfs device stats /

showed all counters at 0.

The failing nvidia-utils package itself passed:

zstd -t /var/cache/pacman/pkg/nvidia-utils-610.43.03-3-x86_64.pkg.tar.zst

Later it also passed:

bsdtar -tf /var/cache/pacman/pkg/nvidia-utils-610.43.03-3-x86_64.pkg.tar.zst >/dev/null

So the same archive that pacman had reported as a truncated tar archive could be completely read by both zstd and libarchive tools.

BIOS / platform change

At this point I stopped retrying the full package transaction.

The system was using an i9-14900KF on motherboard BIOS 0502 from 2022.

I updated the ASUS motherboard BIOS to version 3107 using EZ Flash 3 and loaded the default BIOS settings.

XMP and AI overclocking were left disabled.

I am not claiming this proves the BIOS or CPU instability was the root cause. This is only the important change in the test environment.

Result after BIOS update

Back in the Arch live system, I mounted the Btrfs root subvolume, EFI partition and pacman cache subvolume and entered the system with arch-chroot.

pacman -Dk was still clean.

I first tested the exact package which had previously failed:

bsdtar -tf /var/cache/pacman/pkg/nvidia-utils-610.43.03-3-x86_64.pkg.tar.zst >/dev/null && echo ARCHIV_OK

Result:

ARCHIV_OK

Then:

pacman -U --overwrite \* /var/cache/pacman/pkg/nvidia-utils-610.43.03-3-x86_64.pkg.tar.zst

completed successfully.

I then reran:

pacman -Qqn | pacman -S --overwrite \* -

All 1169 packages were reinstalled successfully.

The transaction completed all post-transaction hooks:

(30/30) Updating X fontdir indices...

and returned to the shell without an error.

DKMS recovery

I had temporarily disabled the DKMS pacman hooks during recovery by masking them in /etc/pacman.d/hooks.

After the package reinstall I removed those masks.

dkms status was initially empty, although the NVIDIA source tree existed:

/usr/src/nvidia-610.43.03
/usr/src/nvidia-610.43.03/dkms.conf

I manually installed the NVIDIA DKMS module for the target kernel:

dkms install nvidia/610.43.03 -k 7.1.3-arch1-3

This successfully built and installed:

nvidia.ko.zst
nvidia-uvm.ko.zst
nvidia-modeset.ko.zst
nvidia-drm.ko.zst
nvidia-peermem.ko.zst

Then:

mkinitcpio -P

completed successfully.

Verification

A new full check:

pacman -Qkk > /root/qkk-after.txt 2>&1

showed:

grep -c Unrecognized /root/qkk-after.txt

Result:

0

The only remaining altered-file summaries were:

filesystem: 127 total files, 2 altered files
java-runtime-common: 21 total files, 2 altered files

The filesystem differences were normal modified backup/configuration files such as /etc/fstab, /etc/passwd, /etc/group, etc.

The Java differences were the expected default Java symlinks:

/usr/lib/jvm/default
/usr/lib/jvm/default-runtime

Finally:

pacman -Dk
No database errors have been found!

The installed system booted successfully.

Current checks:

uname -r
7.1.3-arch1-3
dkms status
nvidia/610.43.03, 7.1.3-arch1-3, x86_64: installed

nvidia-smi works and detects the RTX 5070 Ti. Xorg, KWin and Plasma are using the GPU.

The current kernel log contains no Btrfs corruption, NVMe I/O/reset/timeout, checksum errors or NVIDIA Xid errors.

Conclusion

I cannot prove the old BIOS / CPU platform state caused the repeated extraction failures.

However, the observed sequence was:

Freeze during a package upgrade.
Large-scale package/file and pacman metadata damage.
Repeated Truncated tar archive failures on different packages.
Btrfs device error counters remained zero.
The failing NVIDIA package passed zstd -t and bsdtar -tf.
BIOS was updated from 0502 to 3107 and defaults were loaded.
The exact failing NVIDIA package then installed successfully.
The complete 1169-package reinstall completed successfully.
The system now passes pacman database checks and boots normally.

If anybody has seen the same pattern, especially on a 13th/14th generation Intel desktop CPU, I would be interested to know whether this looks like a known platform-instability failure mode or whether another explanation fits the evidence better.

Lone_Wolf

Administrator

From: Netherlands, Europe

Registered: 2005-10-04

Posts: 15,218

Re: System corruption after freeze during pacman upgrade: zero-byte files

Please put program output in [ code ] [ /code ] tags, see https://bbs.archlinux.org/help.php#bbcode
This will make your post much easier to read.

Are you asking for help to determine the root cause ?

The machine froze during a large package upgrade.

The journal of that boot would help to figure out what happened.

---

Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.

clean chroot building not flexible enough ?
Try clean chroot manager by graysky

seth

Member

From: Won't reply 2 private help req

Registered: 2012-09-03

Posts: 76,836

Re: System corruption after freeze during pacman upgrade: zero-byte files

especially on a 13th/14th generation Intel desktop CPU, I would be interested to know whether this

You mean like https://en.wikipedia.org/wiki/Raptor_La … tion_issue ?
Make ab-so-fuckingly sure to have the latest https://wiki.archlinux.org/title/Microcode and hope it's not too late

I didn't read the WOT but filesystem inconsistencies are expectable if your system goes tits up during an update (and inherently massive amounts of disk writes) - the causes of that can however be manyfold.

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

cryptearth

Member

Registered: 2024-02-03

Posts: 2,259

Re: System corruption after freeze during pacman upgrade: zero-byte files

haven't read all the blog - but: if you do have snapshots and timeshift in place ... why not use it and just load/restore a previous checkpoint?