invalid or corrupted package (checksum) in custom repository.

Veldora · 2024-03-07 15:05:06

This could be a repost of this topic here:
https://bbs.archlinux.org/viewtopic.php?id=285338

But since there is no suitable solution in this post and I want to prevent necrobumping at all cost, I create a new topic.

---

Hi
I have my own repository with some self-maintained PKGBUILD and some from the AUR, (e.g. teamviewer), but they all have the same issue.

When creating an empty repository by executing repo-add repo.db.tar.gz and adding all compiled binaries via repo-add repo.db.tar.gz *.pkg.tar.zst to it, publishing to a webserver and executing pacman -Sy <package> it does what it should.

The package gets installed without any issues.

However, when deleting the source, re-clone from the AUR, rebuilding the package and deleting the old package from the repo with repo-remove repo.db.tar.gz <package>, refreshing the repo via repo-add repo.db.tar.gz -n -R *.pkg.tar.zst, moving the newly built <package>.pkg.tar.zst to the repo folder and adding it again by typing repo-add repo.db.tar.gz *.pkg.tar.zst, refreshing again and publishing the files via rsync -a --delete --progress to the webserver and then trying to install the package on the client, it fails, saying, that the package is invalid or corrupted, with the hint to the checksum.

:: File /var/cache/pacman/pkg/teamviewer-15.51.5-1-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (checksum)).
Do you want to delete it? [Y/n]

When doing the above steps manually by deleting the package from the repo, refreshing the repo, publishing the repo to the webserver, building the package and then adding the package to the repo, while refreshing the repo again and then publishing to the webserver, it works out of the box again.

When downloading the package via curl or wget from the repo and installing it via pacman -U it works without any problems.
The sha256sum of all packages (on the buildserver, on the webserver, in the cache and the downloaded package) are all the same.

Is there anything I can do?
The package building is done the traditional way using makepkg --cleanbuild --syncdeps --noconfirm --clean and not an aur-helper.

Has anyone stumbled over the same issue and could give me a hint on how to solve this issue?

Last edited by Veldora (2024-03-07 15:20:40)

WorMzy · 2024-03-07 15:31:01

executing pacman -Sy <package> it does what it should.

A partial upgrade? Don't do that.

As for the error, it sounds like you're pushing two different "builds" of teamviewer-15.51.5-1-x86_64.pkg.tar.zst (technically nothing is built in this package, it just repackages a deb file, but the package metadata [specifically .BUILDINFO and .PKGINFO] changes) to the same remote server, and a client is (rightly) telling you that v1 of the package it previously downloaded is different to v2 currently advertised by the remote server.

You should increment the pkgrel value when rebuilding a package so clients that already downloaded the 'old' version know to download the 'new' version.

Veldora · 2024-03-07 16:01:41

WorMzy wrote:

A partial upgrade? Don't do that.

It's not meant as a partial upgrade. I know it's dangerous.
I just used it to install the package and sync the repos, when it's not installed.

But I see, that it's also not advised to use. I'll change to pacman -Syu <package>

WorMzy wrote:

You should increment the pkgrel value when rebuilding a package so clients that already downloaded the 'old' version know to download the 'new' version.

Hm, afaik, the pkgrel should only be used, when e.g. buildflags got changed, where the official version stayed the same, and not when just rebuilding the package, while nothing has changed. So when correcting a typo, it says, to not increment the pkgrel, for example

WorMzy wrote:

and a client is (rightly) telling you v1 of the package it previously downloaded is different to v2 currently advertised by the remote server.

Well, if that's so, how does the client compare it, when the package has been uninstalled by pacman -Rs teamviewer and the package has been deleted from the package cache?

Are there still any leftovers of the file, which remain on the system, so it can compare the package which it is trying to install?

Last edited by Veldora (2024-03-07 16:08:19)

WorMzy · 2024-03-07 17:33:26

The configure flags is just an example. The most common cause of a pkgrel increment in binary repos is probably when a rebuild is required due to a soname bump in a dependency. Aside from the pkgrel, the PKGBUILD doesn't change at all (unless there's build breakage that needs addressing). e.g.

I don't see typo mention in the man page, so I'm not sure what you mean by 'it' there, but any rebuild of a package meant for binary distribution from a central source should increment the pkgrel because any rebuilt package will have a different checksum, even if there is only a few seconds between builds:

$ auracle clone teamviewer
$ cd teamviewer
$ PKGEXT=.pkg.tar SRCDEST=. makepkg --nodeps -f
$ md5sum teamviewer-15.51.5-1-x86_64.pkg.tar
309ced47732d4b3972166255e801dc1e  teamviewer-15.51.5-1-x86_64.pkg.tar
$ PKGEXT=.pkg.tar SRCDEST=. makepkg --nodeps -f
$ md5sum teamviewer-15.51.5-1-x86_64.pkg.tar               
62fc0fd217e91bb8daeb80a742b10d7d  teamviewer-15.51.5-1-x86_64.pkg.tar

Where I think confusion may be arising is that the package maintainers on the AUR don't need to do this for their AUR packages, because each consumer of that package is responsible for rebuilding when required (i.e. for soname bumps). They only need to bump the pkgrel when the PKGBUILD itself changes in some way that affects the build/packaging process.

Well, if that's so, how does the client compare it, when the package has been uninstalled by pacman -Rs teamviewer and the package has been deleted from the package cache?

Scenario 1: you haven't updated the clients sync database, so you have an out-of-date sync database to compare the checksum of the downloaded file against (run pacman -Syu rather than pacman -S)

Scenario 2: you haven't actually deleted the 'old' file from pacman's cache, so you're comparing an old package to the new checksum (rm /var/cache/pacman/pkg/teamviewer-15.51.5-1-x86_64.pkg.tar.zst then try again)

Scenario 3: the download is getting corrupted somehow (memtest, fsck, etc. are your friends in this case)

Veldora · 2024-03-08 08:51:24

WorMzy wrote:

I don't see typo mention in the man page, so I'm not sure what you mean by 'it' there

Well, I've atleast read it in the AUR submission guidelines:

AUR Submission Guidelines wrote:

When releasing a new version of the packaged software, update the pkgver or pkgrel variables to notify all users that an upgrade is needed. Do not update those values if only minor changes to the PKGBUILD such as the correction of a typo are being published.

So possibly this only affects the AUR and not official packages or packages, I maintain in my own repo.

WorMzy wrote:

Scenario 1: you haven't updated the clients sync database, so you have an out-of-date sync database to compare the checksum of the downloaded file against (run pacman -Syu rather than pacman -S)

Well, since the client uses the Arch Snapshots as a mirror and I've tested it using pacman -Syu teamviewer, where it didn't work, I don't think that's the case.

WorMzy wrote:

Scenario 2: you haven't actually deleted the 'old' file from pacman's cache, so you're comparing an old package to the new checksum (rm /var/cache/pacman/pkg/teamviewer-15.51.5-1-x86_64.pkg.tar.zst then try again)

After deleting the file from the cache, I checked using ls -la /var/cache/pacman/pkg | grep teamviewer and it didn't show me any results.
So I think, I deleted all of the "old" packages.

WorMzy wrote:

Scenario 3: the download is getting corrupted somehow (memtest, fsck, etc. are your friends in this case)

This however was my first theory, but I couldn't find a possible cause.
On the other hand, if the download is getting corrupted, shouldn't it be the case, that every download from every package in my repo should get corrupted and not only the "rebuilt" ones?

But in this case, updating the pkgrel somehow makes sense to solve the problem, if the version didn't change.
(The package does also get built against the configured arch snapshot, via a modified archlinux oci image)

Now I just somehow need to automate that in the building process.
(I don't know if there's a tool for that or not, but somehow saving the version in a static file and if it didn't change on the next rebuild, bump the pkgrel)

I don't know if the thread can be marked as solved, since there are still some questions open about, why some packages work and some don't, even if all the "old" packages got deleted. That kinda leaves me thinking, that pacman doesn't remove all left-overs of the package, when uninstalling it.

Arch Linux

#1 2024-03-07 15:05:06

invalid or corrupted package (checksum) in custom repository.

#2 2024-03-07 15:31:01

Re: invalid or corrupted package (checksum) in custom repository.

#3 2024-03-07 16:01:41

Re: invalid or corrupted package (checksum) in custom repository.

#4 2024-03-07 17:33:26

Re: invalid or corrupted package (checksum) in custom repository.

#5 2024-03-08 08:51:24

Re: invalid or corrupted package (checksum) in custom repository.

Board footer