An enhanced makepkg tool on github

mcloaked · 2022-09-03 13:21:50

A recent post by another Arch user on the arch-general mailing list, announced a new tool he has built with enhanced facilities compared to the standard makepkg tool.

I am copying his announcement to the Arch forum in case it is of interest to other Arch users who don't monitor the Arch ML.

"When building packages (makepkg + PKGBUILD), It would be useful if there was a way to easily rebuild the package whenever a make dependency is updated. One can do this manually of course, by bumping the pkgrel and
rebuilding.

To make it easier I built a tool to do that for me. For example:

When a tool chain used to build a package is updated, it's good practice to rebuild those which use that tool chain. For example, when gcc, cargo, binutils et al are updated, then packages using those tools should also
be updated.

While static linked libraries surely don't demand a rebuild to function, because the prior, older library is part of the binary itself, it's still a good idea to rebuild it. This will pick up fixes, including security related ones, as well as improvements. Of course, it's always sensible to confirm that the application properly builds and works with the newer tool or library.

I've made the code available should it be useful to others.

https://github.com/gene-git/Arch-mkpkg
"

This looks like a really useful new tool for rebuilding packages that you might want to build yourself if there are updated libraries not yet in the Arch repo built packages.

mcloaked · 2022-09-03 15:36:41

Earlier today I posted in the section of the forum Pacman & Package Upgrade Issues about an enhanced new version of makepkg for Arch, written by user GeneC, and then realised those monitoring this section of the forum might also be interested in this. Anyway the new tool is at https://github.com/gene-git/Arch-mkpkg if anyone is interested.

Scimmia · 2022-09-03 15:38:35

Not a new makepkg, a wrapper around makepkg for no good reason.

mcloaked · 2022-09-03 15:40:49

It uses the existing tool but also allows rebuilds if the deps are more recent in a very convenient way. So a wrapper yes, but makes it easier to do rebuilds when the deps change...

Scimmia · 2022-09-03 15:42:36

No, it's about *make*deps, not about deps. It's built on the assumption that things need rebuilt with a new toolchain (they don't) or that things are statically linked, which they generally aren't and this doesn't handle well anyway.

mcloaked · 2022-09-03 15:45:04

Yes apologies - it is about Make Deps, not deps. Which doesn't handle well?

2ManyDogs · 2022-09-03 15:45:17

Topics merged. Do not cross post.

mcloaked · 2022-09-03 15:48:10

Thank you for merging - and apologies. I only realised after the original post it was better here.

Lone_Wolf · 2022-09-04 10:59:29

Ordinarily, if the tool hasn't changed but something in the makedepends list has changed, running makephg will do nothing as it deems the package up to date.

The 'do nothing' is the default makepkg behavior IF a binary with the same version as in the PKGBUILD is present.
If no such binaries are present or the -f / --force flag is used, makepkg will run .

This behavior can also be overriden by using the

man makepkg wrote:

-f, --force
makepkg will not build a package if a built package already exists in the PKGDEST (set in makepkg.conf(5)) directory,
which may default to the current directory. This allows the built package to be overwritten.

Please explain what benefits your tool has using makepkg --force doesn't .

Last edited by Lone_Wolf (2022-09-04 10:59:58)

GeneArch · 2022-09-04 11:29:08

Let's take a specific example or 2 to illustrate.

When gcc/binutils are updated I want to rebuild my kernel packages. I could manually rebuild the package after increasing pkgrel. 'makepkg --force' doesn't help because while it does build a new package it has same version and pkgrel and so will not be installed after it's pushed to the repo. So instead you need to bump pkgrel and rebuild.

You could pay attention and check pacman logs to determine what has been updated and manually bump pkgrel and rebuild - that also means you need to know all the trigger dependencies in your head for every package you build - or you can automate.

In other words you can let mkpkg identify which of the packages (you deemed should trigger a rebuild) has been installed more recently than the last build you did. If you, like me run a tool which builds a rather large number packages every few hours or whatever, this simplifies the task enormously, Now the build tool builds everything and you can check see which packages have been updated sucessfully, failed to build or test or are up to date.

Now if you build many packages, then this automation is especially helpful/convenient - its also less error prone than human doing it.

Here's another example - arch started switching many packages to be compiled with lto. The gnu-efi package was subsequently compiled with " -flto -ffat-lto-object".
The refind boot manager statically links gnu-efi.

Now refind has not changed and so it's up to date as far as makepkg is concerned and yes will continue to work fine. However, I would like to know as soon as possible that refind builds and runs now that the gnu--efi library has been updated / changed (Given the large number of packages I build I doubt I'd remember the what every package uses anyway). I want to know as soon as possible for every package I build.

mkpg automates this for you and rebuilds when needed - unfortunately refind no longer builds against these particular lto flags - so build is broken.

You could have waited until refind itself gets an update and then discover - oh no it doesn't build. By doing this early and knowing refind has not changed - i now know with certainty the problem stems from the gnu-efi rebuild and not from a refind change - without even looking at refind source changes.

hope that helps

GeneArch · 2022-09-04 14:32:28

By the way I started coding this a couple days ago so there's likely room for improvement Nonetheless, it -seems- to be working for all my builds so far.

Lone_Wolf · 2022-09-05 08:55:03

You may be interested in rebuilderd .
(it's in repos and has a archlinux wiki page)

It started in april 2020 and has a different focus then your tool .
The rebuild-detector pacman hook and checkrebuild script could be useful for you or help to improve your code.

GeneArch · 2022-09-05 09:57:40

@lone_wolf thanks for the suggestion. I took a quick peek at rebuilderd and it looks it does everything my existing builder tool does and more.
The checkrebuild script looks useful. Definitely different focus. The latter seems to installs a alpm hook and called by pacman.

thank you!

GeneArch · 2022-10-13 01:07:13

I've released version 2.0 of mkpkg.

This adds fine control over how package dependencies trigger rebuilds based on their version. For example 'python>minor' would trigger a rebuild when the current major.minor version of python is greater than it was at the last build. Can also be explicit of course using 'python>3.10' for example. The readme file has more detail.

I also reorganized and now use poetry for packaging it up.

Source available here: https://github.com/gene-git/Arch-mkpkg

GeneArch · 2022-10-14 13:38:29

Its also now in aur https://aur.archlinux.org/packages/mkpkg - hope others find it useful.

mcloaked · 2022-10-14 16:12:33

Nice work on this package and very nice having dependencies looked after, particularly run time deps.

Arch Linux

#1 2022-09-03 13:21:50

An enhanced makepkg tool on github

#2 2022-09-03 15:36:41

Re: An enhanced makepkg tool on github

#3 2022-09-03 15:38:35

Re: An enhanced makepkg tool on github

#4 2022-09-03 15:40:49

Re: An enhanced makepkg tool on github

#5 2022-09-03 15:42:36

Re: An enhanced makepkg tool on github

#6 2022-09-03 15:45:04

Re: An enhanced makepkg tool on github

#7 2022-09-03 15:45:17

Re: An enhanced makepkg tool on github

#8 2022-09-03 15:48:10

Re: An enhanced makepkg tool on github

#9 2022-09-04 10:59:29

Re: An enhanced makepkg tool on github

#10 2022-09-04 11:29:08

Re: An enhanced makepkg tool on github

#11 2022-09-04 14:32:28

Re: An enhanced makepkg tool on github

#12 2022-09-05 08:55:03

Re: An enhanced makepkg tool on github

#13 2022-09-05 09:57:40

Re: An enhanced makepkg tool on github

#14 2022-10-13 01:07:13

Re: An enhanced makepkg tool on github

#15 2022-10-14 13:38:29

Re: An enhanced makepkg tool on github

#16 2022-10-14 16:12:33

Re: An enhanced makepkg tool on github

Board footer