Suggestion: automatic build system

miko · 2008-11-17 12:26:01

Hello,
I would like to hear some opinions (and possibly some solutions) how to make sure archlinux repositories are in a "stable" state any given time (and hope that it would be the goal for the developers/maintainers of archlinux).

I have no idea how the official packages are created, but I would suspect a kind of algorithm like this one:
For each PACKAGE do
create a new clean base archlinux system from scratch (using VirtualBox or other virtualization software)
makepkg --clean --syncdeps --noconfirm --rmdeps PACKAGE
end

(and as I meet some errors from time to time, while upgrading packages, or building some from ABS tree, i suspect that this process is not that perfect)

This would solve all the issues with missing fields (and their correct values) like arch=(), depends=() and makedepends=(), because the compilation of such package would fail.

I would require each newly committed package to be compiled this way, and to be published in the official repository only if it has built succesfully.

There are some cases (pacman, gcc, glibc upgrade) when all already published packages should be (possibly automatically) recompiled, and some of them (those which no longer compile) should be updated and republished. I just have filled bugs for some packages from extra repository containing options=(NOLIBTOOL) instead of options=(!libtool), and such PKGBUILDs should not be present in official repositories at this time IMO.

I guess there are other issues which would be detected and solved by such automatic build system. In the extreme case, after each new package committed to ABS, the whole set of packages should be recompiled to be sure that archlinux repositories are in a stable state. I know that this seems like a lot of work, but with many users contributing their spare CPU cycles (think: cloud computing, or Archlinux@Home) this would set archlinux to a higher level of quality.

There is one thing I was always missing from PKGBUILD. I mean something like "test=test.sh", which would work like "install=install.sh", except it would run functional test defined in such script when required. The scenario could be like this:
--- openssh PKGBUILD --
pkgname=openssh
testscript="test.sh"
--- openssh test.sh --
test() {
ssh -V
exit $?
}

Then if someone has upgraded openssl and openssh no longer works (because of missing libssl.so), then pacman --test openssh would also run this test and return error.
So after pacman -S openssl I would run:
pacman --test --all
and would get a list of packages which no longer work (because of a missing libssl.so) and which would require a recompilation against a newer openssl.

Regards,
miko

Allan · 2008-11-17 12:41:52

miko wrote:

Hello,
I would like to hear some opinions (and possibly some solutions) how to make sure archlinux repositories are in a "stable" state any given time (and hope that it would be the goal for the developers/maintainers of archlinux).
I have no idea how the official packages are created, but I would suspect a kind of algorithm like this one:
For each PACKAGE do
create a new clean base archlinux system from scratch (using VirtualBox or other virtualization software)
makepkg --clean --syncdeps --noconfirm --rmdeps PACKAGE
end
(and as I meet some errors from time to time, while upgrading packages, or building some from ABS tree, i suspect that this process is not that perfect)
This would solve all the issues with missing fields (and their correct values) like arch=(), depends=() and makedepends=(), because the compilation of such package would fail.
I would require each newly committed package to be compiled this way, and to be published in the official repository only if it has built succesfully.

Virtualization software.... getting a bit carried away there. Just a nice chroot would be fine. And there are nice scripts for creating one (mkarchroot) and building packages in it (makechrootpkg) in the devtools programs.

There are some cases (pacman, gcc, glibc upgrade) when all already published packages should be (possibly automatically) recompiled, and some of them (those which no longer compile) should be updated and republished. I just have filled bugs for some packages from extra repository containing options=(NOLIBTOOL) instead of options=(!libtool), and such PKGBUILDs should not be present in official repositories at this time IMO.

This option would have worked at the time the package was commited to the repos. It was only with pacman-3.2 that deprecated options were removed and correct options was enforced.

I guess there are other issues which would be detected and solved by such automatic build system. In the extreme case, after each new package committed to ABS, the whole set of packages should be recompiled to be sure that archlinux repositories are in a stable state. I know that this seems like a lot of work, but with many users contributing their spare CPU cycles (think: cloud computing, or Archlinux@Home) this would set archlinux to a higher level of quality.

So with every package we update (and note there are many per day) the entire repo gets rebuilt and I assume pushed for users to download in order to keep there system in sync with the Arch repos. Think of the number of packages on your system. That would require you to download (probably) several gigabytes of updates every time a single package was updated....

There is one thing I was always missing from PKGBUILD. I mean something like "test=test.sh", which would work like "install=install.sh", except it would run functional test defined in such script when required. The scenario could be like this:
--- openssh PKGBUILD --
pkgname=openssh
testscript="test.sh"
--- openssh test.sh --
test() {
ssh -V
exit $?
}
Then if someone has upgraded openssl and openssh no longer works (because of missing libssl.so), then pacman --test openssh would also run this test and return error.
So after pacman -S openssl I would run:
pacman --test --all
and would get a list of packages which no longer work (because of a missing libssl.so) and which would require a recompilation against a newer openssl.

Or you could use ldd on binaries and see which packages require the new library. Again there is a script to help with this in the devtools package (lddd). Most library upgrades go to [testing] first to avoid such problems occuring in the [core] and [extra] repos. But sometimes a package that it appears no-one using [testing] has installed gets missed and so a bug report is needed. We have lists for libraries that require a large number of rebuilds so this improves each time.

Daenyth · 2008-11-17 12:49:02

This suggestion seems very poorly thought out and with no regard to any existing processes.

miko · 2008-11-17 14:06:20

Allan wrote:

miko wrote:
I would require each newly committed package to be compiled this way, and to be published in the official repository only if it has built succesfully.
Virtualization software.... getting a bit carried away there. Just a nice chroot would be fine. And there are nice scripts for creating one (mkarchroot) and building packages in it (makechrootpkg) in the devtools programs.

chroot is fine (if your current kernel is up-to-date), I will look at the tools. I just meant to use pristine system to do my compilations.

There are some cases (pacman, gcc, glibc upgrade) when all already published packages should be (possibly automatically) recompiled, and some of them (those which no longer compile) should be updated and republished. I just have filled bugs for some packages from extra repository containing options=(NOLIBTOOL) instead of options=(!libtool), and such PKGBUILDs should not be present in official repositories at this time IMO.
This option would have worked at the time the package was commited to the repos. It was only with pacman-3.2 that deprecated options were removed and correct options was enforced.

So does that mean that it is a known and accepted fact that there could be some packages in the repos which do not compile on a current system?
Is there (or should be) an "age limit" of how long can a package live in a repository without being recompiled? Like no longer than 6 months? (And I do mean "recompiling to check if it still compiles on a current system", not necessarily "recompiling to publish the new version").

I guess there are other issues which would be detected and solved by such automatic build system. In the extreme case, after each new package committed to ABS, the whole set of packages should be recompiled to be sure that archlinux repositories are in a stable state. I know that this seems like a lot of work, but with many users contributing their spare CPU cycles (think: cloud computing, or Archlinux@Home) this would set archlinux to a higher level of quality.
So with every package we update (and note there are many per day) the entire repo gets rebuilt and I assume pushed for users to download in order to keep there system in sync with the Arch repos. Think of the number of packages on your system. That would require you to download (probably) several gigabytes of updates every time a single package was updated....

Note that I meant "recompiled", not "published". I meant of recompilation of every package as a way to ensure that this package still compiles. If it does, then nothing needs to be published. OTOH if it does not, you know that this package also requires some update, and after this update you then publish it.

There is one thing I was always missing from PKGBUILD. I mean something like "test=test.sh", which would work like "install=install.sh", except it would run functional test defined in such script when required. The scenario could be like this:
--- openssh PKGBUILD --
pkgname=openssh
testscript="test.sh"
--- openssh test.sh --
test() {
ssh -V
exit $?
}
Then if someone has upgraded openssl and openssh no longer works (because of missing libssl.so), then pacman --test openssh would also run this test and return error.
So after pacman -S openssl I would run:
pacman --test --all
and would get a list of packages which no longer work (because of a missing libssl.so) and which would require a recompilation against a newer openssl.
Or you could use ldd on binaries and see which packages require the new library.

Yes, but you have got an idea. Sometimes a binary wouldn't run because of a missing directory, wrong access rights etc. And it would be up to package maintaner to add different tests for such cases, building a kind of "knowledge base" over the time.

Again there is a script to help with this in the devtools package (lddd). Most library upgrades go to [testing] first to avoid such problems occuring in the [core] and [extra] repos. But sometimes a package that it appears no-one using [testing] has installed gets missed and so a bug report is needed. We have lists for libraries that require a large number of rebuilds so this improves each time.

I know this is lots of work for people, this is why I suggested an automatic attempt - to cover 100% of packages.

Daenyth wrote:

This suggestion seems very poorly thought out and with no regard to any existing processes.

Yes, as I have written, I admit I have no idea how the arch package system works under the surface, and so I don't know all the tools. I know that other distros are using some form of continuous integration, to such degree that you know who's commit caused the package to compile or to fail to compile on differenf platforms.
OTOH I am not so much interested in the processes and tools, but in the final effect, which is a high quality of archlinux repositories.
But I am glad that the devs are aware of this and are using some helper tools and processes. And I hope that some day I will be able to compile every package from ABS on a freshly created archlinux out of the box.

Daenyth · 2008-11-17 14:09:49

I think that most of this could be dropped -- but a test suite might be interesting. Shouldn't be hard to implement. At the very least, I use "make test" in my build() when the package supports it.

miko · 2008-11-17 15:51:34

Daenyth wrote:

I think that most of this could be dropped -- but a test suite might be interesting. Shouldn't be hard to implement. At the very least, I use "make test" in my build() when the package supports it.

This works for the build time ("test and forget"). My proposition would work at runtime (ie. every time the user requested it), so users would run it from time to time even for the oldest packages ("test if still working"). Those are complementary solutions.

Arch Linux

#1 2008-11-17 12:26:01

Suggestion: automatic build system

#2 2008-11-17 12:41:52

Re: Suggestion: automatic build system

#3 2008-11-17 12:49:02

Re: Suggestion: automatic build system

#4 2008-11-17 14:06:20

Re: Suggestion: automatic build system

#5 2008-11-17 14:09:49

Re: Suggestion: automatic build system

#6 2008-11-17 15:51:34

Re: Suggestion: automatic build system

Board footer