You are not logged in.

#1 2017-05-28 13:19:11

amish
Member
Registered: 2014-05-10
Posts: 333

[solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Everytime I create a package via makepkg, I notice that it also packages a file named .BUILDINFO inside the package file.

And this file has list of ALL the packages installed in my system (obtained via pacman -Q)

This adds 50KB to all the packages that I create. (and also leaks the information about what is installed in my system)

Not sure why this information is required to be distributed inside the package file but I do not want it to be packaged.

So how do I remove this file without modifying /usr/bin/makepkg? (or make it not have list of all packages in my system)

Shouldn't it be optional?

I googled and searched this forum (for "remove BUILDINFO", "without BUILDINFO") but could not find anything related.

I noticed that official arch packages DO NOT have .BUILDINFO file. So does official packages use different makepkg?

Sorry in advance if I overlooked something somewhere.

Last edited by amish (2017-05-28 15:20:11)


Forum signature: I discuss. I put my thoughts strongly. But I definitely respect all developers and time they put in.

Offline

#2 2017-05-28 13:46:19

Scimmia
Bug Wrangler
Registered: 2012-09-01
Posts: 6,892

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Build in a clean chroot and there's no issue.

Offline

#3 2017-05-28 14:11:55

Eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 1,971

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Official Arch Linux packages do indeed have .BUILDINFO files, assuming they were build with a moderately recent version of pacman/makepkg. Though I can't make any promises about what other distros which model themselves after Arch and share a common package manager do.

e.g. Manjaro, Antergos.

As Scimmia said, if you build in a clean chroot (using makechrootpkg from the devtools package) you won't have this issue. The only packages that will be listed in the .BUILDINFO are the base-devel packages and the packages that your package depends on.

Offline

#4 2017-05-28 15:18:29

amish
Member
Registered: 2014-05-10
Posts: 333

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Yes you are right.  New official packages do have .BUILDINFO.

Earlier I checked 2-3 (small sized) packages which were actually old and hence did not have .BUILDINFO.

For clean chroot, I tried following:
https://wiki.archlinux.org/index.php/De … nience_Way

But it appears that it will install lots of things first. And I need to setup lots of things too.

Looked like too much time as well as space consuming.

Its like additional arch system just for the sake of building package. (and not have all packages listed in .BUILDINFO)

I build many packages.

What package X depends on, package Y may not depend on. And yet both packages will have all packages (whether it depends on OR not) in .BUILDINFO. Unless I have separate chroot for each package. (which is kind of crazy)

Finally I gave up and chose easy way out

i.e. commented this line in /usr/bin/makepkg

#       printf "installed = %s\n" "${pkglist[@]}"

PS: I still fail to understand why should .BUILDINFO have information about all the packages installed in system? Isnt "depends" in .PKGINFO enough? Atleast it should be optional for packager to decide whether to include list or not.

PS2: It also leaks the directory path where the package was built. So I also changed line which prints builddir in .BUILDINFO

Last edited by amish (2017-05-28 15:21:22)


Forum signature: I discuss. I put my thoughts strongly. But I definitely respect all developers and time they put in.

Offline

#5 2017-05-28 15:53:41

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 6,075

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Amish, this is beginning to sound like an x-y problem.

If you build packages for your own system, you already know where you are building and what is installed.

Are you building binary packages intended to be used by other people ?


Booting with apg Openrc, NOT systemd.
Automounting : not needed, i prefer pmount
Aur helpers : makepkg + my own local repo === rarely need them

Offline

#6 2017-05-28 16:47:11

amish
Member
Registered: 2014-05-10
Posts: 333

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Yes I am building for others.

But even if it was just for me only, I am still wondering why that information is needed to be stored in package?

Because everyone knows that base-devel group is required and rest of the dependency is already stored in depends directives

So why this redundant information? Am curious to know.

Kind of goes against arch philosophy of keeping it minimal and simple


Forum signature: I discuss. I put my thoughts strongly. But I definitely respect all developers and time they put in.

Offline

#7 2017-05-28 19:42:33

Scimmia
Bug Wrangler
Registered: 2012-09-01
Posts: 6,892

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

It's necessary for reproducible builds.

Offline

#8 2017-05-29 00:07:07

Allan
Supreme Leader
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,727
Website

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

If you are building for other people, you should definitely be using a clean chroot.  It is the only way to ensure you don't add unspecified dependencies.

Offline

#9 2017-05-29 01:23:03

amish
Member
Registered: 2014-05-10
Posts: 333

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Thats what I am curious to know. How and why is it necessary?

Isnt "makedepends" and "depends" inside .PKGINFO sufficient to re-build?

Isnt "makepkg -s" sufficient?

Even manually:
To reproduce a build ... we install base-devel. (i.e. all packages in base-devel)

And also install package dependencies (which is stored in .PKGINFO or PKGBUILD)

So why is it required to have that information again in .BUILDINFO?

@Allan
I assume the dependencies you are talking about is same as what is put in PKGBUILD.

So instead of:

run_pacman -Q

why cant we have

run_pacman -Q $(run_pacman -Qgq base-devel) ${depends[@]} ${makedepends[@]} ...

This way it will not disclose everything in .BUILDINFO and work in chroot and non-chroot environment.


Forum signature: I discuss. I put my thoughts strongly. But I definitely respect all developers and time they put in.

Offline

#10 2017-05-29 02:06:05

Eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 1,971

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Because in the real world, this is a thing: https://wiki.gentoo.org/wiki/Project:Qu … pendencies

It is important to know the full set of packages that the system knew about at the time the package was compiled. This is also why clean chroot builds are recommended. So too, the builddir is important because software can record the directory it was compiled in.

Last edited by Eschwartz (2017-05-29 02:06:54)

Offline

#11 2017-05-29 02:22:36

fsckd
Forum Fellow
Registered: 2009-06-15
Posts: 4,173

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

amish wrote:

And also install package dependencies (which is stored in .PKGINFO or PKGBUILD)

So why is it required to have that information again in .BUILDINFO?

Dependencies listed in .PKGINFO are insuffient for reproducable builds.


aur S & M :: forum rules :: Community Ethos
Resources for Women, POC, LGBT*, and allies

Offline

#12 2017-05-29 02:55:21

amish
Member
Registered: 2014-05-10
Posts: 333

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Finally that link explains certain things.

But then as mentioned in that link itself -its more a case of "lazy devs" who do not do (--disable-XXX) in Configure or equivalent scripts.

I am assuming that this forum is right place for discussion (or atleast for my own understanding)

I am wondering few things.

1) What is the purpose of .BUILDINFO?

For developers who may want to rebuild the package - right?

Then shouldn't it be placed in .src.pkg.tar.xz? Why is it required to be  placed in binary pkg also?

2) Is the purpose of .BUILDINFO to specify packages that were in build system when it was built?

If yes, then why would "end user" of binary package be interested in it? So why include it in binary package?

OR

3) Is the purpose of .BUILDINFO to specify list of packages "end-user" may require to re-build the package?

If yes, then why does it need to have whole list of packages?
Isnt .PKGINFO (which has list of depends and makedepends and optdepends) file enough?

And even if we have to have .BUILDINFO - then isnt my suggestion above of listing specific packages instead of whole "pacman -Q" more appropriate?

This way it works with or without chroot.

Last edited by amish (2017-05-29 02:56:38)


Forum signature: I discuss. I put my thoughts strongly. But I definitely respect all developers and time they put in.

Offline

#13 2017-05-29 03:12:03

Eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 1,971

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

1) src.tar.z is not a built package, src.tar.gz is e.g. what you get in the AUR. Built packages contain the environment in which they were built.

2) Yes. End users may not be interested in it, but most end users are interested in very few things at all. The people working on reproducible builds care though, as do people debugging why packages fail to work. e.g. a recent example is that I built Firefox/Thunderbird with gcc7 without realizing it and didn't know why it wasn't working, but the .BUILDINFO contains the versions of installed packages and could tell me that I had gcc7 installed.

3) No, that's what the .PKGINFO is for, containing the "depends" info.

4) If you think you are going to convince Allan to stop using .BUILDINFO, you have another think coming. We already know all the arguments, after more extensive discussion and brainstorming than you have used in this merely-day-old thread.
The decision to add .BUILDINFO was not made without due consideration.

Also, no one should distribute built packages without using clean chroots. Not because they have something to hide, merely because it is the best way to ensure the packages actually work as expected.

Offline

#14 2017-05-29 04:41:59

Allan
Supreme Leader
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,727
Website

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Eschwartz wrote:

4) If you think you are going to convince Allan to stop using .BUILDINFO, you have another think coming.

My opinions are correct.

Offline

#15 2017-05-29 05:47:08

amish
Member
Registered: 2014-05-10
Posts: 333

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

I never said stop using BUILDINFO. (just asked if it is really necessary and why and if I want to remove it how do i do it.)

If you read my first post then I am simply asking if there is way to NOT list packages. (or make it optional).

My own 1 line hack in to "/usr/bin/makepkg" does NOT remove BUILDINFO. It keeps the file but without build time package information.

Not every developer has resources to have separate chroot for each package they are building.

And  I am not insisting on anything. I had already found my solution and had marked discussion as [solved]

Rest of the posts were for my own knowledge / understanding only.


Forum signature: I discuss. I put my thoughts strongly. But I definitely respect all developers and time they put in.

Offline

#16 2017-05-29 06:03:47

fsckd
Forum Fellow
Registered: 2009-06-15
Posts: 4,173

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

It is perfectly fine and reasonable to ask questions to improve your understanding on how and why. Please continue to do so.

Reproducible builds is something people interested in the security of Arch Linux have been working on for the past year. It is a very useful thing to have, especially from a security perspective, but that may not be obvious to everyone.


aur S & M :: forum rules :: Community Ethos
Resources for Women, POC, LGBT*, and allies

Offline

#17 2017-05-29 07:00:59

amish
Member
Registered: 2014-05-10
Posts: 333

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Ok I read more about BUILDINFO.

(Question I was asking was why BUILDINFO is required but answer I was getting was why chroot is required)

Here is why BUILDINFO is required. (For people who land here in future and have no clue about why it is required)

Main purpose of BUILDINFO is NOT what I thought was. i.e. it is not really to rebuild package or detect dependencies. (because for rebuilding a package PKGBUILD is enough)

But to verify that person who built the package has not altered any source code.

BUILDINFO includes all the packages along with version used to build any package.

If someone wants to verify that the packager has no bad intention.

All they need is to install the packages specified in BUILDINFO (same versions)

And build it. (reproduce it)

If reproduced build and packaged build matches .. that means there is no source code change made by packager.

Otherwise it should raise eyebrows!

Last edited by amish (2017-05-29 07:12:36)


Forum signature: I discuss. I put my thoughts strongly. But I definitely respect all developers and time they put in.

Offline

#18 2017-05-29 07:56:19

Allan
Supreme Leader
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,727
Website

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

amish wrote:

Not every developer has resources to have separate chroot for each package they are building.

You only need one.   It is 620M for the base chroot.

Offline

#19 2018-06-13 10:57:39

dreieck
Member
Registered: 2012-10-08
Posts: 9

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

amish wrote:

Everytime I create a package via makepkg, I notice that it also packages a file named .BUILDINFO inside the package file.

And this file has list of ALL the packages installed in my system (obtained via pacman -Q)

I was also annoyed by this, but was more annoyed by the time and CPU ressources each pacman -Q run consumes. (I have a single core machine.)

I am building just for myself. And do not wantr to use a chroot because of added maintenance requirements.

I solved the problem by patching makepkg. I created a customizepkg-hook, so that when pacman is updated (pacman provides makepkg), it will be automatically rebuilt, applying my patch.

The patch to the pacman source tree is:

diff -r -U1 pacman-5.1.0.orig/scripts/makepkg.sh.in pacman-5.1.0.new/scripts/makepkg.sh.in
--- pacman-5.1.0.orig/scripts/makepkg.sh.in     2018-06-13 12:14:30.313734761 +0200
+++ pacman-5.1.0.new/scripts/makepkg.sh.in      2018-06-13 12:18:43.626567915 +0200
@@ -736,3 +736,3 @@
        write_pkginfo > .PKGINFO
-       write_buildinfo > .BUILDINFO
+       # write_buildinfo > .BUILDINFO # Disabled in user-built variant of 'makepkg'.
 

And it get's applied by adding a line (patch -Np1 -i <path-to-above-patch>) to the prepare()-function in the PKGBUILD via customizepkg-scripting.

Offline

#20 2018-06-13 11:25:01

progandy
Member
Registered: 2012-05-17
Posts: 2,893

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

There is a simple solution that works without patching makepkg:

$ cat ~/.config/pacman/makepkg.conf 
function write_buildinfo() {
	msg2 "Don't write .BUILDINFO"
}

| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Online

#21 2018-06-13 15:14:23

Eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 1,971

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Whyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

It calls pacman *once*. Well, twice if you count a mild inefficiency which I've already posted a patch to fix.

If you don't explicitly use -dd, it calls pacman -T anyways.
If you use VCS sources, it will unconditionally call pacman -T.
If you use -r, it calls -qq several times to figure out which packages to uninstall.
If you had to install any dependencies, it loads pacman to do a full-blown transaction.

The first time you execute pacman, it reads the entire database, which can be slightly slow, then afterwards it is in the HDD disk cache. The number of cores is not really a factor here, even on my very old single-core laptop (which I actually no longer use). You're almost assuredly running pacman at least once, close enough to the time buildinfo is generated that the database will still be in the disk cache.

Offline

#22 2018-06-14 10:43:00

dreieck
Member
Registered: 2012-10-08
Posts: 9

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Eschwartz wrote:

Whyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy

It calls pacman *once*.

On my system I see a run of pacman -Qi <all packages installed on my system> (a very long list of arguments) when makepkg is generating the .BUILDINFO. And this run consumes a lot of ressources for about 2 minutes on my system. Every time. Even if I build two packages short after each other.

Last edited by dreieck (2018-06-14 10:44:13)

Offline

#23 2018-06-14 22:15:44

Eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 1,971

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

Two minutes in order to run pacman -Qi > /dev/null seems quite horrible, but this is I think the first time I heard of it being that slow for anyone, no matter how old/slow their HDD.

Offline

#24 2018-06-15 07:59:48

dreieck
Member
Registered: 2012-10-08
Posts: 9

Re: [solved][with a hack] Remove .BUILDINFO from pkg.tar.xz file?

progandy wrote:

There is a simple solution that works without patching makepkg:

$ cat ~/.config/pacman/makepkg.conf 
function write_buildinfo() {
	msg2 "Don't write .BUILDINFO"
}

Thanks a lot, did not know that those functions can be overridden in the makepkg.conf.

This is elegant solution.

Offline

Board footer

Powered by FluxBB