Preserve timestamps of files installed in package()

jlindgren · 2019-02-16 16:21:21

How can I tell makepkg not to modify the timestamps of files in my package?

(Use case: I frequently build and install software locally, and I want to start using pacman for the installation step. I am installing two packages, and package B depends on package A. If all the headers in package A get a new timestamp every time the package is built and installed, then building package B takes much longer because every source file including those headers is recompiled.)

For example, after I run makepkg on the following PKGBUILD, I want the timestamp of testfile to be Jan 1, 2001. But some part of makepkg is updating the timestamp to today's date.

Test case:

pkgname=testpkg
pkgver=0.1
pkgrel=1
pkgdesc="Test package"
arch=('x86_64')

package() {
  # create a file with a timestamp of Jan 1, 2001
  touch -t 200101010000 "$pkgdir/testfile"
}

I've verified that "touch" does create a file with a timestamp of Jan 1, 2001. But at some point after package() returns, the timestamp of testfile is updated to today's date.

$ ls -l pkg/testpkg
total 0
-rw-r--r-- 1 john john 0 Feb 16 11:03 testfile

Why is makepkg modifying the timestamp, and how can I change this behavior?

Last edited by jlindgren (2019-02-16 16:46:30)

graysky · 2019-02-16 16:29:09

I think you need to modify the DTS after the cp step. I don't know why the original DTS isn't retained (same with rsync -a).

Last edited by graysky (2019-02-16 16:30:46)

jlindgren · 2019-02-16 16:42:14

graysky wrote:

I think you need to modify the DTS after the cp step. I don't know why the original DTS isn't retained (same with rsync -a).

I am not sure that I understand, or possibly my example was confusing. To be clear, the problem still occurs if you remove the "cp -a" step from my example. It's the timestamp of the original installed file (pkg/testpkg/testfile) that is being modified, not the timestamp of the copy.

I'll remove the "cp -a" step from my example so that it's more clear. Can you provide an example of the solution you are recommending?

loqs · 2019-02-16 16:54:00

https://git.archlinux.org/pacman.git/co … 333242e79b

graysky · 2019-02-16 16:56:55

Right, I'm saying that I don't know what is modifying the DTS and that if you want it to be different, modify it after it is written to the destination. Maybe someone more knowledgeable can suggest a more elegant solution.

Using your code:

package() {
  cat /dev/null > "$pkgdir/testfile"
  touch -t 200101010000 "$pkgdir/testfile"
}

Alternatively, you can use `touch -r $SOURCE $DESTINATION` to just use the DTS from $SORUCE on $DESTINATION.

Piri · 2019-02-16 17:53:08

Makepkg seems to modify timestamps of $srcdir as well as of $pkgdir.
The line that seems responsible for $pkgdir:

https://git.archlinux.org/pacman.git/tr … sh.in#n708

Last edited by Piri (2019-02-16 17:53:33)

jlindgren · 2019-02-16 18:39:35

loqs wrote:

https://git.archlinux.org/pacman.git/co … 333242e79b

Thank you. The commit message for that change explains what is happening and why.

This behavior of makepkg (changing *all* timestamps to $SOURCE_DATE_EPOCH) seems a bit heavy-handed. Debian's recommendation[1] is to rewind only *newer* timestamps back to $SOURCE_DATE_EPOCH, which seems to me like a more considered approach, and would solve my particular problem:

One can reasonably assume that all source timestamps are before SOURCE_DATE_EPOCH and all builds take place after it. This means we can efficiently both preserve source-based timestamps and omit build-specific timestamps, by rewriting timestamps more recent than SOURCE_DATE_EPOCH back to the latter.

[1] https://reproducible-builds.org/specs/s … ate-epoch/

jlindgren · 2019-02-16 18:48:21

https://bugs.archlinux.org/task/61785

eschwartz · 2019-02-17 01:19:37

Why is the package being continually rebuilt?

The timestamp on a file is in the vast, vast majority of cases, the timestamp of the time the "make install" step executed "install -m644 foo/myheader.h $(DESTDIR)/usr/include/foo/myheader.h" (which does not save timestamps).

The timestamp of a file checked out from git is already nondeterministically "whenever that specific file was last git pulled".

The timestamp of a file in a release tarball is traditionally nearly as meaningless, depending on the archival method it may be directly derived from the timestamp of a git commit/tag, it may be based on the time "tar -c" was run, or it may be the last time the file was touched on the developer's machine (which is not the same as the last time it changed).

A package that is being rebuilt without functional changes to the source code maybe shouldn't be being rebuilt. A package that is being rebuilt because of functional changes maybe should be causing software to be rebuilt.

I assert that all packages already had their timestamps changed at random based on the install time, and that your package and your package alone, which played weird and wonderful games with the timestamp, is the sole, singular package which did not fall prey to the timestamp modification nature of the standard install/cp commands.

jlindgren · 2019-02-17 04:19:18

eschwartz wrote:

Why is the package being continually rebuilt?

There are two packages, let's call them A and B. I'm rebuilding them frequently because I develop them.

eschwartz wrote:

A package that is being rebuilt without functional changes to the source code maybe shouldn't be being rebuilt. A package that is being rebuilt because of functional changes maybe should be causing software to be rebuilt.

It's not quite that simple. In this case, package B depends on the public headers installed by package A. When I make a small change to A that doesn't affect those public headers, I want to be able to do a quick delta rebuild of both packages without triggering a recompile of every single source file in B. On the other hand, when I *do* change one of those headers in A, I want "make" to see the changed timestamp and rebuild everything in B that depends on that particular header.

I assert that all packages already had their timestamps changed at random based on the install time, and that your package and your package alone, which played weird and wonderful games with the timestamp, is the sole, singular package which did not fall prey to the timestamp modification nature of the standard install/cp commands.

Maybe I like weird and wonderful games ...

In all seriousness, preserving the timestamps during the "make install" step doesn't seem that weird to me. But there are probably other ways I could achieve a fast rebuild, like modifying the build environment of package B to use the uninstalled/local headers of package A instead of the installed ones. I might try that approach next.

Lone_Wolf · 2019-02-17 11:52:25

I want to be able to do a quick delta rebuild of both packages without triggering a recompile of every single source file in B.

Have you tried enabling ccache through options() array ?
https://wiki.archlinux.org/index.php/Ccache

jlindgren · 2019-02-18 15:38:27

Thank you all for the suggestions. I've worked out a script that builds (make) and installs (make install) both packages to a temporary directory without involving makepkg, and then I use makepkg/pacman to install the contents of that directory system-wide. This ends up being better for my workflow because I can work out any incompatibilities between the two packages before making any system-wide changes.

I won't mark the thread as "solved" because the original question (preserving timestamps in makepkg) remains unanswered. If someone finds a way to do this without patching makepkg, I'd still be interested to know, but it's less relevant/necessary for me now.

sebalis · 2024-03-30 21:46:27

I came across this thread today and would like to add two remarks.

First, and this might have been obvious to @jlindgren after having been pointed to SOURCE_DATE_EPOCH, it is possible to set that variable before calling makepkg to create a package with an older timestamp. For example:

SOURCE_DATE_EPOCH=$( stat -c%Y /some/file ) makepkg

would produce a package with timestamps set to the modification time of an existing file. I used a file that was part of the existing installation of the package that I wanted to create. I wanted to recreate the package file to be able to reinstall that specific version (should have configured an AUR package cache really).

Second, for any other logic, it should not be hard to modify makepkg. The touch commands in the script are in all but one case executed via find commands, and these could be given an extra -newer [file] or -newermt [seconds] option to achieve the desired effect.

Arch Linux

#1 2019-02-16 16:21:21

Preserve timestamps of files installed in package()

#2 2019-02-16 16:29:09

Re: Preserve timestamps of files installed in package()

#3 2019-02-16 16:42:14

Re: Preserve timestamps of files installed in package()

#4 2019-02-16 16:54:00

Re: Preserve timestamps of files installed in package()

#5 2019-02-16 16:56:55

Re: Preserve timestamps of files installed in package()

#6 2019-02-16 17:53:08

Re: Preserve timestamps of files installed in package()

#7 2019-02-16 18:39:35

Re: Preserve timestamps of files installed in package()

#8 2019-02-16 18:48:21

Re: Preserve timestamps of files installed in package()

#9 2019-02-17 01:19:37

Re: Preserve timestamps of files installed in package()

#10 2019-02-17 04:19:18

Re: Preserve timestamps of files installed in package()

#11 2019-02-17 11:52:25

Re: Preserve timestamps of files installed in package()

#12 2019-02-18 15:38:27

Re: Preserve timestamps of files installed in package()

#13 2024-03-30 21:46:27

Re: Preserve timestamps of files installed in package()

Board footer