makepkg always reverts local code changes when using version control

Malvineous · 2012-06-10 04:00:35

Hi all,

I have some packages in the AUR that compile the latest git versions of software. This is done by putting git commands in the build() function to download the code from the repository and then compile it (here's an example).

After compilation the source tree gets littered with build products, which need to be removed before recompiling a new version to avoid errors. There are many ways to do this, but all of them will (must) remove any changes you might have made to the code to avoid errors after the upstream code has changed.

Sometimes however, you want to keep these changes and compile your modified code (like when the latest upstream won't compile and you're trying to fix it, so you can send in a patch.) makepkg has the -e option to do this with normal packages, which prevents the original source from being extracted over the top of your changes. Unfortunately the -e option doesn't work when the code is coming out of a version control repository, because makepkg isn't extracting any files at all. So "makepkg -e" just calls build(), which then happily wipes out any changes you've made to the code before recompiling.

The problem is that build() is only supposed to compile the code, but when the code is coming from a version control repository, build() is also downloading the code so the -e option has no effect.

Is there a way around this? It seems like PKGBUILD files need a download() function which handles the version control retrieval, so that build() again only compiles the code. Looking at the docs I can't see anything like this that already exists, so please let me know if I'm missing something!

tomk · 2012-06-10 06:34:37

Comparing your given example to /usr/share/pacman/PKGBUILD-git.proto (provided by the abs package), it seems you have omitted this:

rm -rf "$srcdir/$_gitname-build"
git clone "$srcdir/$_gitname" "$srcdir/$_gitname-build"
cd "$srcdir/$_gitname-build"

Last edited by tomk (2012-06-10 06:35:08)

Malvineous · 2012-06-10 06:53:45

No, I haven't omitted it, but it's a very inefficient way of doing things. Those commands wipe the entire repository and download the *entire* history every time you rebuild, whereas my PKGBUILD either downloads only the latest version (no history) or only the changes since you last recompiled.

But either way, you still lose any changes you have made locally when you run 'makepkg -e'.

tomk · 2012-06-10 07:28:22

Building in $srcdir/$_gitname-build instead of $srcdir/$_gitname means that $srcdir/$_gitname does not get "littered with build products", which you cited above as the root cause of your problem.

Obviously, you can do it any way you want, but you asked if you were missing something, so I answered.

Malvineous · 2012-06-10 08:12:36

Sorry, I meant was I missing some functionality that makepkg provides, not was I literally missing some commands!

Compiling in a copy of the repository could work, but it seems a bit clunky. It still loses any changes you make, because 'git clone' doesn't copy changes you haven't committed to a repository. And once you commit them, the next 'git pull' can easily fail due to conflicts. So this method requires a lot more work.

The way I have done it works without requiring copies or committing every change to the local copy of the repository, but there's no way to skip running some of the commands. Maybe what I need to do is add a download() function to makepkg/PKGBUILD which can be skipped with 'makepkg -e', which would solve the problem once and for all.

But thanks for your suggestions!

Allan · 2012-06-10 11:26:52

Doing

rm -rf "$srcdir/$_gitname-build"
git clone "$srcdir/$_gitname" "$srcdir/$_gitname-build"
cd "$srcdir/$_gitname-build"

does not redownload anything. It clones the repo that has already been downloaded. Then you can apply your patch as you would in a non-git PKGBUILD

Malvineous · 2012-06-10 12:20:31

Yes I now realise that after installing abs and looking at the PKGBUILD-git.proto file. Interestingly PKGBUILD-git.proto does a clone of the entire repository though (history and all) whereas the wiki recommends using --depth 1 to avoid downloading all the history as it's not needed in this case. Maybe PKGBUILD-git.proto should be updated to include that?

At any rate, the problem remains that you have to make a patch of your changes each time you make a change. The issue I often come up against with git-based PKGBUILD files is that the upstream git has a build-related bug in it, and I need to tweak the code and recompile until it works (often many times, not just once or twice.) Missing #include lines are particularly common when cross compiling for win32. There is currently no way of editing the code and re-running makepkg without your changes getting undone.

Trilby · 2012-06-10 12:23:48

Comment the git lines in the PKGBUILD.

Malvineous · 2012-06-10 12:29:49

Ha, well, yes, that would do the trick wouldn't it? Still, it'd be nice if the -e option worked as advertised in these cases :-)

I think this suggestion is by far the quickest though!

Trilby · 2012-06-10 19:04:53

Perhaps if there is a environment variable that makepkg sets when passed the -e flag, or if one could access the command line parameters from the PKGBUILD, you could put in a conditional test and only run git if the -e flag was not passed.

Malvineous · 2012-06-11 01:41:01

Hmm, that's true. I was thinking I'd have to parse the command line somehow, but now that you mention it, makepkg does set $NOEXTRACT to 1 if the -e option is specified. By only running the git commands if $NOEXTRACT == 0 it would solve the problem! Good suggestion - thanks!

Malvineous · 2012-06-11 14:24:51

Ok, this works well and I've made a patch to do this with all the version control templates in the abs package. I'm not sure where to send the patch though! The bug tracker doesn't seem to have many patches on it, and either does the arch-dev-public mailing list. Is there a preferred method for sending in patches to Arch packages?

tomk · 2012-06-11 15:00:59

You're submitting a feature request against the abs package, so the bug tracker is the right place.

Allan · 2012-06-11 22:18:52

ABS is dead so this will never get included. Also, th way -git, -svn etc packages are handled is getting a complete overhaul for the next pacman release which will invalidate any change made to the PKGBUILD prototype.

sitquietly · 2012-06-11 22:50:22

Allan wrote:

ABS is dead so this will never get included.....

Does that mean "abs is feature complete and we do not anticipate updating it" or does it mean "we will be abandoning abs as a feature of Archlinux"??

Arch Linux

#1 2012-06-10 04:00:35

makepkg always reverts local code changes when using version control

#2 2012-06-10 06:34:37

Re: makepkg always reverts local code changes when using version control

#3 2012-06-10 06:53:45

Re: makepkg always reverts local code changes when using version control

#4 2012-06-10 07:28:22

Re: makepkg always reverts local code changes when using version control

#5 2012-06-10 08:12:36

Re: makepkg always reverts local code changes when using version control

#6 2012-06-10 11:26:52

Re: makepkg always reverts local code changes when using version control

#7 2012-06-10 12:20:31

Re: makepkg always reverts local code changes when using version control

#8 2012-06-10 12:23:48

Re: makepkg always reverts local code changes when using version control

#9 2012-06-10 12:29:49

Re: makepkg always reverts local code changes when using version control

#10 2012-06-10 19:04:53

Re: makepkg always reverts local code changes when using version control

#11 2012-06-11 01:41:01

Re: makepkg always reverts local code changes when using version control

#12 2012-06-11 14:24:51

Re: makepkg always reverts local code changes when using version control

#13 2012-06-11 15:00:59

Re: makepkg always reverts local code changes when using version control

#14 2012-06-11 22:18:52

Re: makepkg always reverts local code changes when using version control

#15 2012-06-11 22:50:22

Re: makepkg always reverts local code changes when using version control

Board footer