[SOLVED] Second execution of `makepkg` causes `patch does not apply`

leuko · 2022-08-23 15:14:40

I packaged a project that integrates a submodule and apply patches both to the superproject and to the submodule. The problem: When I build the PKGBUILD for the second time, the patch to the superproject is successful but the patch to the submodule fails with `patch does not apply`. The reason is that the submodule is checked out with patches already applied (somehow). I do not understand why the superproject is clean when I restart `makepkg` but not the submodule. Here are the relevant sections:

_name=xrt                                                                                                                                                                                                                                    
pkgbase=$_name-git
...
source=(                                                                                                                                                                                                                                     
        $_name::git+https://github.com/xilinx/XRT                                                                                                                                                                                            
        git+https://github.com/Xilinx/dma_ip_drivers                                                                                                                                                                                         
        xrt-fixes-for-current-kernels-and-gcc.patch::https://github.com/Xilinx/XRT/pull/6908.patch                                                                                                                                           
        dma-ip-drivers-fixes-for-current-kernels.patch::https://github.com/Xilinx/dma_ip_drivers/pull/176.patch                                                                                                                              
)
...
prepare() {                                                
        echo Patch $_name                                  
        git -C $_name apply $srcdir/xrt-fixes-for-current-kernels-and-gcc.patch

        # Submodule integration based on
        # https://wiki.archlinux.org/title/VCS_package_guidelines#Git_submodules

        # Patch dma_ip_drivers for current kernels
        git -C $_name config \
                submodule.src/runtime_src/core/pcie/driver/linux/xocl/lib/libqdma.url \
                ../dma_ip_drivers
        git -C $_name submodule update
        echo Patch $_name/src/runtime_src/core/pcie/driver/linux/xocl/lib/libqdma
        git -C $_name/src/runtime_src/core/pcie/driver/linux/xocl/lib/libqdma \
                apply $srcdir/dma-ip-drivers-fixes-for-current-kernels.patch
}
...

Whole PKGBUILD is on AUR: https://aur.archlinux.org/cgit/aur.git/ … ?h=xrt-git

Does someone have an idea?

Last edited by leuko (2022-08-24 10:02:12)

loqs · 2022-08-23 15:17:29

What if you use makepkg -C to clean $srcdir at the start of the second build?

leuko · 2022-08-23 15:44:05

If I remove $srcdir, then there is no problem.

Until now I did not have to remove the $srcdir when I make a PKGBUILD that applies patches in `prepare()`. The patches to the submodule seem to be problematic. Is this a bug or feature?

Trilby · 2022-08-23 15:52:05

I'm not very well versed in git, but I suspect the submodule update doesn't change the existing submodule code if there aren't new commits in that submodule - so you still have the "old" code that has already been patched, right? Perhaps using --force for submodule update would refresh the code regardless of a lack of commits resulting in the unpatched code being able to accept the patch.

That said, is this really a problem? The patch isn't applied because it already has been applied. So you should be set.

leuko · 2022-08-24 10:01:13

Trilby wrote:

That said, is this really a problem? The patch isn't applied because it already has been applied. So you should be set.

`git apply` errors out if the patch is already applied, so `makepkg` exits too in this case

Trilby wrote:

I'm not very well versed in git, ...

but you had the right hunch , providing `--force` to submodule update solves the problem.

My hunch is that `makepkg` force updates the superproject, but does not have any idea about submodules. I have to force update submodules myself.

Last edited by leuko (2022-08-24 10:01:28)

Trilby · 2022-08-24 12:21:01

I don't think makepkg does the equivalent of --force. But rather if there have been no new commits (to the main project repos, aka repos listed in source=) then nothing needs to be done at all so you never see this error. I'd bet if there were new commits in both the main project and the submodule, you'd also not see this error as git would update the submodule code (which reverts back to the unpatched state).

But if there have been commits in the main project but not in the submodule, then makepkg gets the new code for the main project (which is then not yet patched) but the 'git submodule update' detects no new commits and leaves the already-patched submodule code untouched.

leuko · 2022-08-24 13:26:13

Your ideas make sense, so I continued experimenting. Indeed the superproject is very active during the day but the submodule did not have any commits at all in the last two months. To rule your hunch out I restarted makepkg multiple times, but `submodule update --force` approach works flawlessly.

I think the reason is that `makepkg` issues a `git reset --hard` on the source branches everytime and names the branch `makepkg`:

...
==> Extracting sources...                                                                                                                                                                                                                    
  -> Creating working copy of xrt git repo...                                                                                                                                                                                                
Reset branch 'makepkg'                                                                                                                                                                                                                       
  -> Creating working copy of dma_ip_drivers git repo...                                                                                                                                                                                     
Reset branch 'makepkg'
...

I also tried the git process manually. `git reset` leaves the modifications, but `--hard` also get rid of them. The crux is that submodule is left untouched by the superproject's `git reset --hard`.

`makepkg` also resets the submodule `dma_ip_drivers`, but I attach it to the superproject using a submodule update, so the state of the submodule stays the same without `submodule update --force`.

Another solution could be to use a symbolic link to the submodule branch created by `makepkg`.

Additional edit: `git reset --hard` does not touch files not tracked by git. For example build folders created during make.

Last edited by leuko (2022-08-24 13:37:12)

Lone_Wolf · 2022-08-24 13:44:11

Some comments on the PKGBUILD

# Submodule integration based on
# https://wiki.archlinux.org/title/VCS_package_guidelines#Git_submodules

The parts of that guideline your prepare() function uses are so small I wonder if it's relevant to mention it.

Most archlinux packages use patch to apply patches, why do you use git apply ?
https://wiki.archlinux.org/title/Patching_packages

Last edited by Lone_Wolf (2022-08-24 13:44:32)

schard · 2022-08-24 13:59:23

man makepkg wrote:

--noprepare
Do not run the prepare() function in the PKGBUILD.

leuko · 2022-08-24 14:30:20

Lone_Wolf wrote:

The parts of that guideline your prepare() function uses are so small I wonder if it's relevant to mention it.

I thought that the link could be informational for other PKGBUILDers. I did not have any idea how to integrate submodules in a PKGBUILD and had to search for a long time so another link to the source could make this information more accessible in the search engines. IMHO the contribution of the guideline in essence is how to integrate submodules, which is what I did in my `prepare()`.

Lone_Wolf wrote:

Most archlinux packages use patch to apply patches, why do you use git apply ?
https://wiki.archlinux.org/title/Patching_packages

The patches that I download (from Github) are generated probably by git and could be applied by patch too. I sometimes have problems with getting the paths matched up with `patch`, so I thought `git apply` is less error-prone than `patch`.

While researching I found out `git am` which additionally commits the changes compared to `git apply` (which are mere modifications). `git am` solves the submodule problem better:

prepare() {
    echo Patch $_name
    git -C $_name am $srcdir/xrt-fixes-for-current-kernels-and-gcc.patch

    echo Patch $_name/src/runtime_src/core/pcie/driver/linux/xocl/lib/libqdma
    git -C dma_ip_drivers \
        am $srcdir/dma-ip-drivers-fixes-for-current-kernels.patch

    # Patch dma_ip_drivers for current kernels
    git -C $_name config \
        submodule.src/runtime_src/core/pcie/driver/linux/xocl/lib/libqdma.url \
        ../dma_ip_drivers
    git -C $_name submodule update
}

`git am` commits the changes to the sourced submodule and `git submodule update` gets the committed patches.

Additional words on `git am`: The patches that I download are actual commits proposed to a repository, so I find `git am` better than a mere `patch`, because the commits carry additional information like the commit message which are shown during `makepkg` process.

I find this solution awesome! Thanks for your ideas dear Lone_Wolf

Last edited by leuko (2022-08-24 14:45:00)

leuko · 2022-08-24 14:35:30

schard wrote:

man makepkg wrote:
--noprepare
Do not run the prepare() function in the PKGBUILD.

An average novice AUR user is not going to analyze the PKGBUILD IMHO. If the default configuration of a command solves the problem (just repeating `makepkg`), I would go for that.

~~But your solution also makes sense, thanks~~
Edit: Unfortunately --noprepare does not help after all, see next posts.

Last edited by leuko (2022-08-26 05:33:39)

Trilby · 2022-08-24 15:49:46

Wouldn't --noprepare result in a failure anytime there were new commits in the main project repo (which would be the only times makepkg would have to run anways)? In such cases the patch(es) would not be applied to that main project code.

In any case, you could also just ignore errors from the patch attempts on the submodule code so makepkg continues regardless (e.g., append `||:` to those lines).

Last edited by Trilby (2022-08-24 15:51:46)

leuko · 2022-08-26 05:30:49

Trilby wrote:

Wouldn't --noprepare result in a failure anytime there were new commits in the main project repo (which would be the only times makepkg would have to run anways)? In such cases the patch(es) would not be applied to that main project code.

You are right, I did not see that! When I run `makepkg --noprepare`, then the working branches are reset by makepkg, so the patches are lost. In short, without prepare() the patches won't be applied.

Trilby wrote:

In any case, you could also just ignore errors from the patch attempts on the submodule code so makepkg continues regardless (e.g., append `||:` to those lines).

Personally I do not like to override errors unless I am sure that no further errors happen. For example if new commits break the patches, then I would also override them. Still, this is a pretty quick and practical solution.

Arch Linux

#1 2022-08-23 15:14:40

[SOLVED] Second execution of `makepkg` causes `patch does not apply`

#2 2022-08-23 15:17:29

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#3 2022-08-23 15:44:05

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#4 2022-08-23 15:52:05

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#5 2022-08-24 10:01:13

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#6 2022-08-24 12:21:01

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#7 2022-08-24 13:26:13

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#8 2022-08-24 13:44:11

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#9 2022-08-24 13:59:23

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#10 2022-08-24 14:30:20

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#11 2022-08-24 14:35:30

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#12 2022-08-24 15:49:46

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

#13 2022-08-26 05:30:49

Re: [SOLVED] Second execution of `makepkg` causes `patch does not apply`

Board footer