You are not logged in.

#1 2018-08-21 15:32:04

gen2arch
Member
Registered: 2013-05-16
Posts: 182

SOLVED: ABS custom kernel build changed completely

Hi,

I normally build and install a custom kernel via ABS and the localmodconfig/modprobe-db mechanism.

With kernel version 4.18.3 the PKGBUILD file changed considerably, especially the "prepare" section (where I normally added two lines "sudo /usr/bin/modprobed-db recall" and
"make localmodconfig").

Also: kernel sources are no longer downloaded from kernel.org but a complete git checkout (2.2 GB) is performed after updpkgsums.

The built kernel has a "dirty" string attached to it.

Is there any documentation of how to build a custom kernel with this new framework?

How can I get rid of the annoying "dirty" tag to my kernel name?

Thanks!

Last edited by gen2arch (2018-08-25 09:40:19)

Offline

#2 2018-08-21 15:57:05

loqs
Member
Registered: 2014-03-06
Posts: 17,192

Re: SOLVED: ABS custom kernel build changed completely

https://bbs.archlinux.org/viewtopic.php?id=236702 covers the dirty tag.  You have not posted your modified PKGBUILD.

Offline

#3 2018-08-21 17:35:52

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,424
Website

Re: SOLVED: ABS custom kernel build changed completely

Moving to Creating and Modifying Packages...


Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

#4 2018-08-21 20:35:01

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: SOLVED: ABS custom kernel build changed completely

The dirty tag should only be an issue if you're running make kernelrelease during package() which is run in fakeroot.

But the current core/linux PKGBUILD uses --save-scmversion as the first thing it does, before patching or .config modification or anything, which short-circuits the kernel build system detection logic as it relies on the saved scmversion.

You should be modifying the .config *after:

  cp ../config .config
  make olddefconfig

...

This discussion is unlikely to get anywhere quickly until we know exactly what changes you do make, which means posting your PKGBUILD.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#5 2018-08-21 20:56:25

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

Thanks everyone.

I do

asp update; asp export linux

The PKGBUILD I get thus looks quite different compared to say a couple of weeks ago.

In this official PKGBUILD I change the kernel name

#pkgbase=linux               # Build stock -ARCH kernel
pkgbase=linux-me       # Build kernel with a different name

and add only three lines (-->) in the "prepare" section; it looks like this now:


prepare() {
  cd $_srcname


-->  patch -p1 < /home/me/git/kernel_gcc_patch/enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch

  msg2 "Setting version..."
  scripts/setlocalversion --save-scmversion
  echo "-$pkgrel" > localversion.10-pkgrel
  echo "$_kernelname" > localversion.20-pkgname

  local src
  for src in "${source[@]}"; do
    src="${src%%::*}"
    src="${src##*/}"
    [[ $src = *.patch ]] || continue
    msg2 "Applying patch $src..."
    patch -Np1 < "../$src"
  done

  msg2 "Setting config..."
  cp ../config .config

-->  sudo /usr/bin/modprobed-db recall
-->  make localmodconfig

  make olddefconfig

  make -s kernelrelease > ../version
  msg2 "Prepared %s version %s" "$pkgbase" "$(<../version)"
}

The patch I apply is from https://github.com/graysky2/kernel_gcc_patch.

Offline

#6 2018-08-21 21:34:22

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: SOLVED: ABS custom kernel build changed completely

There's your problem then.

-->  patch -p1 < /home/me/git/kernel_gcc_patch/enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch

You should instead add this to source=() and let the for loop apply this patch.

That would ensure it runs *after*

  msg2 "Setting version..."
  scripts/setlocalversion --save-scmversion

You patched the sources, dirtying them, and only afterwards did you set the version (which came with the dirty tag you just caused).


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#7 2018-08-22 05:33:01

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

Eschwartz wrote:

There's your problem then.

-->  patch -p1 < /home/me/git/kernel_gcc_patch/enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch

You should instead add this to source=() and let the for loop apply this patch.

Can I simply put the path to the patch into the source section like so?

source=(
  "$_srcname::git+https://git.archlinux.org/linux.git?signed#tag=v$_srcver"
  config         # the main kernel config file
  60-linux.hook  # pacman hook for depmod
  90-linux.hook  # pacman hook for initramfs regeneration
  linux.preset   # standard config files for mkinitcpio ramdisk
  /home/me/git/kernel_gcc_patch/enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch
)

And: is there any discussion/documentation as to why the kernel ebuild changed so considerably, why is the mechanism kernel.org sources + arch patches abandoned?

Thanks

Offline

#8 2018-08-22 11:56:55

loqs
Member
Registered: 2014-03-06
Posts: 17,192

Re: SOLVED: ABS custom kernel build changed completely

See https://www.kernel.org/category/site-news.html
Edit:

/home/me/git/kernel_gcc_patch/enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch

Why the full path rather than placing the file in the same directory?

Last edited by loqs (2018-08-22 12:05:08)

Offline

#9 2018-08-22 12:19:44

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: SOLVED: ABS custom kernel build changed completely

Also: kernel sources are no longer downloaded from kernel.org but a complete git checkout (2.2 GB) is performed after updpkgsums.

By the way, that amount of data is about the same as downloading the kernel (~100MB) 23 times, so counting full downloads for mainline releases (4.X.0) and patches for stable releases (4.0.Y), that is less than 5 years of kernel building. Afterwards you are building the kernel basically for free if you keep the git repository around.
Edit: And if you ever have to bisect the kernel, you'd have to clone the git repository anyways.

Clarification: Since the patches are now useless for cryptographically signed sources, the 2.2GB mark is reached in no time at all. See following reply by Eschwartz.

Last edited by progandy (2018-08-22 16:57:00)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#10 2018-08-22 16:03:31

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: SOLVED: ABS custom kernel build changed completely

progandy wrote:

Also: kernel sources are no longer downloaded from kernel.org but a complete git checkout (2.2 GB) is performed after updpkgsums.

By the way, that amount of data is about the same as downloading the kernel (~100MB) 23 times, so counting full downloads for mainline releases (4.X.0) and patches for stable releases (4.0.Y), that is less than 5 years of kernel building. Afterwards you are building the kernel basically for free if you keep the git repository around.
Edit: And if you ever have to bisect the kernel, you'd have to clone the git repository anyways.

Did you read loqs' link? https://www.kernel.org/minor-changes-to … changelogs

The sole reason for moving to git in the first place, is because kernel.org will no longer provide PGP signatures for patches; it is either use git, or download unique 100MB files for every single patch release.

There have been exactly 23 core/linux version updates spanning both 4.x.0 mainline releases and 4.x.y stable releases, since 4.16.12-1 which was in May of this year.

Five years of kernel building would be humongously larger than that, assuming you start counting from now rather than including historical releases that had incremental stable patches.

Last edited by eschwartz (2018-08-22 16:05:44)


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#11 2018-08-22 16:54:08

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: SOLVED: ABS custom kernel build changed completely

Eschwartz wrote:

Yes, I just compared it to the old numbers, because even 5 years are not much if you keep using linux.


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#12 2018-08-23 07:50:08

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

loqs wrote:

See https://www.kernel.org/category/site-news.html
Edit:

/home/me/git/kernel_gcc_patch/enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch

Why the full path rather than placing the file in the same directory?

Thanks, I did that, worked well!

Offline

#13 2018-08-23 07:54:45

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

progandy wrote:

Also: kernel sources are no longer downloaded from kernel.org but a complete git checkout (2.2 GB) is performed after updpkgsums.

By the way, that amount of data is about the same as downloading the kernel (~100MB) 23 times, so counting full downloads for mainline releases (4.X.0) and patches for stable releases (4.0.Y), that is less than 5 years of kernel building. Afterwards you are building the kernel basically for free if you keep the git repository around.

I usually deleted the entire "linux" build directory after building the kernel and basically started anew with every kernel release (asp update, asp export linux and so on).

How do I have to upgrade if I keep the "linux" build directory with the complete git checkout inside? Is the upgrade then done via "asp update" or via "git pull"?

Thanks.

Offline

#14 2018-08-23 07:56:29

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

I see, thanks for the link!

Offline

#15 2018-08-24 16:48:29

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

What is the interplay between asp and git with regard to the full git checkout the official linux kernel ebuild is performing now? What is the proper way to upgrade to a newer kernel?

Thanks

Offline

#16 2018-08-24 16:52:48

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: SOLVED: ABS custom kernel build changed completely

There is no interplay at all.

The "git" program does not care if you clone one repository (the linux.git sources downloaded by makepkg) inside of another one (the PKGBUILD repository downloaded by asp). If that's not what you were referring to then I don't have any idea what you mean.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#17 2018-08-24 18:39:23

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

Eschwartz wrote:

There is no interplay at all.

The "git" program does not care if you clone one repository (the linux.git sources downloaded by makepkg) inside of another one (the PKGBUILD repository downloaded by asp). If that's not what you were referring to then I don't have any idea what you mean.

Sorry for being unclear, I meant: "asp update / asp export linux" does some sort of update of the kernel ebuild; but on the other hand, as a git checkout is now done, one would think that upgrading involves some sort of "git pull" command; so in my view there are two concurrent update mechanisms involved,  and I was wondering about their order.

Here's how the "linux" directory looks like:

tree -L 2 linux
linux
├── 60-linux.hook
├── 90-linux.hook
├── PKGBUILD
├── archlinux-linux
│   ├── FETCH_HEAD
│   ├── HEAD
│   ├── branches
│   ├── config
│   ├── description
│   ├── hooks
│   ├── info
│   ├── objects
│   ├── packed-refs
│   └── refs
├── config
├── enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch
├── linux.install
├── linux.install.pkg
├── linux.preset
├── pkg
│   ├── linux-me
│   ├── linux-me-docs
│   └── linux-me-headers
└── src
    ├── 60-linux.hook -> /home/me/ABS/linux/60-linux.hook
    ├── 90-linux.hook -> /home/me/ABS/linux/90-linux.hook
    ├── archlinux-linux
    ├── config -> /home/me/ABS/linux/config
    ├── enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch -> /home/me/ABS/linux/enable_additional_cpu_optimizations_for_gcc_v8.1+_kernel_v4.13+.patch
    ├── linux.preset -> /home/me/ABS/linux/linux.preset
    └── version

I cannot simply redo "asp export linux" to upgrade the ebuild, because the directory "linux" exists already. But if I remove it, a complete git clone has to be redone. This is what I meant by "interplay": how / by what order of steps can I keep the existing git clone (so a git pull suffices and not an other entire new cloning of the sources), while nevertheless upgrading the ebuild via asp?

Offline

#18 2018-08-24 18:45:13

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: SOLVED: ABS custom kernel build changed completely

If you use a git repository (asp checkout) instead of an export it works without problems. With asp export, it might work if you use the -f switch.
Edit: Or configure makepkg to store all sources in a central directory.

Last edited by progandy (2018-08-24 18:46:11)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#19 2018-08-25 09:39:41

gen2arch
Member
Registered: 2013-05-16
Posts: 182

Re: SOLVED: ABS custom kernel build changed completely

progandy wrote:

If you use a git repository (asp checkout) instead of an export it works without problems. With asp export, it might work if you use the -f switch.
Edit: Or configure makepkg to store all sources in a central directory.

Didn't continue "asp checkout" (because didn't understand what to do with the resulting directories), "asp -f export linux" gave a subsequent error with makepkg, but the last option with a central source directory worked great, I think its also the most straightforward one: I defined the variable SRCDEST in /etc/makepkg.conf and that separation of build directory and source directory solved it for me!

Thanks again progandy.

Offline

Board footer

Powered by FluxBB