You are not logged in.

#1 2013-06-22 22:56:39

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Advice on compiling kernel from git

I am a reluctant would-be kernel-compiler. However, needs must.

In order to troubleshoot the bug affecting booting with the EFI STUB loader in recent kernels, I need to compile a kernel from  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git using a .config and see whether I can boot it. I figure I'll worry about where to find a .config later. For now, I'm still trying to decide how to get started.

Looking at the wiki, I take it that I need to do this the "traditional" way rather than the easier way. However, I realise this is probably quite dumb but I'm confused by the very first step which involves obtaining the source (https://wiki.archlinux.org/index.php/Ke … ing_source). According to the wiki, the first step involves downloading a tar ball. I take it that I should instead check out the source from git. However, I would like to know if this changes subsequent steps. That is, can I just follow the steps in the wiki to compile code from git or do I need to adapt them in some way? Or follow a different set of instructions?

Also, I would like to confirm that I should be able to install the kernel I will hopefully manage to compile alongside Arch's. (I don't hold out much hope of my efforts proving successful so I'm counting on this safety net.)

EDIT: So git ≠ svn.

REF: https://bbs.archlinux.org/viewtopic.php?id=156670

Last edited by cfr (2013-07-03 02:45:50)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#2 2013-06-22 23:11:29

karol
Archivist
Registered: 2009-05-06
Posts: 25,440

Re: Advice on compiling kernel from git

Yes, you can have many kernels installed at the same time. Traditionally, you pick the one you want to use from the bootloader.
All the steps should be the same, no matter if you get the source from a tarball or from git.

Offline

#3 2013-06-22 23:13:40

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Advice on compiling kernel from git

Why not use linux-git from the AUR?

Offline

#4 2013-06-23 14:41:17

mich41
Member
Registered: 2012-06-22
Posts: 796

Re: Advice on compiling kernel from git

git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
cd linux
git checkout v3.2.1  # if you need some particular version, use "master" or skip this step altogether to get current development version
# any changes/patches you need
zcat /proc/config.gz >.config # get config from running kernel
make oldconfig   # it may ask you about some new option which aren't set in Arch config, press "h" for help or ENTER for default value
make menuconfig  # change "local version" as advised by wiki and whatever else you want to change
make -j number-of-cpus

Then use "make modules_install" and "cp blahblah/bzImage" to install and generally follow the wiki.

As long as you change "local version" to something else than "ARCH" and don't overwrite any files in /boot you will be able to boot stock kernel.

Offline

#5 2013-06-23 22:13:44

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

Thanks. I didn't know about linux-git. However, it seems it will build an older version which may not be what they want. In any case, I figure I will need to be able to build without that to do a git bisect? (Though I have only a very general idea of what that is right now.)

I'm trying out the makelocalmodconfig, basically hitting enter for all the additional questions to get the defaults, I hope.

Last edited by cfr (2013-06-24 01:41:34)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#6 2013-06-24 01:02:14

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Advice on compiling kernel from git

The git packages simply clone from whatever is in the current tree, so a git package is not technically out of date until it actually no longer builds.  Try it an see.  I just built it a minute ago just fine, and ended up with linux 3.10rc7.  I had been thinking about using the git package for a while, but unfortunately I was not able to get the broadcom-wl package to compile, and unfortunately there is not other module that this particular wifi card will work with.

Edit: If you are using localmodconfig, just make sure that you have everything loaded that you want built into the kernel.  Also, if you use any 3rd party modules, these are obviously not going to be found in the kernel config.  For example, I use the r8168 package because the r8169 module has never worked quite right.  So if I were using localmodconfig, it would probably be easiest for me to simply unload r8168 and then load r8169 temporarily, since I rarely use ethernet.  I have noticed that if I leave r8168 loaded, it throws an error, and then everything in the ethernet drivers is selected to be built.

Last edited by WonderWoofy (2013-06-24 01:05:31)

Offline

#7 2013-06-24 01:17:29

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

WonderWoofy wrote:

The git packages simply clone from whatever is in the current tree, so a git package is not technically out of date until it actually no longer builds.  Try it an see.  I just built it a minute ago just fine, and ended up with linux 3.10rc7.  I had been thinking about using the git package for a while, but unfortunately I was not able to get the broadcom-wl package to compile, and unfortunately there is not other module that this particular wifi card will work with.

I didn't realise this as the package specifies a version so I assumed it would work to build that version. A bit like ttf-google-webfonts-git builds a particular version, I guess.

Edit: If you are using localmodconfig, just make sure that you have everything loaded that you want built into the kernel.  Also, if you use any 3rd party modules, these are obviously not going to be found in the kernel config.  For example, I use the r8168 package because the r8169 module has never worked quite right.  So if I were using localmodconfig, it would probably be easiest for me to simply unload r8168 and then load r8169 temporarily, since I rarely use ethernet.  I have noticed that if I leave r8168 loaded, it throws an error, and then everything in the ethernet drivers is selected to be built.

Mmm. Yes. I don't actually plan to use this kernel which simplifies things somewhat. In fact I deliberately didn't connect extra things on the grounds that it would compile faster without. If I were planning to use the kernel, it would be different. I suspect I could actually make it much more minimal if I knew what I was doing, actually.

At any rate, I've compiled something. I am now hesitating before installing it as I'm always worried that installing stuff will spray things everywhere. (I hardly ever install stuff as root unless it is a package or I am just stowing.) Still, I guess it should just install the modules. I'm not quite sure what

/bin/sh /home/software/src/kernel/linux/scripts/depmod.sh /sbin/depmod 3.10.0-rc7-CFR ""

does although the script looks innocuous enough.

Note that I'm not asking anybody to tell me anything at this point. I'm just talking myself into actually doing it.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#8 2013-06-24 01:28:41

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

So I decided to try linux-git using the .config I made earlier. However, linux-git depends on linux-firmware-git and linux-firmware-git conflicts with linux-firmware. Moreover, linux-firmware is required by linux and linux-lts. linux-firmware-git provides linux-firmware so that's not a problem but I'm wondering how safe this is and also why I didn't have to uninstall linux-firmware to build the kernel from git without the package linux-git.

Note that I am hoping somebody may respond to this.


EDIT: I guess that make modules_install is going to force the install of updated firmware anyway but I don't understand how it is then possible to retain the stock kernel. Can that function with the firmware from git?

EDIT EDIT: I guess the answer is that the firmware is virtually identical except there is a little bit missing from linux-firmware-git. Hopefully not a critical bit.

Last edited by cfr (2013-06-24 02:06:23)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#9 2013-06-24 01:45:03

Max-P
Member
Registered: 2011-06-11
Posts: 164

Re: Advice on compiling kernel from git

Aren't linux-firmwares just more or less binary blobs used to make some chips run? ucode images for wireless cards and stuff like that... The package is even a "Any" architecture in the repos. It put loads of .bin files in /usr/lib/firmware/.

Best way to know is to try not updating it first and see if it works? The package was last updated 2 weeks ago and tracks a git repo that was last updated a month ago. I think it's safe to just ignore the linux-firmware-git package and remove it from the PKGBUILD of linux-git for your needs. (I don't even think this is a required part of the kernel building as they are just support files for chips that needs proprietaty blobs (wireless cards, GPUs, some USB devices and some sound cards too). Neither linux-ck and linux-pf seems to require it either).

Offline

#10 2013-06-24 02:08:19

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

I think you are probably right but actually I've just removed linux-firmware and replaced it. I checked. There are almost no differences except one directory is missing from the git version. I read the readme and I don't need the stuff it supports. What I will probably do is remove it with -Rd when I'm done and install linux-firmware. Just to be safe.

Last edited by cfr (2013-06-24 02:10:33)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#11 2013-06-24 04:21:19

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Advice on compiling kernel from git

I too installed linux-firmware-git and for the very limited number of devices that had firmware in that package, things still worked just fine.  I would like to note though that when I tried to use the early rc's of 3.9, if I didn't use the linux-firmware-git package my wireless card would not work at all.  That was with the Intel card I no longer use though, so my firmware is not provided by the broadcom-wl (sta) package... the one that doesn't compile against linux 3.10 sad 

Instead of -Rd'ing linux-firmware-git and installing linux-firmware, simply change the depends in either the PKGBUILD (if it is before you compile) or in /var/lib/pacman/local/linux-git-3.10*/desc (if you have alrady compiled).

Offline

#12 2013-06-27 22:18:51

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

Thanks very much for the help.

I ended up using an edited linux-git PKGBUILD (changed the depends for linux-firmware-git to linux-firmware) and a modified version of Arch's .config. (Basically just taking out stuff I thought I didn't need.) Compiled fine but I somehow managed to leave out an essential ingredient. However, I finally got it to work and aside from the fact that I arbitrarily rendered my machine temporarily unbootable due to unrelated idiocy, it all seems to work fine.

That is, I can reproduce the EFI stub bug using this kernel: it will not boot that way but boots fine with grub. So now we wait and see wink.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#13 2013-07-02 22:22:19

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

Update: so now I need to recompile using a patch.

The patch bit is easy: I just put it in a sub-directory named patches. I assume, though, that it would be best to use the same code I used before to rule out the influence of other changes in the meantime.

I am wondering how best to do this. If I have understood correctly, changing

# _gitrev=""

to the git revision should use the same source code. I am not sure what git revisions look like. The package version is 3.10.rc7.8.gacdb37c. So is the git revision number just the final part of that?

_gitrev="7.8.gacdb37c"

None of it looks like an svn revision number. I'm a bit out of my depth with git: svn is about the limit of my experience with version control systems because it integrates nicely with LaTeX for versioning documents. I did try reading the man page for gitrevisions as suggested in the PKGBUILD comment but I must have missed something as I do not feel much the wiser.

Whatever line that should be, will this redownload the repository or will it reuse the existing copy? If the former, is there any way to reuse the existing copy (just seems a waste)? If the latter, should I do anything special to clean up from the last build before compiling?


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#14 2013-07-02 23:08:36

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: Advice on compiling kernel from git

You need a git revision. Read "man gitrevisions" for possible formats. The pkgver is a modified git describe string and therefore a modified revision. If you want to use it, it should be this I think (undo the sed in pkgver):
_gitrev="v3.10.rc7-8-gacdb37c"

Last edited by progandy (2013-07-02 23:39:56)


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#15 2013-07-02 23:18:45

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

progandy wrote:

You need a git revision. Read "man gitrevisions" for possible formats. The pkgver is a modified git describe string and therefore a modified revision. If you want to use it, it should be this I think:
_gitrev="v3.10.rc7-8-gacdb37c"

Thanks. So the name of a "commit object" or "revision parameter" just is a revision number? I thought that a git revision must be something else for some reason which is why the man page didn't help much. Thanks for clarifying that.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#16 2013-07-03 00:09:18

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

I seem to have done something wrong as I ended up with a different kernel version as far as I can tell: package is 3.10.2904.g42daabf as opposed to 3.10.rc7.8.gacdb37c. However, it did seem to patch it so maybe it will work for testing purposes... (Maybe it doesn't really matter to use the same source as before anyway.)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#17 2013-07-03 00:10:32

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Advice on compiling kernel from git

With git sources, it will update the version number to reflect the version as reported by the git repository rather than what is in the pkgbuild.  This is totally normal.

Offline

#18 2013-07-03 00:12:54

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

Why? I mean what does setting _gitrev do and why is that not reflected in the package name?

Incidentally, I tried to find information about using git pkgbuilds on the wiki when I started doing this but I couldn't really find anything. I did find some things aimed at creating them but not very much and nothing really about differences with using them (as opposed to regular pkgbuilds).

Last edited by cfr (2013-07-03 00:14:47)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#19 2013-07-03 00:48:05

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Advice on compiling kernel from git

My bad cfr, I scrolled up and I apparently missed a big part of the conversation.  I had just read that last post of yours and took it totally out of context.  Using _gitrev is something I have not had to do before.

Last edited by WonderWoofy (2013-07-03 00:48:21)

Offline

#20 2013-07-03 01:04:12

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: Advice on compiling kernel from git

pkgver is executed before the source is set to the specified revision. I suppose you better keep _gitrev empty and add #commit=COMMITID to the git url in the source array, i.e.:

source=("git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git#commit=acdb37c"
        config.{i686,x86_64})

This works if the shortened commit id (taken from the pkgver) is uniqe in the repo. If not, use the full commit id:
    #commit=acdb37c361dc87e165889a504e291c1e82ae133c


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#21 2013-07-03 01:14:55

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

Thanks! I'll try the kernel I've compiled and if it works, I'll redo it with the commit set so that I can determine if the patch is responsible. (If it doesn't work, I guess there won't be much point unless I'm asked to.)


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#22 2013-07-03 01:17:47

s3kt0r
Member
Registered: 2009-01-20
Posts: 208

Re: Advice on compiling kernel from git

progandy wrote:

pkgver is executed before the source is set to the specified revision. I suppose you better keep _gitrev empty and add #commit=COMMITID to the git url in the source array, i.e.:

source=("git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git#commit=acdb37c"
        config.{i686,x86_64})

This works if the shortened commit id (taken from the pkgver) is uniqe in the repo. If not, use the full commit id:
    #commit=acdb37c361dc87e165889a504e291c1e82ae133c

I was fighting with that, thank you

However, even if you set _gitrev to _gitrev="v3.10" or the particular commit (8bb495e3f02401ee6f76d1b1d77f3ac9f079e376), it will still build v3.10, although version will then be overwriten by pkgver.


box1: Arch (linux-3.17-rc5)
box2: Gentoo (linux-3.17-rc5)
wm: subtle

Offline

#23 2013-07-03 02:44:52

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

Yes. I just found that although the package version number jumped, the kernel config still lists 3.10.0-rc7 so I think I built the right kernel but just the package is mislabelled. For my purposes, the label is neither here nor there.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#24 2013-07-12 22:03:34

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 7,130

Re: Advice on compiling kernel from git

Is anybody having issues booting the current code from git? I cannot boot a kernel built with code from yesterday or today with either the stub loader or grub.


CLI Paste | How To Ask Questions

Arch Linux | x86_64 | GPT | EFI boot | refind | stub loader | systemd | LVM2 on LUKS
Lenovo x270 | Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz | Intel Wireless 8265/8275 | US keyboard w/ Euro | 512G NVMe INTEL SSDPEKKF512G7L

Offline

#25 2013-07-12 22:09:20

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,414

Re: Advice on compiling kernel from git

Let me give it a whirl real quick and I'll let you know.

Offline

Board footer

Powered by FluxBB