[SOLVED] Building Android kernel images in Arch

mradermaxlol · 2016-01-10 11:45:33

Hello there!

So, I need to build some Android kernel. All the guides on the net provide instructions for Debian-based distros, although the process is likely the same. However, there is only one issue - packages required to build the thing are named *slightly* different in Arch, or are only on AUR. I think there would be two questions:

1) What are Arch's alternatives to:
git-core gnupg flex bison gperf libsdl1.2-dev libesd0-dev libwxgtk2.8-dev squashfs-tools build-essential zip curl libncurses5-dev zlib1g-dev pngcrush schedtool g++-multilib lib32z1-dev lib32ncurses5-dev lib32readline-gplv2-dev libxml2-utils u-boot-tools libc6-dev x11proto-core-dev libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 libgl1-mesa-dev mingw32 tofrodos python-markdown xsltproc
from Debian?

2) Is there a better way to compile the kernel? If you like, the source is here.
I'm trying to build with binary Linaro based on top of GCC 5.2, from their Latest branch, using ARM-EABI version.

Thanks in advance.

Last edited by mradermaxlol (2022-03-02 09:48:03)

graysky · 2016-01-10 11:56:06

mradermaxlol wrote:

1) What are Arch's alternatives to

Dunno... you can try using pkgfile to search for them one-by-one but that sounds painful.

mradermaxlol wrote:

2) Is there a better way to compile the kernel?

Did you see https://wiki.archlinux.org/index.php/An … ng_android?

Last edited by graysky (2016-01-10 11:59:12)

Mr.Elendig · 2016-01-10 12:19:52

Generally you can just strip out -dev and the package name will be very similar. Just a tiny bit of pattern matching and common sense will be enough.

Arch package names are generally the same as the base name of the project: so libncurses-dev -> ncurses, git-core -> git and so on.

Last edited by Mr.Elendig (2016-01-10 12:21:54)

mradermaxlol · 2016-01-10 15:06:34

Did you see https://wiki.archlinux.org/index.php/An … ng_android?

Thanks, looks like I needed that.

mradermaxlol · 2016-01-10 15:07:05

Mr.Elendig wrote:

Generally you can just strip out -dev and the package name will be very similar. Just a tiny bit of pattern matching and common sense will be enough.
Arch package names are generally the same as the base name of the project: so libncurses-dev -> ncurses, git-core -> git and so on.

Heh, I did it that way; however, some packages were still missing.

mradermaxlol · 2016-01-10 15:15:44

graysky, I get makepkg error when trying to compile lib32-ncurses5-compat-libs.
Here's some text: http://pastebin.com/TrTEvTuf

anatolik · 2016-01-10 15:37:36

arm-none-eabi is toolchain is already in official repo and you can use that to compile the kernel https://www.archlinux.org/packages/comm … -eabi-gcc/

Honestly I am not sure why you need that amount of lib32 packages. You just need a set of standard tools like mkbootimg/dtc. For example here is how I compile kernel for Pixel C (it is arm64 architecture):

make -C kernel ARCH=arm64 dragon_defconfig

make -C kernel ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j5
lz4 -20 -z -f kernel/arch/arm64/boot/Image Image.lz4
./generate-its-script.sh -a arm64 Image.lz4 kernel/arch/arm64/boot/dts/tegra/*.dtb | dtc -I dts -O dtb -p 1024 > Image.fit

# Use ramdisk from Android build tree out/target/product/dragon/ramdisk.img
mkbootimg --kernel Image.fit --ramdisk ramdisk.img --output boot.img.unsigned


cleanup() {
        rm -f ${EMPTY}
}
EMPTY=$(mktemp /tmp/tmp.XXXXXXXX)
trap cleanup EXIT
echo " " > ${EMPTY}

futility vbutil_keyblock --pack boot.img.keyblock --datapubkey kernel_data_key.vbpubk --signprivate kernel_data_key.vbprivk
futility vbutil_kernel --pack boot.img --keyblock boot.img.keyblock --signprivate kernel_data_key.vbprivk --version 1 --vmlinuz boot.img.unsigned --config ${EMPTY} --arch arm --bootloader ${EMPTY} --flags 0x1

fastboot flash boot boot.img
fastboot continue

That's all build instructions. In your case you don't need kernel image signing (futility) and thus instructions are even simpler.

mradermaxlol · 2016-01-10 15:47:11

anatolik wrote:

arm-none-eabi is toolchain is already in official repo and you can use that to compile the kernel https://www.archlinux.org/packages/comm … -eabi-gcc/
Honestly I am not sure why you need that amount of lib32 packages. You just need a set of standard tools like mkbootimg/dtc. For example here is how I compile kernel for Pixel C (it is arm64 architecture):
make -C kernel ARCH=arm64 dragon_defconfig

make -C kernel ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j5
lz4 -20 -z -f kernel/arch/arm64/boot/Image Image.lz4
./generate-its-script.sh -a arm64 Image.lz4 kernel/arch/arm64/boot/dts/tegra/*.dtb | dtc -I dts -O dtb -p 1024 > Image.fit

# Use ramdisk from Android build tree out/target/product/dragon/ramdisk.img
mkbootimg --kernel Image.fit --ramdisk ramdisk.img --output boot.img.unsigned


cleanup() {
        rm -f ${EMPTY}
}
EMPTY=$(mktemp /tmp/tmp.XXXXXXXX)
trap cleanup EXIT
echo " " > ${EMPTY}

futility vbutil_keyblock --pack boot.img.keyblock --datapubkey kernel_data_key.vbpubk --signprivate kernel_data_key.vbprivk
futility vbutil_kernel --pack boot.img --keyblock boot.img.keyblock --signprivate kernel_data_key.vbprivk --version 1 --vmlinuz boot.img.unsigned --config ${EMPTY} --arch arm --bootloader ${EMPTY} --flags 0x1

fastboot flash boot boot.img
fastboot continue
That's all build instructions. In your case you don't need kernel image signing (futility) and thus instructions are even simpler.

Okay, will try that toolchain. That's what I get with Linaro 5.2: http://pastebin.com/Gqny8q7Y

mradermaxlol · 2016-01-10 15:54:35

mradermaxlol wrote:

anatolik wrote:
arm-none-eabi is toolchain is already in official repo and you can use that to compile the kernel https://www.archlinux.org/packages/comm … -eabi-gcc/
Honestly I am not sure why you need that amount of lib32 packages. You just need a set of standard tools like mkbootimg/dtc. For example here is how I compile kernel for Pixel C (it is arm64 architecture):
make -C kernel ARCH=arm64 dragon_defconfig

make -C kernel ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j5
lz4 -20 -z -f kernel/arch/arm64/boot/Image Image.lz4
./generate-its-script.sh -a arm64 Image.lz4 kernel/arch/arm64/boot/dts/tegra/*.dtb | dtc -I dts -O dtb -p 1024 > Image.fit

# Use ramdisk from Android build tree out/target/product/dragon/ramdisk.img
mkbootimg --kernel Image.fit --ramdisk ramdisk.img --output boot.img.unsigned


cleanup() {
        rm -f ${EMPTY}
}
EMPTY=$(mktemp /tmp/tmp.XXXXXXXX)
trap cleanup EXIT
echo " " > ${EMPTY}

futility vbutil_keyblock --pack boot.img.keyblock --datapubkey kernel_data_key.vbpubk --signprivate kernel_data_key.vbprivk
futility vbutil_kernel --pack boot.img --keyblock boot.img.keyblock --signprivate kernel_data_key.vbprivk --version 1 --vmlinuz boot.img.unsigned --config ${EMPTY} --arch arm --bootloader ${EMPTY} --flags 0x1

fastboot flash boot boot.img
fastboot continue
That's all build instructions. In your case you don't need kernel image signing (futility) and thus instructions are even simpler.
Okay, will try that toolchain. That's what I get with Linaro 5.2: http://pastebin.com/Gqny8q7Y

I think that's it:

/bin/sh: arm-linux-androideabi-gcc: команда не найдена
/home/maxik/hwbuild/kernel/scripts/Makefile.build:307: ошибка выполнения рецепта для цели «scripts/mod/empty.o»
make[5]: *** [scripts/mod/empty.o] Ошибка 127
/home/maxik/hwbuild/kernel/scripts/Makefile.build:443: ошибка выполнения рецепта для цели «scripts/mod»
make[4]: *** [scripts/mod] Ошибка 2
/home/maxik/hwbuild/kernel/Makefile:515: ошибка выполнения рецепта для цели «scripts»
make[3]: *** [scripts] Ошибка 2

anatolik · 2016-01-10 15:59:21

/bin/sh: arm-linux-androideabi-gcc: команда не найдена

Find in your build script where it hardcodes cross-compiler arm-linux-androideabi- and replace it with arm-none-eabi-

anatolik · 2016-01-10 20:22:40

Following patch over 'experimental' branch in the github repo you provided allows me to run

$ ./mk -o=TARGET_BUILD_VARIANT=user y600 n k

successfully and create a kernel image.

diff --git a/kernel/Makefile b/kernel/Makefile
index 07b3d37..e44867f 100755
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -195,7 +195,7 @@ export KBUILD_BUILDHOST := $(SUBARCH)
 #ARCH          ?= $(SUBARCH)
 #CROSS_COMPILE ?= $(CONFIG_CROSS_COMPILE:"%"=%)
 ARCH           ?= arm
-CROSS_COMPILE  ?= arm-linux-androideabi-
+CROSS_COMPILE  ?= arm-none-eabi-
 
 # Architecture as present in compile.h
 UTS_MACHINE    := $(ARCH)
diff --git a/mediatek/build/libs/config.mk b/mediatek/build/libs/config.mk
index daef587..86e8ff7 100755
--- a/mediatek/build/libs/config.mk
+++ b/mediatek/build/libs/config.mk
@@ -27,7 +27,7 @@ $(2): PRIVATE_FILE_LIST := $(3)
 $(2): $(3) $(4)
        @echo "[CONFIG] generate $(2)"
        @mkdir -p $(dir $(2))
-       @python $(4) $(3) > $(2)
+       @python2 $(4) $(3) > $(2)
 endef
 
 define .mtk.config.generate-auto-rules
@@ -58,7 +58,7 @@ else
        @echo "[CONFIG] generate $$@"
 endif
        $$(hide) if [ -e $$@ ]; then chmod u+w $$@; else mkdir -p $$(dir $$@); fi
-       $$(hide) python $$(MTK_ROOT_BUILD)/tools/config/merge-project.py $$(MTK_PROJECT_CONFIGS) > $$@
+       $$(hide) python2 $$(MTK_ROOT_BUILD)/tools/config/merge-project.py $$(MTK_PROJECT_CONFIGS) > $$@
 )
 endef

mradermaxlol · 2016-01-11 04:47:58

Yup, I've also succeeded in that; however, I used venv2 + exported CROSS_COMPILE as arm-linux-armeabi-. That's a dirty one, but it builds a kernel. However, I'm NOT able to use MTKtools to rebuild an image (ramdisk + kernel) - the thing does it wrong: Alex provided me with his ramdisks, and it didn't work neither with my compiled kernel nor his. Looks like I'll have to create a virtual machine :C Or is there some other way to do it?

Also, I'm thinking of adding "arm-*eabi*-" for compiler. This one should work, right? It will allow to use basically any toolchain.

anatolik · 2016-01-11 05:26:00

The repo you gave is one for Linux kernel. It does not contain projects needed to build a ramdisk. You need full Android repo checkout to build ramdisk. But if your goal is to modify the kernel only then prebuilt ramdisk (e.g. extracted from running device) should work fine.

it didn't work neither with my compiled kernel nor his

What exactly the problem is?

mradermaxlol · 2016-01-11 05:34:44

anatolik wrote:

The repo you gave is one for Linux kernel. It does not contain projects needed to build a ramdisk. You need full Android repo checkout to build ramdisk. But if your goal is to modify the kernel only then prebuilt ramdisk (e.g. extracted from running device) should work fine.
it didn't work neither with my compiled kernel nor his
What exactly the problem is?

The ramdisk is extracted from device's stock kernel, or Alex's kernel image (it is avalible on 4PDA). Usjng that initrd you are able to build an image. So, the thing is that on my machine that damn tool builds the image wrong, as Alex's builds from Ubuntu 14.04 VM work for everyone, and even with kernels from my machine. On Arch I haven't managed to create a bootable one yet.

mradermaxlol · 2016-01-11 06:23:53

anatolik wrote:

arm-none-eabi is toolchain is already in official repo and you can use that to compile the kernel https://www.archlinux.org/packages/comm … -eabi-gcc/
Honestly I am not sure why you need that amount of lib32 packages. You just need a set of standard tools like mkbootimg/dtc. For example here is how I compile kernel for Pixel C (it is arm64 architecture):
make -C kernel ARCH=arm64 dragon_defconfig

make -C kernel ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j5
lz4 -20 -z -f kernel/arch/arm64/boot/Image Image.lz4
./generate-its-script.sh -a arm64 Image.lz4 kernel/arch/arm64/boot/dts/tegra/*.dtb | dtc -I dts -O dtb -p 1024 > Image.fit

# Use ramdisk from Android build tree out/target/product/dragon/ramdisk.img
mkbootimg --kernel Image.fit --ramdisk ramdisk.img --output boot.img.unsigned


cleanup() {
        rm -f ${EMPTY}
}
EMPTY=$(mktemp /tmp/tmp.XXXXXXXX)
trap cleanup EXIT
echo " " > ${EMPTY}

futility vbutil_keyblock --pack boot.img.keyblock --datapubkey kernel_data_key.vbpubk --signprivate kernel_data_key.vbprivk
futility vbutil_kernel --pack boot.img --keyblock boot.img.keyblock --signprivate kernel_data_key.vbprivk --version 1 --vmlinuz boot.img.unsigned --config ${EMPTY} --arch arm --bootloader ${EMPTY} --flags 0x1

fastboot flash boot boot.img
fastboot continue
That's all build instructions. In your case you don't need kernel image signing (futility) and thus instructions are even simpler.

Hmm, so, there is a way to create a DIY-buildscript? Will try that ASAP. And, eh, how does the mkbootimg util work? May I just specify the extracted ramdisk dir and kernel binary?
Slrry if the question is silly, I'm just not at my PC right now, thus can't check how it acts.

anatolik · 2016-01-11 06:37:17

mkbootimg is a tool located in android-tools Arch package. It gets kernel blob and ramdisk blob and creates an Android boot partition. The tool is very simple and description can be found here https://android.googlesource.com/platfo … /bootimg.h

there is a way to create a DIY-buildscript?

Creating a boot image is as simple as compiling kernel and concatenating it with ramdisk. Plus some device-specific steps like signing.

I suggest you to understand what is that build script (from your repo) does and turn it into simple bash script like mine above. You'll be surprised how simple these build steps are. Plus you will learn kernel build process better.

tool builds the image wrong

My guess is that build script can't find some tool and silently skips the step producing wrong kernel blob. Check logs for more info.

mradermaxlol · 2016-01-11 07:24:20

anatolik wrote:

mkbootimg is a tool located in android-tools Arch package. It gets kernel blob and ramdisk blob and creates an Android boot partition. The tool is very simple and description can be found here https://android.googlesource.com/platfo … /bootimg.h
there is a way to create a DIY-buildscript?
Creating a boot image is as simple as compiling kernel and concatenating it with ramdisk. Plus some device-specific steps like signing.
I suggest you to understand what is that build script (from your repo) does and turn it into simple bash script like mine above. You'll be surprised how simple these build steps are. Plus you will learn kernel build process better.
tool builds the image wrong
My guess is that build script can't find some tool and silently skips the step producing wrong kernel blob. Check logs for more info.

Thanks for the help, I appreciate it. Again - will try all that once I'm home.

However, there's a trouble with that tool (MTKtools) - that's just a perl script and it doesn't produce logs. I can give you a link to the instruction with a link to that archive with MTKtools: http://4pda.ru/forum/index.php?showtopi … ry28283208

wget http://r2ahh.ru/files/android/android_boot_tools.tar.gz
To get the archive.

Last edited by mradermaxlol (2016-01-11 07:27:35)

mradermaxlol · 2016-01-11 08:13:33

My guess is that there are differences between Debian and Arch versions of.CPIO binary.

mradermaxlol · 2016-01-11 13:23:04

anatolik wrote:

mkbootimg is a tool located in android-tools Arch package. It gets kernel blob and ramdisk blob and creates an Android boot partition. The tool is very simple and description can be found here https://android.googlesource.com/platfo … /bootimg.h
there is a way to create a DIY-buildscript?
Creating a boot image is as simple as compiling kernel and concatenating it with ramdisk. Plus some device-specific steps like signing.
I suggest you to understand what is that build script (from your repo) does and turn it into simple bash script like mine above. You'll be surprised how simple these build steps are. Plus you will learn kernel build process better.
tool builds the image wrong
My guess is that build script can't find some tool and silently skips the step producing wrong kernel blob. Check logs for more info.

Alright, checked the build again, with fresh tools. That thing actually creates a CPIO image of ramdisk and uses mkbootimg to create a working image, but... Hmm, it just doesn't work. I've also noticed that images generated are:
1) Old MTKTools: 4204544 bytes
2) New MTKTools: 4171766 bytes
3) mkbootimg from your bootimg-tools-git: 4171766 bytes
4) mkbootimg from android-tools: 4171766 bytes
5) Forgot with which tool I did it, but got an image of 3.5Mb

Alex's bootable image weighs 4.4Mb.

mradermaxlol · 2016-01-11 15:01:17

Oh, dammit, the problem wasn't where it was searched.

The ramdisks were packed wrongly, and that was the reason. Now I'm able to get a working kernel.

Thanks everyone!

anatolik · 2016-01-11 16:53:22

Glad you sorted out this problem. I suggest you to share your experience with kernel hacking at Arch in your Android community e.g. using blog post. I think Arch suits better for development than other distros.

Another recommendation I found useful is to create your own shell script that compiles kernel and builds boot.img. Basically replace the MTKTools with your own simple solution. This way you learn about Android kernel build process a lot and compilation will be much faster (that is important if you plan to do frequent recompilations).

mradermaxlol · 2016-01-11 18:09:35

anatolik wrote:

Glad you sorted out this problem. I suggest you to share your experience with kernel hacking at Arch in your Android community e.g. using blog post. I think Arch suits better for development than other distros.
Another recommendation I found useful is to create your own shell script that compiles kernel and builds boot.img. Basically replace the MTKTools with your own simple solution. This way you learn about Android kernel build process a lot and compilation will be much faster (that is important if you plan to do frequent recompilations).

Well, after looking into that script and its debug logs (even though I'm dumb at Perl ) I can see that there's just a simple "Make ramdisk cpio from dir" script using GZip and CPIO, and then it's just a parse into mkbootimg tool. I think I'll make a bash script, or something similar in Python.

Trilby · 2016-01-11 18:15:01

mradermaxlol, please use quotes sparingly. Quote the relevant parts of previous posts where necessary. Generally if you are replying to a post immediately before yours in an orderly conversation with few participants, then I'd say no quoting at all is needed. If there is a specific part of a previous post you'd like to comment on, then quote only that part.

Also, you can edit your posts to add new information. Several posts up, you quoted an entire long post that came immediately before yours just to say "I'll try that", then you quoted that quote of the long quote in the very next post to report your results. This makes the thread very hard to read.

mradermaxlol · 2016-01-11 20:53:54

Woops, sorry. I was writing from a mobile phone and editing there is awful. That will (hopefully) not happen again

Arch Linux

#1 2016-01-10 11:45:33

[SOLVED] Building Android kernel images in Arch

#2 2016-01-10 11:56:06

Re: [SOLVED] Building Android kernel images in Arch

#3 2016-01-10 12:19:52

Re: [SOLVED] Building Android kernel images in Arch

#4 2016-01-10 15:06:34

Re: [SOLVED] Building Android kernel images in Arch

#5 2016-01-10 15:07:05

Re: [SOLVED] Building Android kernel images in Arch

#6 2016-01-10 15:15:44

Re: [SOLVED] Building Android kernel images in Arch

#7 2016-01-10 15:37:36

Re: [SOLVED] Building Android kernel images in Arch

#8 2016-01-10 15:47:11

Re: [SOLVED] Building Android kernel images in Arch

#9 2016-01-10 15:54:35

Re: [SOLVED] Building Android kernel images in Arch

#10 2016-01-10 15:59:21

Re: [SOLVED] Building Android kernel images in Arch

#11 2016-01-10 20:22:40

Re: [SOLVED] Building Android kernel images in Arch

#12 2016-01-11 04:47:58

Re: [SOLVED] Building Android kernel images in Arch

#13 2016-01-11 05:26:00

Re: [SOLVED] Building Android kernel images in Arch

#14 2016-01-11 05:34:44

Re: [SOLVED] Building Android kernel images in Arch

#15 2016-01-11 06:23:53

Re: [SOLVED] Building Android kernel images in Arch

#16 2016-01-11 06:37:17

Re: [SOLVED] Building Android kernel images in Arch

#17 2016-01-11 07:24:20

Re: [SOLVED] Building Android kernel images in Arch

#18 2016-01-11 08:13:33

Re: [SOLVED] Building Android kernel images in Arch

#19 2016-01-11 13:23:04

Re: [SOLVED] Building Android kernel images in Arch

#20 2016-01-11 15:01:17

Re: [SOLVED] Building Android kernel images in Arch

#21 2016-01-11 16:53:22

Re: [SOLVED] Building Android kernel images in Arch

#22 2016-01-11 18:09:35

Re: [SOLVED] Building Android kernel images in Arch

#23 2016-01-11 18:15:01

Re: [SOLVED] Building Android kernel images in Arch

#24 2016-01-11 20:53:54

Re: [SOLVED] Building Android kernel images in Arch

Board footer