We are of course going to pick most of our information from the official kernel building guide, though, unlike the guide, we're going to go all the way through, and not stop before we are actually running our recompiled kernel.
This time, we'll use Linux as our development platform (more convenient). The first thing I need to warn you about then is, if you're running a pure 64 bit distro (such as Slackware x64), you're going to find that you must install a 32-bit compatibility layer to run the 32 bit debug utilities. This is a bit annoying, and I'm not going to guide you through that, so make sure you sort your platform issues, okay?
Installing the toolchain
Anyway, first order of the day is to get the official arm toolchain, which I'm going to install in
/usr/local/share/
since I plan to keep using it for some time:# cd /usr/local/share/ # git clone https://android.googlesource.com/platform/prebuilts/gcc/linux-x86/arm/arm-eabi-4.6 Cloning into 'arm-eabi-4.6'... remote: Sending approximately 124.64 MiB ... remote: Counting objects: 33, done remote: Finding sources: 100% (33/33) remote: Total 580 (delta 146), reused 580 (delta 146) Receiving objects: 100% (580/580), 124.64 MiB | 715 KiB/s, done. Resolving deltas: 100% (146/146), done.
Now, let's add that arm toolchain to our path:
# export PATH=$PATH:/usr/local/share/arm-eabi-4.6/bin # arm-eabi-gcc --version arm-eabi-gcc (GCC) 4.6.x-google 20120106 (prerelease) Copyright (C) 2011 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Ideally, of course, you'd want to add this export into your
.profile
or something so that you don't have to set it up every time.Dude, where's the kernel source?
OK, now we want to fetch the kernel source of course, so let's do just that. The one thing you need to know is that the platform name for the Nexus 7 2013 is msm, thus:
# mkdir /usr/src/android # cd /usr/src/android/ # git clone https://android.googlesource.com/kernel/msm.git kernel Cloning into 'kernel'... remote: Sending approximately 850.89 MiB ... remote: Counting objects: 50642, done remote: Finding sources: 100% (5223/5223) remote: Getting sizes: 100% (699/699) remote: Compressing objects: 99% (11626/11627) remote: Total 3195022 (delta 2625647), reused 3194640 (delta 2625583) Receiving objects: 100% (3195022/3195022), 850.15 MiB | 716 KiB/s, done. Resolving deltas: 100% (2634067/2634067), done.
As can be seen from the above, expect a download of about 850 MB.
# cd kernel # ls #
Huh, there's nothing in there?
# git show commit f688a7654b339885e689d0f95f38b9daa3d85c0f Author: Jean-Baptiste Queru <jbq@google.com> Date: Thu Nov 17 12:44:41 2011 -0800 empty commit lines 1-5/5 (END) # git branch -a * master remotes/origin/HEAD -> origin/master remotes/origin/android-msm-2.6.35 remotes/origin/android-msm-3.9-usb-and-mmc-hacks remotes/origin/android-msm-flo-3.4-jb-mr2 remotes/origin/android-msm-mako-3.4-jb-mr1 remotes/origin/android-msm-mako-3.4-jb-mr1-fr remotes/origin/android-msm-mako-3.4-jb-mr1-kgsl remotes/origin/android-msm-mako-3.4-jb-mr1.1 remotes/origin/android-msm-mako-3.4-jb-mr2 remotes/origin/android-msm-sony-cm-jb-3.0 remotes/origin/master
Mmmm, OK, maybe the flo branch then?
# git checkout android-msm-flo-3.4-jb-mr2 Checking out files: 100% (41678/41678), done. Branch android-msm-flo-3.4-jb-mr2 set up to track remote branch android-msm-flo-3.4-jb-mr2 from origin. Switched to a new branch 'android-msm-flo-3.4-jb-mr2' root@stella:/usr/src/android/kernel# git show commit 9e52a2195889f4f03ddfcaefb19c0b42ec1d9070 Author: Praveen Chavan <pchavan@codeaurora.org> Date: Fri Mar 29 17:28:31 2013 -0700 qseecom: Fix issue with incomplete command exiting prematurely A process waiting on a signal can be awaken by any signal. We need to only continue processing when the condition of the wait event is met. Change-Id: Ib2102babbb505876f89b04399729e6ff5a475605 Signed-off-by: Mona Hossain <mhossain@codeaurora.org> Signed-off-by: Praveen Chavan <pchavan codeaurora.org>
Let me spare you the suspense there: As "documented" on their page, Google don't actually use branches or even tags for their development. Instead they force you to use an sha1 reference that's not attached to any helpful entity, in order to figure out the branch you should pick. Moreover, what they advise you to do to get that SHA is go for another large download of pointless binary data (650 MB) containing all the revision of their kernel files. Well, not everybody's running on Google Fibre... yet, and my ISP also enforces quotas, so, since we're smarter than this, we're going to avoid downloading 650 MB in exchange of a few SHA bytes. Instead, we'll head directly to https://android.googlesource.com/device/asus/flo-kernel and look at the most recent "flo: prebuilt kernel", which, at the time of this post, is:
79816e3 flo: prebuilt kernel by Jean-Baptiste Queru - 2 months ago jb-mr2-dev master tools_r22.2 android-4.3_r2.1
If you click on the link, you'll see part of the SHA we are really after here:
365a6e0 gpu: ion: Minimize allocation fallback delay
That
365a6e0
is the ID we're seeking, so let's use that (and also reuse the official kernel name, 'android-4.3_r2.1', for our branch name):# git checkout -b android-4.3_r2.1 365a6e0 Switched to a new branch 'android-4.3_r2.1' # ls AndroidKernel.mk CREDITS Kbuild MAINTAINERS README arch/ crypto/ firmware/ include/ ipc/ lib/ mm/ samples/ security/ tools/ virt/ COPYING Documentation/ Kconfig Makefile REPORTING-BUGS block/ drivers/ fs/ init/ kernel/ make_defconfig.sh* net/ scripts/ sound/ usr/
That's more like it!
NB: We could be even smarter by just looking at the kernel revision provided by our device (
3.4.0-g365a6e0
), drop the whole part before the g
and recognize that this is the partial SHA1 we are after. Then again, since there's a chance a new kernel was out and you're running an old or custom one...Compiling the Android kernel
Now we need to set the variables that will tell the OS that we're cross compiling (again, something that you may want to do in your
.profile
using something like alias cross='export ARCH="arm";export SUBARCH="arm";export CROSS_COMPILE="arm-eabi-"'
):# export ARCH=arm # export SUBARCH=arm # export CROSS_COMPILE=arm-eabi-
Finally, we can initialize our default config and get going:
# make flo_defconfig HOSTCC scripts/basic/fixdep HOSTCC scripts/kconfig/conf.o SHIPPED scripts/kconfig/zconf.tab.c SHIPPED scripts/kconfig/zconf.lex.c SHIPPED scripts/kconfig/zconf.hash.c HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT) warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT) # make menuconfig HOSTCC scripts/kconfig/lxdialog/checklist.o HOSTCC scripts/kconfig/lxdialog/inputbox.o HOSTCC scripts/kconfig/lxdialog/menubox.o HOSTCC scripts/kconfig/lxdialog/textbox.o HOSTCC scripts/kconfig/lxdialog/util.o HOSTCC scripts/kconfig/lxdialog/yesno.o HOSTCC scripts/kconfig/mconf.o HOSTLD scripts/kconfig/mconf scripts/kconfig/mconf Kconfig warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT) warning: (ARCH_MSM_KRAITMP && ARCH_MSM_CORTEX_A5) selects HAVE_HW_BRKPT_RESERVED_RW_ACCESS which has unmet direct dependencies (HAVE_HW_BREAKPOINT)
Now that we're in the kernel config, let's start with something real basic, and change
General setup ---> Local version - append to kernel release
setting it to something like "hello". Hopefully, we'll be able to boot our recompiled kernel and see that string appear in the system info, to confirm that we are indeed using our own kernel. Off we go with building the whole thing then:# make -j8 (...) OBJCOPY arch/arm/boot/zImage Kernel: arch/arm/boot/zImage is ready
We have a kernel — now what?
Aaaaand, that's pretty much where the Google guide stops. What do you mean, you also want to run your newly compiled kernel on your device?
If that's the case, then you should download more than 16 GB worth of build files from the infamous aosp (Android Open Source Project) suite —I wish I was kidding, see below— and then figure out how to inject your kernel in there. Good luck!
My usage data (in MB) after downloading the full aosp. (Careful analysis may be required to notice the small difference in data usage) |
The 16 GB of the full aosp (I was already 2 GB in for that day, hence the 18 GB above) sure put me dangerously close of getting a not so friendly call from my ISP for going over the monthly quota (again, not everybody is running on Google Fibre), so I might try to spare you this trouble.
If you're not going to download the aosp to try to figure out this mess, the first thing you should know then is that, as opposed to what is the case on most devices, testing your kernel on Android isn't as simple as pointing your bootloader to it. It must be included into a custom boot image (
boot.img
) along with a RAM disk and some other stuff.Crafting an Android boot.img
The format of this Android boot images can be found in the
bootimg.h
of the mkbootimg
tool from the android platform/system/core
source (at least up to Android 10). I guess we have to start somewhere, and we'll need to produce our own images, so let's start by fetching and recompiling mkbootimg
then.NB: Since I duplicated and improved on the original tools (see below), you can, and probably should, use https://github.com/pbatard/bootimg-tools.git instead of https://android.googlesource.com/platform/system/core for the git clone URL below.
# git clone https://android.googlesource.com/platform/system/core bootimg-tools Cloning into 'bootimg-tools'... remote: Counting objects: 92, done remote: Finding sources: 100% (92/92) remote: Total 19302 (delta 11674), reused 19302 (delta 11674) Receiving objects: 100% (19302/19302), 5.87 MiB | 655 KiB/s, done. Resolving deltas: 100% (11674/11674), done. # cd bootimg-tools/libmincrypt/ # gcc -c *.c -I../include # ar rcs libmincrypt.a *.o # cd ../mkbootimg # gcc mkbootimg.c -o mkbootimg -I../include ../libmincrypt/libmincrypt.a mkbootimg.c: In function 'main': mkbootimg.c:245:9: warning: assignment discards 'const' qualifier from pointer target type [enabled by default] # cp mkbootimg /usr/local/bin/ # cd ../cpio # gcc mkbootfs.c -o mkbootfs -I../include # cp mkbootfs /usr/local/bin/
With the above, we'll be able to invoke
mkbootimg
and optionally mkbootfs
to create an image. If you follow this guide closely, you'll see that we're not even actually going to use mkbootfs
, but I kept it there just in case.Now, we don't want to recreate our RAM disk from scratch, or have to guess all the parameters we'll need to use to create one, so let's pick a working one. For this you have two choices:
- Use a backup utility on your rooted Android device, and save the partition that contains the boot image. For instance, if you installed TWRP when rooting your device, and boot into recovery, then you will be able to save the 16MB Boot partition, which is what we are after.
- Download the latest factory image (at the time of this writing, it's JSS15R, 359MB big), extract it in full (tar then zip), and then pick the
boot.img
image from the zip (in JSS15R, that would beboot.img
withinimage-razor-jss15r.zip
). You also want to be mindful thatbootloader-flo-flo-03.14.img
is the bootloader image, and NOT what we are after.
For the remainder of this guide, even as it requires a 359 MB download, I'll go with using
boot.img
from the factory image.Next thing we need is a way to unpack that
boot.img
file. There are a couple of unpack-bootimg.pl
/split-bootimg.pl
perl scripts dating back from 2008, so we can pick one of those and then use a binary editor take a look at the .img
and extract the addresses and parameters we should provide mkbootimg
, and try to cross-reference those with... Aaaaagh, I can't stand it any more!!!That's "Aaaaagh!" with five A's
What the £$%^& is wrong with Android developers?!? Why the heck aren't these unpacker scripts providing you with the full set of parameters you need to use to repack an image, instead of having to look at a binary header, or some mysterious settings file in a 16GB repository to find them? Also, why, in 5 years, hasn't anyone, including Google, improved these tools and scripts? The thing is, even
mkbootimg
has issues and won't work on Windows (when compiled with MinGW) due to the use of non-Windows friendly open()
, read()
, write()
et cetera.That does it!
I'm gonna write my own unpack tool in C, fix
mkbootimg
in the process, and put it all on github.Sheesh, as if I didn't have better things to do... I hope you're happy!
So off we go to the directory you compiled
mkbootimg
(NB: I you cloned from https://github.com/pbatard/bootimg-tools.git
earlier, you can skip the wget
line):# cd /usr/src/android/bootimg-tools/mkbootimg/ # wget https://raw.github.com/pbatard/bootimg-tools/master/mkbootimg/unmkbootimg.c # gcc -o unmkbootimg unmkbootimg.c # cp unmkbootimg /usr/local/bin/
Now at last, you have the minimum of
unmkbootimg
, mkbootimg
and mkbootfs
installed in your path, and we can get going with our testing of the kernel.I'll assume that you have a
/usr/src/android/boot
where you copied your boot.img
, so let's get cracking:# cd /usr/src/android/boot/ # ls boot.img # unmkbootimg -i boot.img kernel written to 'kernel' (6640200 bytes) ramdisk written to 'ramdisk.cpio.gz' (399979 bytes) To rebuild this boot image, you can use the command: mkbootimg --base 0 --pagesize 2048 --kernel_offset 0x80208000 --ramdisk_offset 0x82200000 --second_offset 0x81100000 --tags_offset 0x80200100 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=flo user_debug=31 msm_rtb.filter=0x3F ehci-hcd.park=3' --kernel kernel --ramdisk ramdisk.cpio.gz -o boot.img # ls boot.img kernel ramdisk.cpio.gz
As an aside that you don't need to run, but since it should be elementary that this is the basic functionality you want from a proper boot image unpack tool, we can confirm that the data provided by the unpack tool will produce a
boot.img
that is binary identical to the original one:# mkbootimg --base 0 --pagesize 2048 --kernel_offset 0x80208000 --ramdisk_offset 0x82200000 --second_offset 0x81100000 --tags_offset 0x80200100 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=flo user_debug=31 msm_rtb.filter=0x3F ehci-hcd.park=3' --kernel kernel --ramdisk ramdisk.cpio.gz -o myboot.img # ls boot.img kernel myboot.img ramdisk.cpio.gz # cmp -l boot.img myboot.img #
Now, THIS is what you want to be able to do from an Android image unpacking tool. Is it really that much to ask?
Crafting an Android boot.img 2: 'Electric boogaloo'
Moving on. Since we just want to test a kernel, we shouldn't really have to touch the
cpio
image (ramdisk), but then again, my goal here is to give you as many pointers as I can, so we might as well see how we craft our own ramdisk while we're at it. What we're going to do here, as an academic exercise, is add an it_works
file at the root of the filesystem, which we'll look for after we booted, to confirm that we are able to use our modified stuff all the way through:# mkdir ramdisk # cd ramdisk # gunzip -c ../ramdisk.cpio.gz | cpio -iu # ls charger* init* proc/ sys/ data/ init.flo.rc* property_contexts system/ default.prop init.flo.usb.rc* res/ ueventd.flo.rc dev/ init.rc* sbin/ ueventd.rc file_contexts init.trace.rc* seapp_contexts fstab.flo init.usb.rc* sepolicy # touch it_works # ls charger* init* it_works sepolicy data/ init.flo.rc* proc/ sys/ default.prop init.flo.usb.rc* property_contexts system/ dev/ init.rc* res/ ueventd.flo.rc file_contexts init.trace.rc* sbin/ ueventd.rc fstab.flo init.usb.rc* seapp_contexts # find . | cpio -o -H newc | gzip > ../myramdisk.cpio.gz 1410 blocks # cd .. # ls boot.img kernel myramdisk.cpio.gz ramdisk/ ramdisk.cpio.gz
We're finally set for the last part, where we copy the kernel we compiled earlier, and invoke
mkbootimg
with the set of parameters we got from unmkbootimg
, and use both our modified kernel and cpio image:# cp /usr/src/android/kernel/arch/arm/boot/zImage . # ls boot.img kernel myramdisk.cpio.gz ramdisk/ ramdisk.cpio.gz zImage* # mkbootimg --base 0 --pagesize 2048 --kernel_offset 0x80208000 --ramdisk_offset 0x82200000 --second_offset 0x81100000 --tags_offset 0x80200100 --cmdline 'console=ttyHSL0,115200,n8 androidboot.hardware=flo user_debug=31 msm_rtb.filter=0x3F ehci-hcd.park=3' --kernel zImage --ramdisk myramdisk.cpio.gz -o myboot.img # ls boot.img kernel myboot.img myramdisk.cpio.gz ramdisk/ ramdisk.cpio.gz zImage*
Finally, a custom
boot.img
we can test. Let's press on by copying this myboot.img
file into the directory we have adb
and fastboot
installed (see our earlier post about rooting the Nexus 7) and run the following set of commands which, unlike what many other guides seem to advise (what the heck, guys?), is NOT going to flash the kernel/boot.img but simply run it from memory. This means that, in case there's any mishap, you can simply power the Nexus off and you'll be good as new:# ./adb start-server * daemon not running. starting it now on port 5037 * * daemon started successfully * # ./adb reboot bootloader # ./fastboot boot myboot.img downloading 'boot.img'... OKAY [ 0.223s] booting... OKAY [ 0.023s] finished. total time: 0.246s
The moment of truth!
After a long few seconds of anxiety, where the device doesn't seem to do anything, you should see either one of these two things:- The Nexus' multicoloured 'X', telling you that your boot.img file was fine and that the tablet is now booting it, or
- The Google logo with the open lock, indicating that the device didn't accept your boot image and is restarting with the one it has in flash (or complete freezout, if which case you should press the power button until your device turns off)
A few things you should recognize above:
- That
hello
appended to the3.4.0
kernel version, that's the "local version" string we chose to add to our kernel when we recompiled root@stella
as the user who compiled this kernel (yeah, I tend to compile stuff as root - so sue me!)- That
365a6e0
after theg
(for 'git') in the version number, that's the SHA1 from the git commit we had to pick to compile our kernel.
it_works
file is there, so we can also add whatever we want on the initial filesystem. Neat!From there on, you can go ahead and tweak your kernel and initial filesystem exactly as you see fit. And after you have tested that everything works as it should, you can go ahead and flash the boot partition with your shiny custom boot.img, using:
fastboot flash boot boot.img
For the record, if you actually modify the source of your kernel, you should see a
-dirty
qualifier appended to your kernel version, which is normal, and won't prevent you to run a modified kernel.Now what are you doing still reading this guide: get hacking your device already!