Linux Kernel Compilation

Steps in compiling a kernel:

• Installing the sources.
• Configuring the kernel (choosing which features and Drivers to compile).
• Compiling the kernel (i.e. typing a single command, and watching...).
• Installing the compiled kernel.
• Updating the boot loader to recognize the new kernel.
• Booting...
• Making the new kernel become the default.

Kernel "Types"

• The Linux kernel comes in two variants - the "vanilla" kernel, and the distribution's kernel.
• The "vanilla" kernel is the kernel officially released by Linus, or by a member of the community appointed by Linus (e.g. kernels version 2.4.X are officially maintained and released by Marcelo Tosatti).
• The distribution's kernel is normally a "vanilla" kernel, with many patches on top of it that either did not get accepted to the "vanilla" kernel, or that the distribution's maker back-ported from newer kernels.
• Thus, normally the latest "vanilla" kernel has more features...
• ... while the distribution's kernel has gone through more orderly testing and could be more stable.

Getting The Source

• The sources of linux kernels are available via the Internet, or on the distribution's CDs.
• Of-course, there are many mirrors, and since we're talking about not-so-small files (around 30MB), we better know our mirrors.
• In Israel, one may try Iglu's mirror, at http://www.iglu.org.il/, or check Hamakor's Israeli mirrors list, at http://mirror.hamakor.org.il/.

The Source Of The Distribution's Kernel

• The source code of the distribution's kernel comes as another package on the installation CDs.
• On RedHat 9, for example, it is stored in a file named 'kernel-source-2.4.20-8.i386.rpm'.
• Installing it is done like installing any other RPM package:

       rpm -Uvh /path/to/kernel-source-2.4.20-8.i386.rpm
       

• If The distribution's maker updated the kernel, the new kernel sources would be found among the updates.
• In RedHat 9, the current file is kernel-2.4.20-28.9.src.rpm, found in the SRPMs (Source-RPMs) directory.

The Source Of The Vanilla Kernel

• The "vanilla" kernel is normally kept at ftp.kernel.org, as a tar.gz (or tar.bz2 - better compressed) file.
• For example, the latest 2.4 kernel for now is found at ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.24.tar.bz2
• Unpack these kernel sources as root:

       cd /usr/src/
       tar xjf /path/to/linux-2.4.24.tar.bz2
       

• You will get a new directory named 'linux-2.4.24', containing the sources.

"Readying" The Sources

• Before we do anything with the sources, we should make sure the source tree does not contain any old object files or configuration information...
• ... That got there accidentally.
• This is especially true for distribution kernels - they tend to contain various stale files.
• To do this, go into the source directory and run the command:

  make mrproper

Configuring The Kernel

• Before we compile the kernel, we need to configure it.
• This includes telling it which drivers and features to compile ...
• ... and how to compile them (as modules or inside the kernel's main file).
• Several configuration programs are supported by the kernel, to be launched using one of the following commands:

  make config

  A simple text-mode program, that asks a zillion questions one after the other. Not recommended.

  make menuconfig

  A full-screen text-mode program. Use it if you don't have X windows running, or if you're connected from a remote location.

  make xconfig

  A Graphical program. Use this one when you can.

• There is yet another, which will be shown later...

Tips Regarding Configuring The Kernel

• Each item has a help section - read it.
• Each help text contains a suggestion of what to do if you're not sure. Use these suggestions, indeed.
• The first time around, take a tour around the different options, without changing them.
• The configuration process generates a file called ".config" in the top directory of the kernel sources. Keep a backup before making changes.
• Don't configure a kernel when you don't have enough free time - it's a long task (1-2 hours) the first time around.
• Do not despair - eventually, you'll configure kernels in 5 minutes ;)

Compiling The Kernel

• Once configured, Compiling the kernel is easy.
• First, make sure we start afresh (takes a few seconds):

  make clean

• Then, prepare the dependencies list (might take a minute):

  make depend

Compiling The Kernel (Cont.)

• Then, compile the kernel's main part (might take 5-30 minutes):

  make bzImage

  After this step, we should have the following new file:

  [root@simey linux]# ls -l arch/i386/boot/bzImage 
  -rw-r--r--    1 root   root   1064017 Jan 16 01:53 arch/i386/boot/bzImage
       

• Then, compile the kernel modules (might take 5-30 minutes):

  make modules

Compilation Errors

• A normal compilation process might result several warnings.
• But errors should not happen, unless you're compiling a non-stable kernel.
• Make sure you started from a fresh compilation (i.e. ran 'make clean').
• Make sure that you have read/write access to the source tree (i.e. if the sources belong to 'root', compile them as user 'root').

Installing The New Kernel

• Installing the kernel is split into two parts:
  1. First, installing the kernel itself.
  2. Then, installing the kernel modules.
• Before we install the kernel, we want to make sure we do not overrun our current kernel, or a previously existing kernel.
• So we will install the kernel itself manually.
• Avoiding overrunning the kernel modules is more difficult, since the location of the modules is hard-coded into the kernel (its version number) and the module loading tools.

Installing The Kernel's Main File

• Runnable kernels are expected to be in the /boot directory.
• Simple way to install the kernel (assuming it is version 2.4.20-8):

  cp arch/i386/boot/bzImage /boot/vmlinuz-2.4.20-8

• Make sure the name is unique, and especially different then the current kernel...
• ... cause if we got a broken kernel, we will want to be able to switch back to the current kernel.

Installing The Kernel Modules

• The kernel modules will normally be placed in /lib/modules/<version>
• If the compiled kernel is of the same version as an existing kernel, we better first back-up the modules library:

  cp -rp /lib/modules/2.4.20-8 /lib/modules/2.4.20-8.old

• This assumes that the older kernel can boot into a working system without working loadable modules...
• ... or that we can boot from a rescue CD/floppy to restore the modules.
• And now install the modules:

  make modules_install

The "initrd" (Init Ram-Disk) File

• Sometimes, we need to pack modules that are needed before the kernel can access the disk partitions, into the 'initrd' file.
• To do this, we need to use the 'mkinitrd' command:

  mkinitrd /boot/initrd-2.4.20-8.img 2.4.20-8

  This creates an initrd file named '/boot/initrd-2.4.20-8.img', containing hard-disk and file-system related modules, for kernel version 2.4.20-8.
• The initrd command uses the modules installed under /lib/modules/, so it must be executed after make modules_install.

Updating The Boot Loader

• Once we installed the new kernel and its new modules, we need to tell the boot loader about it.
• Generally, we need to supply the following information:
  • Kernel image file path (e.g. /boot/vmlinuz-2.4.20-8).
  • Partition of the root directory - copy it from the spec of the current kernel.
  • Kernel parameters - may be copied from the current kernel, if it has not changed drastically.
  • Optional initrd file path.
  • Label.

Instructions For "lilo"

• If your system uses "lilo" as the boot loader, the config file is normally at /etc/lilo.conf
• A normal entry for our example kernel would look like this:

  image = /boot/vmlinuz-2.4.20-8
  initrd=/boot/initrd-2.4.20-8.img
  label = rh9-mykernel
  append="hdc=ide-scsi"
  root=/dev/hda1

• the 'root' entry may be omitted, if we have 'root=current' in the global section.
• If we do not need initrd, the initrd entry may be omitted.

Instructions For "lilo" (Cont.)

• Finally, run "lilo" to make the actual update of the boot loader:

  [root@simey ~]# lilo
  Added linux-2.4.18
  Added rh9-mykernel
  Added dos

  MANY people forget this step!

Instructions For "grub"

• For systems with "grub" as their boot loader, the config file is normally at /boot/grub/grub.conf
• An entry for our example kernel would look like this:

  title Red Hat Linux (2.4.20-8)
          root (hd0,0)
          kernel /boot/vmlinuz-2.4.20-8 ro root=/dev/hda1 hdc=ide-scsi
          initrd /boot/initrd-2.4.20-8.img

• The disks and partitions are numbered by bus numbers, rather then by letters: hd0,0 means /dev/hda1.
• If we have /boot on its own partitions, the path names should NOT contain /boot, and the "root" entry should point to the /boot partition.
• No need to run anything after updating the config file - grub will read it during system boot.

Booting The New Kernel

• Once everything is set, reboot your machine.
• At the boot loader's prompt/menu, chose the new kernel.
• Watch the boot messages - are they similar to what you are familiar with? Are there any errors or failures?
• If boot fails - reboot again with the previous (working) kernel.
• If boot succeeds, check that everything works (including networking, sound card, modem...).

Troubleshooting The Boot Process

• The boot process logs all its messages into /var/log/messages
• Possibly also in /var/log/dmesg
• Look in those files for error messages, and try to analyze them or look them up on the Internet.
• Sometimes you'll notice a problem resulting from kernel configuration - reconfigure, recompile from scratch (make clean...), re-install and try again.
• If you see that all modules fail to load with unresolved symbols, make sure you compiled and installed the modules.

Boot Problems - No Root Device

• Symptom: you get the message that no root device was found.
• Reboot to the previous kernel.
• Check the exact syntax of your lilo/grub entry for the new kernel. Are you sure you specified the root device entry properly?
• Make sure you compiled the drivers for your hard-disk and the partitions into the kernel image, or you have them in the initrd image.

Boot Problems - "lilo" says "LI"

• Symptom: you get the message "LI" (or a similar message) and then nothing, or an endless loop of garbage.
• Reboot to the previous kernel.
• Check the syntax of the lilo entry for the kernel - something there is terribly wrong.
• If you can't see what is wrong, delete the entry, and create it again. Often mistakes done in the first copy+paste do not occur in the second attempt.

When Installing A Newer Kernel...

• Check the file 'Documentation/Changes' in the source directory, for packages that might need upgrading before booting the new kernel.
• Don't configure the new kernel from scratch - copy your previous kernel's '.config' file to the new kernel's source directory, and run make oldconfig - you will only need to answer questions about new features.
• If you used a distribution's kernel previously, it normally has a config file matching the installed kernel.
• On RedHat, this file is usually under /boot/config-<version>

External Device Drivers

• Various types of hardware might have drivers available not as part of the kernel sources.
• Perhaps this is a new driver, that didn't yet get accepted into the normal kernel.
• Or the driver is maintained by the manufacturer of the hardware...
• ... and possibly delivered in binary-only mode?
• We need to be able to install such drivers on our own, then.

External Device Drivers Formats

• An external device driver might come with full sources, that we need to compile. This is the best scenario.
• An external device driver might come as a binary-only module. In this case we need to get a driver that was compiled specifically for the kernel version we are using.
• If we use a less-commonly-used distribution, we might not find a driver for our distribution's kernel...
• At which time we either dumb the hardware, or take the best "vanilla" kernel for which there is a binary driver available.
• The driver could come as a hybrid - half source, and half binary. the Source-part need to be compiled against our kernel, and it'll make sure the binary part works.

Notes On External Device Drivers Compilation

• You need to have your kernel source tree configured properly, matching your running kernel.
• You don't have to actually compile the kernel source tree in this case.
• Sometimes the driver has a script that will compile the driver for you, hiding the actual compilation process.
• If it thinks your kernel sources are under /lib/modules/2.4.20-8/build, this is ok - this should be a symbolic link pointing to the right location of the sources....
• ... But it does not hurt to make sure.
• Otherwise, read the driver's installation instructions, and follow them carefully.

Originally written by guy keren