Minimalist Gentoo Builds, Revisited

Sunday 13 July 2014 04:45 +00:00

Last year (wow, time files), I posted a guide for building minimalist Gentoo systems for embedded devices like the Raspberry Pi using crossdev. The process I outlined there, while functional, was unweildy and confusing, and fell apart if you wanted to build multiple systems with different configurations. I have improved the process somewhat, making it easier to follow and eliminate the need to rebuild the toolchain for each system you want to build.

Since my last post, Minimalist Gentoo for the Raspberry Pi, the package set has changed somewhat:

app-shells/bash
app-arch/bzip2
sys-apps/coreutils
sys-apps/file
sys-apps/findutils
sys-apps/gawk
sys-apps/grep
app-arch/gzip
sys-apps/kbd
sys-apps/kmod
sys-apps/less
sys-apps/openrc
sys-apps/net-tools
sys-process/procps
sys-apps/sed
sys-apps/shadow

Staging Areas

As before, cross-compiling takes place in multiple stages. This time, there are only two:

build root
deployment root

Again, we'll be using the buildpkg FEATURES flag, so each package only has to be built once.

Build Root

The build root is where everything gets installed as the system is being built. This includes all the packages we want, plus their runtime dependencies, and their build dependencies as well.

Deployment root

While the build root could theoretically be copied as-is to the final filesystem, it's better to use the binary packages built in the build root and install them in an alternate location. In this way, only the desired packages and their runtime dependencies are installed on the final system, not build dependencies.

Crossdev

If you already have a crossdev toolchain you'll probably want to remove it and start over:

crossdev -C armv6j-hardfloat-linux-gnueabi

If you are prompted to recursively remove the directory, say yes.

Create a new toolchain (I still recommend the stable versions):

crossdev -S -t armv6j-hardfloat-linux-gnueabi

Configuration

The Portage configuration generated by crossdev is broken, so the easiest thing to do is start from scratch.

First, create the Portage configuration directory structure in your build root (I am using /var/tmp/rpi-build as my build root, but it can be whatever you want):

mkdir -p /var/tmp/rpi-build/etc/portage/profile

Next, create a make.defaults file in the profile directory and put the following content in it:

ARCH=arm
ELIBC=glibc
ACCEPT_KEYWORDS="arm"
FEATURES="-news buildpkg"
USE="arm bindist minimal"

This is necessary because Portage EAPI 5 made ARCH and ELIBC "profile-only" variables, so they can't go in make.conf anymore. The contents of make.conf are still important though, so create it next:

CFLAGS="-Os -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s  --sysroot=/var/tmp/rpi-build"
CXXFLAGS="${CFLAGS}"
LDFLAGS="--sysroot=/var/tmp/rpi-build -Wl,--sysroot=/var/tmp/rpi-build -L=/usr/lib"

# Add any other variables you like here, such as MAKEOPTS, USE, etc.

NOTE: The CFLAGS in this example are for a Raspberry Pi. If you are building for something else, like a Beagle Bone, make sure you adjust them accordingly. DO NOT, however, change or remove the --sysroot flag, as it is the key to making the whole thing work.

You can also create package.use, package.keywords, etc directories in /var/tmp/rpi-build/etc/portage, just like you would with a regular system.

Now, tell Portage to use this configuration instead of the default:

export ROOT=/var/tmp/rpi-build
export PORTAGE_CONFIGROOT=$ROOT

Bootstraping the Build Root

In order to use the --sysroot flag for the compiler/linker, glibc and kernel-headers need to be installed in the target directory. We've created a chicken-and-egg problem by putting --sysroot in make.conf; if we try to build glibc in the sysroot, it will try to use the glibc already there (which of course doesn't exist). Thus, we have to temporarily override CFLAGS and LDFLAGS to get things going:

CLFAGS= LDFLAGS= armv6j-hardfloat-linux-gnueabi-emerge --oneshot --noreplace virtual/libc virtual/os-headers

This will reset the compiler and linker flags back to their defaults. While there's nothing wrong with that, the result will not be as highly optimized as it could be. You may want to specify a more complete CFLAGS instead.

Building Packages

Now that the build root is also a sysroot, building subsequent packages is a cinch:

armv6j-hardfloat-linux-gnueabi-emerge --usepkg --noreplace --changed-use bash bzip2 ...

All of these packages should build succesfully. If you add others, you may run into problems. For example, I wanted to build radvd, which depends on libdaemon. The version of libtool shipped with libdaemon is too dumb to understand the sysroot flag, so it fails trying to link with libraries in in the non-existant directory =/usr/lib. There are a few work-arounds, but the best thing to do is file a bug and help get the package fixed.

Deploying

Now that everything has been built and the binary packages have been created, it's time to deploy to the final system (SD card, etc.)

Make sure you follow the necessary instructions for creating and formatting your device's root filesystem (for the Raspberry Pi, see the Raspberry Pi page on Gentoo Wiki). I'll assume you've mounted the root filesystem at /mnt/raspberrypi

export ROOT=/mnt/raspberrypi

Create the empty top-level directories:

mkdir $ROOT/{boot,dev,proc,root,sys,tmp}

Install the binary packages that were created before:

armv6j-hardfloat-linux-gnueabi-emerge --usepkg --ask bash bzip2 ...

Copy the GCC runtime library:

cp /usr/lib/gcc/armv7a-hardfloat-linux-gnueabi/4.7.3/libgcc_s.so.1 $ROOT/lib

Other Bits

The remaining pieces, like clock and timezone settings and the Kernel, are still relevant from my previous post. Check out the "Finishing Up" section.

Embuilder

I've written a tool called embuilder that handles all of this for you. It's not quite finished, and I've not written any documentation at all. It's designed to be able to work with multiple different projects by reading settings from a single configuration file. When I get time, I'll write more about it. For now, here's an example configuration file and usage:

# Project settings
[embuilder]
name = MyRPi

# Portage configuration
[portage]
arch = arm
ctarget = armv6j-hardfloat-linux-gnueabi
# optional - directory containing extra portage configuration (not make.conf, though)
;configroot=configroot

# A group of packages to install in addition to the basics
[tools]
packages = iproute2 nano dhcpcd eudev dropbear

Save the file somewhere. I keep each of my embedded system projects in a separate Mercurial repository. Then, pass the name of the configuration file and the path to the deployment root to the embuilder command:

embuilder myrpi.ini /mnt/raspberrypi

That's it. Embuilder will take care of creating the build root, building binary packages, and deploying everything to the deployment root. It has some other neat features, like post-install scripts, overlay files, etc. that I will cover soon.