Compiling Qt with Docker multi-stage and multi-platform

This is a guest post from Viktor Petersson, CEO of Screenly.io. Screenly is the most popular digital signage product for the Raspberry Pi. Find Viktor on Twitter @vpetersson.

For those not familiar with Qt, it is a cross-platform development framework that is used in a wide range of products, including cars (Tesla), digital signs (Screenly), and airplanes (Lufthansa). Needless to say, Qt is very powerful. One thing you cannot say about the Qt framework, however, is that it is easy to compile — at least for embedded devices. The countless blog posts, forum threads, and Stack Overflow posts on the topic reveal that compiling Qt is a common headache.

As long-term Qt users, we have had our fair share of battles with it at Screenly. We migrated to Qt for our commercial digital signage software a number of years ago, and since then, we have been very happy with both its performance and flexibility. Recently, we decided to migrate our open source digital signage software (Screenly OSE) to Qt as well. Since these projects share no code base, this was a greenfield opportunity that allowed us to start afresh and explore exciting new technologies for the build process.

Because compiling Qt (and QtWebEngine) is a very heavy operation, we would need to pre-compile and distribute Qt so that the Dockerfile could simply download and include it in the build process (rather than compiling as part of the installation process).

We sat down and created the following requirements for our build process:

• The process must be fully automated from start to finish.
• We need to be able to build Qt/QtWebEngine for all supported Raspberry Pi boards (with the appropriate Qt device profile).
• We should use cross compilation on x86 to speed up the process where it makes sense.
• We need to be able to run the full process on CI, and thus cannot rely on a Raspberry Pi.
• We should confine everything to run inside Docker containers so we do not clutter the host with build packages.

With the above goals in mind, we had a great opportunity to try out the new multi-platform support in Docker. Used in conjunction with multi-stage builds, we were able to get the best of both worlds:

• Use emulation where we cannot cross-compile
• Switch to cross-compilation for the heavy lifting

How does multi-platform in Docker work?

The easiest way to use multi-platform functionality in Docker is to invoke it from the command line. Using the docker buildx, we can tap into new beta functionalities. By running docker buildx build --platform linux/arm/v7 -t arm-build . This command builds the docker image as per the `Dockerfile` in the current directory using ARMv7 emulation. Behind the scenes, Docker runs the whole Docker build process in a QEMU virtualized environment (qemu-user-static to be precise). By doing this, the complexity of setting up a custom VM is removed. Once built, we can even use docker run to launch containers in ARMv7 mode automagically.

Multi-platform, multi-stage and Qt

While multi-platform functionality is a great stand-alone feature, it gets even more powerful when combined with multi-stage builds. Within a single Dockerfile, we’re able to mix and match platforms and copy between the steps. This functionality is exactly what we ended up doing with the Qt build process for Screenly OSE.

Stage 1: ARM

Thanks to the fine folks over at Balena, we are able to use a Raspbian base image in the first stage. We can invoke this step using:

FROM --platform=linux/arm/v7 balenalib/rpi-raspbian:buster as builder

After the above step, we can use Docker as we normally do and execute various RUN commands, such as installing packages etc.. Do note that this container is running emulated using QEMU if the build is not run on ARMv7 hardware. In our case, we use the command to install the Qt build dependencies. The above step also allows us to fully eliminate the need for copying files from either a disk image (which is what the Qt Wiki suggests) or rsync files from a physical Raspberry Pi. 

Stage 2: x86

Once we have installed our dependencies in our ARM step, we can switch over to the builder’s native x86 architecture to avoid emulation and do the cross compile with the following line:

FROM --platform=linux/amd64 debian:buster

Now, we are onto the interesting part. After we have switched over to x86, we can copy files from the previous step. We do this in order to create a sysroot that we can use for Qt. We complete this step by running the following commands:

RUN mkdir -p /sysroot/usr /sysroot/opt /sysroot/lib

COPY --from=builder /lib/ /sysroot/lib/

COPY --from=builder /usr/include/ /sysroot/usr/include/

COPY --from=builder /usr/lib/ /sysroot/usr/lib/

COPY --from=builder /opt/vc/ sysroot/opt/vc/

We now have the best of both worlds. By taking advantage of both multi-step and multi-platform functionality, we generate a sysroot that we can use to build Qt. Since we used a fully functional Raspbian image in our previous step, we are even able to get Qt to pick up all existing libraries.

./configure \

-sysroot /sysroot

As we mentioned in the introduction, compiling Qt is far from straightforward. There are a lot of steps required to compile it successfully. To learn more about the exact steps, you can see the full Dockerfile and script build_qt5.sh. 

To emulate or not to emulate…

Being able to emulate a platform like ARM is amazing and provides a lot of flexibility. However, it does come at a cost. There is a big performance penalty. This issue is the reason why we do not actually compile Qt using emulation. Instead, we use cross-compilation. If you have the ability to cross-compile rather than emulate, know that cross-compilation will give you much better performance.

About Screenly

Screenly is the most popular digital signage product for the Raspberry Pi. If you want to turn a physical screen into a secure, remotely-controllable device (over UI or digital signage API) that can display dashboards, images, videos, and webpages, Screenly makes setup a breeze. Screenly is available in two flavors: an open source version and a commercial version.