Developing cross-platform applications presents a major challenge:, how to ensure that every commit does not break some combinations of operating systems and CPU architectures. Fortunately, thanks an array of online services and open-source tools, this challenge becomes easier to tackle.
For this demo, I have the traditional Hello, world program written in ANSI C/C90 at this repository: github.com/ariya/hello-c90 (feel free to take a look). The objective is to verify its automatic build (for the purpose of continuous integration) for a number of different CPU architectures, operating systems, as well as the C/C++ compilers. Supported CPU architectures are (using Debian nomenclatures) are amd64, i386, i686, armhf, arm64, and mips. Among some C/C++ compilers to be tested are GCC, Clang, TCC, Visual C/C++ (as part of Visual Studio 2017 and also 2019), Pelles C, Digital Mars, as well as MinGW. Obviously, some combinations are not available. For instance, there is no such thing (at least, not yet) as Visual C/C++ for Linux or MinGW targeting macOS.
In this particular blog post, we will use Azure Pipelines, a hosted build system supporting all three major OS: Windows, macOS, and Linux. For the DIY among you, the same setup can be achieved by using something like Jenkins, GitLab CI, TeamCity, and many other alternatives, along with the some build agents for the corresponding OS you want to tackle.
The build itself is configured via the YAML file,
azure-pipelines.yml. There is a job for each unique combination of (Architecture, Operating System, Compiler). For example,
amd64_linux_gcc denotes the build job for binary for Linux on Intel/AMD 64-bit architecture, compiled using GCC. As for now, the total number of those jobs is 16.
The obvious build job is something like this. It is running natively on the hosted agent of Azure Pipelines. We just need to make sure that the right compiler (GCC in this case) is installed. For Linux and macOS, this can be via the package manager, apt and Homebrew, respectively.
- job: 'amd64_linux_gcc' pool: vmImage: 'ubuntu-16.04' steps: - script: sudo apt install -y make gcc displayName: 'Install requirements' - script: gcc --version displayName: 'Verify tools version' - script: CC=gcc make displayName: 'make' - script: file ./hello displayName: 'Verify executable' - script: ./hello displayName: 'Run'
On Windows however, there is no need to do that since the hosted Windows agent is already equipped with Visual Studio. However, because the build is carried out with a Makefile (more specifically,
Makefile.win), we need GNU Make which is installed via Chocolatey. Note that a stage in the build job is verifying the executable (useful to know whether it is built correctly or not) using
file (Linux and macOS) or
For two special Windows compilers, Digital Mars and Pelles C (the Windows flavor of a modified TCC), they need to be installed on the fly since they are not available on the Windows hosted agents. Digital Mars is installed with a little dance with
unzip. Meanwhile, Pelles C is readily available from Chocolatey.
To target non-Intel CPU architectures, we need to use some cross compilers. Since hosted Linux agent of Azure Pipelines supports Docker, the easiest way to achieve this is to use a Docker-based cross compilation using dockcross. This is explained in-depth in my previous blog post, Cross Compiling with Docker. One of such example is the following build job, for building for Linux running on ARM (32-bit). Note that since the resulting exectable is an ARM binary, we ought to use QEMU to run it.
- job: 'armhf_linux_gcc' pool: vmImage: 'ubuntu-16.04' steps: - script: sudo apt install -y qemu-user displayName: 'Install requirements' - script: | git clone --depth 1 https://github.com/dockcross/dockcross.git cd dockcross docker run --rm dockcross/linux-armv7 > ./dockcross-linux-armv7 chmod +x ./dockcross-linux-armv7 displayName: 'Prepare Dockcross' - script: ./dockcross/dockcross-linux-armv7 bash -c '$CC --version' displayName: 'Verify tools version' - script: ./dockcross/dockcross-linux-armv7 make LDFLAGS=-static displayName: 'make' - script: file ./hello displayName: 'Verify executable' - script: qemu-arm ./hello displayName: 'Run'
The same approach using Docker and QEMU works well for other CPU architectures such as MIPS, ARM 64-bit, and in fact Intel x86. The last one is quite necessary, since the hosted agent of Azure Pipelines is running in 64-bit mode. Thus, we use this virtualization layer (QEMU) to verify the correct execution of 32-bit binary.
As an illustration, two examples for MingW are illustrated. The first, MinGW is installed on the Windows agent. This is self explanatory.
- job: 'amd64_windows_mingw' pool: vmImage: 'vs2017-win2016' variables: CC: 'gcc' steps: - script: choco install mingw --version 8.1.0 displayName: 'install MinGW-w64' - script: gcc --version displayName: 'Verify tools version' - script: make displayName: 'make' - script: file hello.exe displayName: 'Verify executable' - script: hello.exe displayName: 'Run'
For the second example, MinGW is used in a cross compilation fashion. Again, we use the Docker-based dockcross to achieve this. The compiler (GCC) runs inside the Docker container on the hosted Linux agent, however it produces a Windows executable. How do we run the resulting executable? QEMU is not suitable here (since we still need to install or run Windows, remember the host is Linux). But, we have WINE to the rescue!
- job: 'i386_windows_mingw_static' pool: vmImage: 'ubuntu-16.04' steps: - script: | git clone --depth 1 https://github.com/dockcross/dockcross.git cd dockcross docker run --rm dockcross/windows-static-x86 > ./dockcross-windows-static-x86 chmod +x ./dockcross-windows-static-x86 displayName: 'Prepare Dockcross' - script: ./dockcross/dockcross-windows-static-x86 bash -c '$CC --version' displayName: 'Verify tools version' - script: ./dockcross/dockcross-windows-static-x86 make displayName: 'make' - script: file ./hello displayName: 'Verify executable' - script: docker run -v $PWD:/app tianon/wine:32 bash -c "wine /app/hello" displayName: 'Run'
In fact, to avoid the hassle of on-the-fly installation/configuration of WINE, we just use the Dockerized WINE.
The whole ordeal of running 16 jobs will take anywhere from 5 minutes to 20 minutes. Obviously, if you are constrainted by the free tier of Azure Pipelines, you can purchase access to more hosted agents or attach your own build agents, which will definitely parallelize and speed things up.
I hope that the idea outlined in this post will inspire to continue to work on more cross-platform apps. Of course, it does not have to be an application written in ANSI C. The concept can be applied to D, Go, Rust, and many other modern compilers.