Developing cross-platform applications presents a major challenge:, how to ensure that every commit does not break some combinations of operating systems and CPU architectures. Fortunately, thanks an array of online services and open-source tools, this challenge becomes easier to tackle.
For this demo, I have the traditional Hello, world program written in ANSI C/C90 at this repository: github.com/ariya/hello-c90 (feel free to take a look). The objective is to verify its automatic build (for the purpose of continuous integration) for a number of different CPU architectures, operating systems, as well as the C/C++ compilers. Supported CPU architectures are (using Debian nomenclatures) are amd64, i386, i686, armhf, arm64, and mips. Among some C/C++ compilers to be tested are GCC, Clang, TCC, Visual C/C++ (as part of Visual Studio 2017 and also 2019), Pelles C, Digital Mars, as well as MinGW. Obviously, some combinations are not available. For instance, there is no such thing (at least, not yet) as Visual C/C++ for Linux or MinGW targeting macOS.
In this particular blog post, we will use Azure Pipelines, a hosted build system supporting all three major OS: Windows, macOS, and Linux. For the DIY among you, the same setup can be achieved by using something like Jenkins, GitLab CI, TeamCity, and many other alternatives, along with the some build agents for the corresponding OS you want to tackle.
The build itself is configured via the YAML file, azure-pipelines.yml. There is a job for each unique combination of (Architecture, Operating System, Compiler). For example, amd64_linux_gcc denotes the build job for binary for Linux on Intel/AMD 64-bit architecture, compiled using GCC. As for now, the total number of those jobs is 16.
The obvious build job is something like this. It is running natively on the hosted agent of Azure Pipelines. We just need to make sure that the right compiler (GCC in this case) is installed. For Linux and macOS, this can be via the package manager, apt and Homebrew, respectively.
- job: 'amd64_linux_gcc'
pool:
vmImage: 'ubuntu-16.04'
steps:
- script: sudo apt install -y make gcc
displayName: 'Install requirements'
- script: gcc --version
displayName: 'Verify tools version'
- script: CC=gcc make
displayName: 'make'
- script: file ./hello
displayName: 'Verify executable'
- script: ./hello
displayName: 'Run'
On Windows however, there is no need to do that since the hosted Windows agent is already equipped with Visual Studio. However, because the build is carried out with a Makefile (more specifically, Makefile.win), we need GNU Make which is installed via Chocolatey. Note that a stage in the build job is verifying the executable (useful to know whether it is built correctly or not) using file (Linux and macOS) or dumpbin (Windows).
For two special Windows compilers, Digital Mars and Pelles C (the Windows flavor of a modified TCC), they need to be installed on the fly since they are not available on the Windows hosted agents. Digital Mars is installed with a little dance with curl and unzip. Meanwhile, Pelles C is readily available from Chocolatey.
To target non-Intel CPU architectures, we need to use some cross compilers. Since hosted Linux agent of Azure Pipelines supports Docker, the easiest way to achieve this is to use a Docker-based cross compilation using dockcross. This is explained in-depth in my previous blog post, Cross Compiling with Docker. One of such example is the following build job, for building for Linux running on ARM (32-bit). Note that since the resulting exectable is an ARM binary, we ought to use QEMU to run it.
- job: 'armhf_linux_gcc'
pool:
vmImage: 'ubuntu-16.04'
steps:
- script: sudo apt install -y qemu-user
displayName: 'Install requirements'
- script: |
git clone --depth 1 https://github.com/dockcross/dockcross.git
cd dockcross
docker run --rm dockcross/linux-armv7 > ./dockcross-linux-armv7
chmod +x ./dockcross-linux-armv7
displayName: 'Prepare Dockcross'
- script: ./dockcross/dockcross-linux-armv7 bash -c '$CC --version'
displayName: 'Verify tools version'
- script: ./dockcross/dockcross-linux-armv7 make LDFLAGS=-static
displayName: 'make'
- script: file ./hello
displayName: 'Verify executable'
- script: qemu-arm ./hello
displayName: 'Run'
The same approach using Docker and QEMU works well for other CPU architectures such as MIPS, ARM 64-bit, and in fact Intel x86. The last one is quite necessary, since the hosted agent of Azure Pipelines is running in 64-bit mode. Thus, we use this virtualization layer (QEMU) to verify the correct execution of 32-bit binary.
As an illustration, two examples for MingW are illustrated. The first, MinGW is installed on the Windows agent. This is self explanatory.
- job: 'amd64_windows_mingw'
pool:
vmImage: 'vs2017-win2016'
variables:
CC: 'gcc'
steps:
- script: choco install mingw --version 8.1.0
displayName: 'install MinGW-w64'
- script: gcc --version
displayName: 'Verify tools version'
- script: make
displayName: 'make'
- script: file hello.exe
displayName: 'Verify executable'
- script: hello.exe
displayName: 'Run'
For the second example, MinGW is used in a cross compilation fashion. Again, we use the Docker-based dockcross to achieve this. The compiler (GCC) runs inside the Docker container on the hosted Linux agent, however it produces a Windows executable. How do we run the resulting executable? QEMU is not suitable here (since we still need to install or run Windows, remember the host is Linux). But, we have WINE to the rescue!
- job: 'i386_windows_mingw_static'
pool:
vmImage: 'ubuntu-16.04'
steps:
- script: |
git clone --depth 1 https://github.com/dockcross/dockcross.git
cd dockcross
docker run --rm dockcross/windows-static-x86 > ./dockcross-windows-static-x86
chmod +x ./dockcross-windows-static-x86
displayName: 'Prepare Dockcross'
- script: ./dockcross/dockcross-windows-static-x86 bash -c '$CC --version'
displayName: 'Verify tools version'
- script: ./dockcross/dockcross-windows-static-x86 make
displayName: 'make'
- script: file ./hello
displayName: 'Verify executable'
- script: docker run -v $PWD:/app tianon/wine:32 bash -c "wine /app/hello"
displayName: 'Run'
In fact, to avoid the hassle of on-the-fly installation/configuration of WINE, we just use the Dockerized WINE.
The whole ordeal of running 16 jobs will take anywhere from 5 minutes to 20 minutes. Obviously, if you are constrainted by the free tier of Azure Pipelines, you can purchase access to more hosted agents or attach your own build agents, which will definitely parallelize and speed things up.
I hope that the idea outlined in this post will inspire to continue to work on more cross-platform apps. Of course, it does not have to be an application written in ANSI C. The concept can be applied to D, Go, Rust, and many other modern compilers.