ariya.io

GTX 1080 Ti for Local LLM

Sat, 28 Feb 2026 15:33:54 -0800

Despite being over eight years old, the NVIDIA GTX 1080 Ti remains a compelling choice for enthusiasts keen on running LLM locally.

Initially launched in early 2017 with a $699 MSRP, this GTX 1080 Ti card quickly earned a “legendary GPU” reputation among tech reviewers and YouTubers. Today, it’s readily available on the second-hand market (particularly in Northern California or on eBay) for around $150, often even less if you’re lucky.

For this modest price, you acquire a card with 11 GB of VRAM, an unconventional yet highly practical configuration for modern LLMs. With quantized models, 11 GB typically offers ample space for model weights and the context windows crucial for RAG workflows, coding assistants, and other use cases. This makes it an ideal sweet spot for hobbyists: affordable enough for experimentation, yet powerful enough to handle practical workloads.

Please note that local LLM inference performance is primarily assessed by two metrics:

Prompt Processing: How quickly the LLM processes the input context.
Token Generation: How rapidly the LLM produces output (the “answer”).

These metrics, often denoted as pp512 and tg128 (where the numbers represent total tokens), can be measured using llama.cpp, a very popular inference engine. As mentioned in our previous article on running LLMs locally with LM Studio and Jan, llama.cpp is one of the engines powering these applications.

For LLM tasks requiring long context windows, such as RAG or coding assistant, pp512 performance is paramount. Slow pp512 leads to noticeable lag as large prompts take seconds to preprocess before any output appears. Meanwhile, for dynamic chats or creative writing, tg128 is more critical, as it dictates the fluidity of the model’s responses. llama.cpp with CUDA

To run benchmarks, you’ll first need a CUDA-enabled build of llama.cpp, which involves installing NVIDIA’s CUDA toolkit and ensuring your development tools are correctly set up. A quick sanity check involves verifying the versions of nvcc, CMake, and gcc:

nvcc --version
cmake --version
gcc --version

If all three commands return version numbers, you’re ready to proceed.

Next, clone the llama.cpp repository and build it with CUDA enabled:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

This compilation process can take some time, so be prepared for a short wait.

Once built, test your setup with a small model like Google’s Gemma-3 1B (approximately 800 MB). You can interact with it via the command line (llama-cli) or launch a lightweight web UI:

./build/bin/llama-server

Running nvtop concurrently will confirm that the GPU is actively utilized during inference.

With llama.cpp configured, benchmarking is straightforward using the included llama-bench tool. For consistent comparisons, Llama-2 7B Q4_0 quantization is a popular choice, being small enough to run comfortably and widely benchmarked.

Execute the following command:

./build/bin/llama-bench -ngl 100 -m llama-2-7b.Q4_0.gguf

The output will provide both pp512 and tg128. While these speeds may not match modern GPUs, they are more than enough for local experimentation. Crucially, the pp512 performance remains competitive enough to make retrieval-heavy workloads (like document-based question answering) viable. The GTX 1080 Ti may not be the fastest, but it offers an excellent entry point into local AI.

Community benchmarks in llama.cpp discussions, illustrate how this GPU compares:

CUDA (NVIDIA GPUs): https://github.com/ggml-org/llama.cpp/discussions/15013
Metal (Apple Silicon): https://github.com/ggml-org/llama.cpp/discussions/4167
ROCm (AMD GPUs): https://github.com/ggml-org/llama.cpp/discussions/15021
Vulkan (cross-platform): https://github.com/ggml-org/llama.cpp/discussions/10879

These threads are invaluable resources for real-world performance data, aiding decisions on building new local inference rigs or repurposing existing hardware.

A sample of these results:

Apple M4 Pro: pp512 = 439 tok/s and tg128 = 50 tok/s
GTX 1080 Ti: pp512 = 1084 tok/s and tg128 = 62 tok/s
RX 9060 XT: pp512 = 1478 tok/s and tg128 = 65 tok/s
RTX 2080 Ti: pp512 = 2890 tok/s and tg128 = 107 tok/s
RTX 3090: pp512 = 5174 tok/s and tg128 = 158 tok/s

The RTX 2080 Ti, the 1080 Ti’s younger sibling, also features 11 GB VRAM. It represents a solid upgrade for increased speed, often found for around $250 on eBay if you’re fortunate with a bid.

For accelerated LLMs that can run on a GTX 1080 Ti with 4-bit quantization and enough VRAM for context processing, popular choices include Qwen 3 8B, Llama 3.1 8B, Gemma 3 12B, and Granite 4.0 Tiny 7B. If speed is a higher priority, smaller variants (e.g., Qwen 3 4B) are also excellent candidates.

Ultimately, the GTX 1080 Ti’s most significant advantage is its cost. At approximately $150 on the used market, it leaves considerable budget for the rest of your system. A complete, cost-effective build can come in under $500, with an example shown in the accompanying photo. We’ll delve deeper into this specific rig and its capabilities in upcoming newsletter articles, so stay tuned!

Note: This article originally appeared on the Remote Browser Substack.

Not Everything is an Agent

Mon, 31 Mar 2025 22:47:17 -0800

“Agent” is likely going to be the word that will cause existential dread to true LLM enthusiasts.

Everyone’s got a different idea of what it means. In our modern age of innovation theater, lots of organizations gleefully slap the “agentic” label on anything that vaguely resembles a regular program (and pocket tons of money). Even a simple HTTP call to an LLM-as-a-Service can be called an agent, if you try desperately hard enough.

The internet, as always, is flooded with “groundbreaking” tutorials on building these so-called agents. Often authored by the latest hypefluencers, they typically involve a few lines (probably generated by whatever coding assistant is currently trending on Hacker News) that compose LangChain and an Ollama instance, often being presented as the pinnacle of AI autonomy. Because why bother with actual innovation when you can just repeat the quasi-boilerplate code ad nauseam?

That’s why I liked it a lot when the Anthropic article, Building effective agents, came out, as it dares to suggest that simply bolting on retrieval or memory to an LLM does not, in fact, make an agent. And chaining or routing? That’s just glorified control flow, folks. Only when an LLM is tasked with truly complex, real-world tasks such as coding or using a computer, does it begin to resemble the autonomous agent we’ve been promised

So how do you identify a real agent? Don’t be fooled by the grand pronouncements of those rearranging deck chairs on the Titanic. Ask for the receipts of successful evaluations! Anecdotal evidence of a few successful LLM calls isn’t that useful. Remember, in the world of LLMs, as in life, the loudest claims are often the emptiest!

Afterburner and Power Limit

Fri, 28 Feb 2025 21:37:19 -0800

Ever witnessed a fighter jet spewing hot flames as it kicks into afterburner? In that moment, efficiency is deliberately sacrificed for maximum acceleration.

In the midst of combat, efficiency means nothing when your life is on the line. The jet engine must keep roaring, before the pilot gets taken down by the enemy (and potentially meets their maker).

A GPU faces a similar fate. When pushed to consume hundreds of watts to churn out LLM tokens at the user’s breakneck speed, there’s no choice but to run as fast as possible, even if sweat is pouring and muscle fatigue reaches its peak.

Fortunately, nvidia-smi, with its pl (power limit) option, can be used to set an upper limit on power consumption, so the GPU doesn’t go completely overboard. Those last few dozen watts often don’t make a significant difference in performance, but they definitely contribute to heat generation, which needs to be monitored.

From the graph (measured with llama-bench, for the Mistral-7B-Instruct model, Q4), it’s evident that pushing the power further doesn’t lead to increased LLM speed. 250, 300, or even 350 watts, it’s more or less the same. Meanwhile, dropping to 200 watts does slightly decrease the speed, but it’s very worthwhile considering the power consumption is reduced by a third.

Saving energy is always a wise choice!

Privacy-Preserving Personal Search Appliance

Thu, 30 Jan 2025 20:27:09 -0800

This is powered by SearXNG, an excellent open-source meta search engine.

Unlike traditional search engines like Google or Bing, SearXNG doesn’t crawl the web and index content. Instead, it leverages other search engines like DuckDuckGo, Qwant, and Mojeek to fetch results while protecting your privacy. This means your personal information isn’t tracked by those upstream services.

With the rise of LLMs and RAG, SearXNG has gained even more popularity. But I’ll dive into that in a future post.

Setting up SearXNG is a breeze. You can use Docker or Podman (my favorite Docker replacement, everyone should use it!) to get it running quickly. In fact, I encourage you to try it on your main machine. You’ll be surprised how easy it is!

A little fun fact about the name. SearXNG is an active fork of SearX. As typically the case, NG usually stands for “next generation”. However, the X in SearX is actually the Greek letter chi, which is often transliterated as “ch”. So, you could think of SearXNG as “searching”.

While you can run SearXNG on your main machine, a dedicated device offers several advantages. You can share it with your family or colleagues and add extra security layers like Tailscale, Wireguard, or the good old OpenVPN.

For my little box, I chose a used Shuttle DH170 with an Intel Core i3 6100 (2 cores, 4 threads) and 16GB of RAM. This might seem like overkill, but it’s more than enough for SearXNG. The 200GB SSD is also plenty of storage. The total cost of the hardware was just $70. I could have saved more by using less RAM and storage, but I had these components on hand.

In terms of power consumption, the appliance idles at around 10W. I haven’t optimized it yet using tools like PowerTOP, but even so, it’s quite efficient. The off-the-shelf x86 architecture offers excellent upgrade potential. I could easily swap in a more powerful CPU like an Intel Core i7 6700 (4 cores, 8 threads) or add more RAM and storage if needed.

Initially, I considered using Proxmox to manage the system and run SearXNG in a container. However, I found this to be too complex. Instead, I opted for a simpler approach using vanilla Debian and Podman.Then, I remembered that there is project, CasaOS, a user-friendly home server OS. It offers a web-based interface for remote management and can easily run SearXNG. If you’re new to home servers, CasaOS is a great way to get started.

For those who prefer a more resource-constrained solution, you could use a Raspberry Pi or similar device. CasaOS also works well on ARM-based systems.

In today’s digital age, privacy is a fundamental right. Unfortunately, our digital footprints are being exploited, and our personal data is being harvested. Rampant privacy violations are becoming the norm. Let’s take proactive steps to protect ourselves and our loved ones!

LLM Inference Machine for $300

Fri, 27 Dec 2024 20:17:14 -0800

You can absolutely run Qwen-2.5 32B. And of course, Llama-3.1 8B and Llama-3.2 Vision 11B are no problem at all.

Now, before you get too excited, there’s a catch: this rig won’t break any speed records (more on that later). But if you’re after a budget-friendly way to do LLM research, this build might be just what you need.

Here’s a breakdown of the parts and the amazing prices I got them for:

AMD Ryzen 5 3400G: $50
Gigabyte X570 motherboard: $30
16 GB DDR4-3200 RAM: $30
512 GB SSD: $20
NVIDIA Tesla M40: $100
Cooler for M40: $30
EVGA 750W PSU: $20
Silverstone HTPC case: $20

The motherboard was a crazy good find, a broken PCIe latch got me a killer deal. The Ryzen 3400G is outdated by today’s standards, with only 4 cores and 8 threads, but for a GPU-focused inference rig, it’s more than enough. Bonus: its Vega iGPU frees up the PCIe slot for the real star of the show, the M40 GPU.

Speaking of the GPU, it’s a Maxwell-era data center card with a massive 24GB of VRAM. That much memory is essential for running hefty 32B models (quantized, of course).

While you can find a used M40 on eBay for around $90 these days, I had to buy an additional cooling solution (two small fans in a 3D-printed shroud), since data center GPUs usually don’t come with coolers or blowers like their consumer counterparts.

Here are the token generation speeds for several instruction-tuned models, quantized to 4-bit (Q4_K_M), measured with llama-bench:

Phi-3.5 Mini: 47 tok/s
Mistral 7B: 30 tok/s
Llama-3.1 8B: 28 tok/s
Mistral Nemo 12B: 19 tok/s
Qwen-2.5 Coder 32B: 7 tok/s

Performance is all relative. Compared to the latest RTX 3000 series, the M40 is definitely the slower sibling: about 5x slower, to be exact. But then again, an RTX 3090 is roughly 10x more expensive. Meanwhile, a more affordable RTX 3080 might limit your options with its 10GB (or 12GB for the enthusiast version) of VRAM.

An RTX 2080 Ti with 11GB VRAM could be a nice upgrade. Prices in the used market are dropping ($250 or less at the time of writing), and it delivers a solid 3x speed boost compared to the M40. Double the cost for triple the speed? That’s a pretty sweet deal!

How about Apple Silicon? The M2 Pro with its Metal GPU is roughly 25% faster than the M40. It wins easily in areas like portability, efficiency, and noise levels, but it comes with a significantly higher cost.

Coding assistance is a proven home-run use case for powerful LLMs. This is where the M40’s massive 24GB VRAM shines, enabling you to run the fantastic Qwen-2.5 Coder 32B model. Pair it with Continue.dev as your coding assistant, and you’ve got a powerful combo that could replace tools like GitHub Copilot or Codeium, particularly for medium-complexity projects.

The best part? Privacy and data security. With local LLM inference, your precious source code stays on your machine.

Now, should I go all in? Is it time to add a second M40?

Deploying an Uberjar to Dokku

Thu, 23 Feb 2023 11:37:36 -0800

Dokku is a self-hosted Platform-as-a-Service (PaaS) that offers a compelling alternative to popular PaaS solutions like Heroku. With built-in support for Linux containers, deploying an application on Dokku is straightforward. However, there is a lesser-known deployment method that involves sending a build artifact, such as a JAR package for Java apps, directly to Dokku.

This deployment method is useful when there is a need to quickly and frequently deploy the latest version of a custom application. By skipping the process of creating a container image, developers can focus on building the artifact for local development. This approach can be applied to packaged applications built with various programming languages, including Python, Java, JavaScript, PHP, etc.

To follow this process, it is necessary to have the packaged Java application in the form of an Uberjar, i.e. a JAR archive that contains all dependencies and can be executed by the JVM without requiring additional packages at runtime. The process assumes that Dokku has been installed on a machine named dokku.homelab.lan, and the dokku command is working properly:

$ dokku version
dokku version 0.29.4

Also, due to some heavy building that is going to happen on that Dokku machine, make sure there is an ample free capacity on the disk, ideally 8 GB or more (depending on the application).

If we are deploying an Uberjar, obviously that Uberjar needs to exist first. For this example, I am using an Uberjar from the open-source edition of Metabase (adjust things to suit your need). Note that Metabase is written in Clojure, not Java, though it runs on JVM. In theory, any other JVM languages (e.g. Kotlin, Scala, etc) can work as well.

$ curl -OL https://downloads.metabase.com/v0.45.2/metabase.jar
$ file ./metabase.jar 
./metabase.jar: Zip archive data, at least v1.0 to extract, compression method=store

Two auxiliary files, a Procfile and a Dockerfile, are required. The Procfile contains a single line of code that specifies how the application is executed, while the Dockerfile details the construction of the container.

The first file, Procfile, is this one-liner:

web: java -jar metabase.jar

The second file, Dockerfile, is not too strange to those who are familiar with Docker:

FROM eclipse-temurin:17
WORKDIR /app
COPY . ./ 
RUN java -version
RUN ls -l /app 
EXPOSE 3000

The base image is set to the latest Long-Term Support (LTS) version of OpenJDK v17 using the Eclipse Temurin distribution from Adoptium. The EXPOSE line indicates the port Metabase uses, which is port 3000. The two optional RUN lines are useful for debugging or resolving any issues that may arise.

Next, we package the necessary files into a tarball by executing the following commands:

$ tar cvf package.tar Procfile Dockerfile metabase.jar
$ file ./package.tar 
./package.tar: POSIX tar archive (GNU)

Before sending the tarball to Dokku, we must create an application. This is achieved by executing the following commands:

$ dokku apps:create metabase
-----> Creating metabase...
-----> Creating new app virtual host file...
$ dokku proxy:ports-set metabase http:80:3000
dokku proxy:ports-set metabase http:80:3000
-----> Setting config vars
       DOKKU_PROXY_PORT_MAP:  http:80:3000

The last command maps the host’s port 80 to the container’s exposed port 3000. And now, the fun starts!

$ cat package.tar | dokku git:from-archive metabase --
-----> Fetching tar file from stdin
-----> Generating build context
       Striping 0 worth of directories from tarball
       Moving unarchived files and folders into place
-----> Updating git repository with specified build context
-----> Cleaning up...
-----> Building metabase from Dockerfile
-----> Setting config vars
       DOKKU_DOCKERFILE_PORTS:  3000
Sending build context to Docker daemon  271.7MB
Step 1/12 : FROM eclipse-temurin:17
Digest: sha256:f6562feb32844d0059616d6e54c6cc3127ccf77fb594ccb98cc4279ca15887ed
Status: Downloaded newer image for eclipse-temurin:17
 ---> 1e117025f42d
Step 2/12 : WORKDIR /app
 ---> Running in 89d26eed69f3
 ---> db157924a857
Step 3/12 : COPY . ./
 ---> 59e836261c66
Step 4/12 : RUN java -version
 ---> Running in 21df4266e534
openjdk version "17.0.6" 2023-01-17
OpenJDK Runtime Environment Temurin-17.0.6+10 (build 17.0.6+10)
OpenJDK 64-Bit Server VM Temurin-17.0.6+10 (build 17.0.6+10, mixed mode, sharing)
 ---> 16d451db8f1a
Step 5/12 : RUN ls -l /app
 ---> Running in 829a2df4f10e
total 265328
-rw-r--r-- 1 root root        92 Jan 31 02:57 Dockerfile
-rw-r--r-- 1 root root 271686194 Jan 31 02:57 metabase.jar
-rw-r--r-- 1 root root        28 Jan 31 02:57 Procfile
 ---> 67b7b1179da7
Step 6/12 : EXPOSE 3000
Step 7/12 : LABEL com.dokku.app-name=metabase
Step 8/12 : LABEL com.dokku.builder-type=dockerfile
Step 9/12 : LABEL com.dokku.image-stage=build
Step 10/12 : LABEL dokku=
Step 11/12 : LABEL org.label-schema.schema-version=1.0
Step 12/12 : LABEL org.label-schema.vendor=dokku
Successfully built a246db231b6f
Successfully tagged dokku/metabase:latest
-----> Releasing metabase...
-----> Checking for predeploy task
       No predeploy task found, skipping
-----> Checking for release task
       No release task found, skipping
-----> Checking for first deploy postdeploy task
       No first deploy postdeploy task found, skipping
-----> Deploying metabase via the docker-local scheduler...
-----> Configuring metabase.dokku.homelab.lan...(using built-in template)
-----> Creating http nginx.conf
       Reloading nginx
=====> Application deployed:
       http://metabase.dokku.homelab.lan

The log may appear lengthy, but the steps it displays should be straightforward. On the Dokku target machine, a container image is constructed using the information in the tarball. From that image, a container is created and deployed using the standard Dokku machinery. If all goes well, the application (Metabase in this instance) will be up and running at the specified hostname.

As you become more familiar with this method, sending build artifacts to Dokku after each change will become a natural part of your workflow!

Continuous Integration for React Native Apps with GitHub Actions

Tue, 29 Dec 2020 18:03:18 -0800

For React Native mobile apps targeting Android and iOS, an easy way to setup its continuous integration is to take advantage of Actions, an automation workflow service provided by GitHub. Even better, for open-source projects, GitHub Action offers unlimited free running minutes (at the time of this writing).

The advantage of React Native is a single code-base targeting two major mobile platforms, iOS and Android. However, care must be taken so that when one developer focuses on implementing features on fixing defects on Android, whatever they check in into the code will not break iOS and vice versa. Ideally, that developer should always check and verify for both platforms. But mistakes happen and the best way to catch them is to ensure that the corresponding continuous integration (CI) is running smoothly to catch those potential problems early on.

Thanks to GitHub Actions supporting running the workflow on macOS and Linux (also actually Windows, but that is not too relevant for this purpose), creating a CI for React Native is easy enough. To follow along, check the sample project (in the style of Hello world) that I have created at github.com/ariya/hello-react-native.

Let us start with the Android build since it is the easiest. Create a file with the name android.yml under the directory .github/workflows. The content should be like this:

name: Android

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-20.04
    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js v12
      uses: actions/setup-node@v1
      with:
        node-version: 12.x

    - run: npm ci

    - run: ./gradlew assembleDebug -Dorg.gradle.logging.level=info
      working-directory: android
      name: Build Android apk (debug)

The above YAML declares that this workflow must be executed for every pull request and also once it is merged, as well as when a commit is pushed into the source repo. The workflow runs on a Ubuntu 20.04 machine which is, thanks to GitHub, already equipped with some development packages, including Java, Android SDK, and many other bits and pieces necessary for Android development. The first step is to check out the code (obvious) followed by another step to pick the Node.js version (12 in this case, feel free to adjust it to your project). The step npm ci will install all the dependencies. The next step after that is invoking Gradle to build the app, just as it is being done for a local development machine.

Once this file is ready, commit it to the repo, push the branch, and voila! GitHub will start to execute that build process for any future branch push and also for all pull requests (for this simple demo project, the build process takes about 3 minutes or less, not bad at all!). If the pull request does not break the Android build, we will see the usual green checkmark, as illustrated below. Of course, if the build breaks, the failure will be displayed and we can track the build log to find out what has gone wrong (this helps to accelerate the troubleshooting).

For completeness, we can also have the build artifact, the APK file generated by Gradle, archived for every workflow run. To do that, add the following lines:

    - uses: actions/upload-artifact@v2
      with:
        name: android-apk
        path: '**/*.apk'

Clicking on the green mark icon on the commit view will lead to the detailed result of Action workflows for that particular commit. We can also find the link to the archived artifact, in this case the APK files. Since the artifacts are retained for some time (based on the project settings, default to 30 days), this can be very handy when we want to troubleshoot a problem. Let us say a certain feature does not work anymore with today’s build but we are confident that the same feature still worked with the build from last week. Rather than checkout out different revisions and rebuild the app, we can just grab the APK files. Since these are built in debug mode, we can comfortably launch it in an emulator and debug it just like an APK that we build on a local development environment.

How about building for iOS? It is not exactly the same but it follow the same principles. Here is a minimalistic workflow file, ios.yml, as a starting point:

name: iOS
on: [push, pull_request]
jobs:
  build:
    runs-on: macos-latest
    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js
      uses: actions/setup-node@v1
      with:
        node-version: 14.x
    - run: npm ci
    - run: xcode-select -p
    - run: pod install
      working-directory: ios
      name: Install pod dependencies
    - name: Build iOS (debug)
      run: "xcodebuild \
        -workspace ios/HelloReactNative.xcworkspace \
        -scheme HelloReactNative \
        clean archive \
        -sdk iphoneos \
        -configuration Debug \
        -UseModernBuildSystem=NO \
        -archivePath $PWD/HelloReactNative \
        CODE_SIGNING_ALLOWED=NO"

The first few lines are just like the Android workflow. The major difference here is that the workflow needs to run on a macOS machine, as iOS SDK isn’t available on neither Linux nor Windows. The build steps follow a similar pattern: check out the code, setup Node.js, and install dependencies. There are two extra steps. The first one is to run xcode-select -p to ensure the readiness of the correct Xcode and its related tool. The second one, pod install, is used to install any dependencies for CocoaPods, assuming that the project is using CocoaPods to manage iOS-specific dependencies (usually it is). After that, we invoke the command-line debug build with Xcode, just like what we would do on a local machine. Since building for iOS is a bit more complicated, for this simple demo project, it will run for around 10 minutes, give or take.

Note that the above YAML files cover the build for iOS and Android. Please do not forget to create another workflow file that runs the tests, typically with Jest, to catch potential regression on the unit tests and/or integration tests. Often times, this workflow is also the best place to run various static and dynamic code analyzers (linters, code formatter, security scanners, etc).

Armed with three workflow YAML files, we fully established a simple and yet powerful continuous integration for React Native apps. Happy developing!

On GitHub Actions with MSYS2

Fri, 31 Jul 2020 20:33:31 -0700

Thanks to the complete GitHub Actions for MSYS2, it is easier than ever to construct a continuous integration setup for building with compilers and toolchains which can run on MSYS2.

The details are available on the official page, github.com/marketplace/actions/setup-msys2. However, perhaps it is best illustrated with a simple but concrete example. As usual, for this illustration, you will see the use of this simplistic Hello, world program in ANSI C. To follow along, check out its repository at github.com/ariya/hello-c90.

Let us take a look at this workflow setup to build this C program with GCC on MSYS2 (on Windows, obviously):

name: amd64_windows_gcc
jobs:
  amd64_windows_gcc:
    runs-on: windows-2019
    defaults:
      run:
        shell: msys2 {0}
    steps:
    - uses: actions/checkout@v2
    - uses: msys2/setup-msys2@v2
      with:
        install: gcc make
    - run: gcc -v
    - run: make CC=gcc
    - run: file ./hello.exe
    - run: ./hello

The important lines are for the setup-msys2 section. The install value allows an easy selection of various packages which shall be installed before proceeding to the next step. For this purpose, it is sufficient to install gcc and make, but YMMV.

The rest is self-explanatory. Please note also the defaults section earlier, this is convenient to set the default shell so that we do not need to explicitly call it for every single run thereafter.

Now let us come up with another variant, this time for Clang instead of GCC (read also my previous post: Clang for Windows)

name: amd64_windows_clang
jobs:
  amd64_windows_clang:
    runs-on: windows-2019
    defaults:
      run:
        shell: msys2 {0}
    steps:
    - uses: actions/checkout@v2
    - uses: msys2/setup-msys2@v2
      with:
        install: make mingw-w64-x86_64-clang
    - run: clang --version
    - run: make CC=clang
    - run: file ./hello.exe
    - run: ./hello

Pretty straightforward, isn’t it? We just change the package to be installed and the compiler to be used. Since the two YAML files are very similar, to avoid a lot of repeated steps, we can parametrize it as follows. This is basically taking advantage of the matrix strategy feature of GitHub Actions.

  amd64_windows:
    runs-on: windows-2019
    strategy:
      matrix:
        compiler: [gcc, clang]
    defaults:
      run:
        shell: msys2 {0}
    steps:
    - uses: actions/checkout@v2
    - uses: msys2/setup-msys2@v2
    - run: pacman --noconfirm -S make gcc
      if: ${{ matrix.compiler == 'gcc' }}
    - run: pacman --noconfirm -S make mingw-w64-x86_64-clang
      if: ${{ matrix.compiler == 'clang' }}
    - run: ${{ matrix.compiler }} --version
    - run: make CC=${{ matrix.compiler }}
    - run: file ./hello.exe
    - run: ./hello

To take it one step further, we can also support both i686 and AMD64 platforms in the same YML, again by parametrizing the architecture. Here is how it looks:

name: windows
jobs:
  windows:
    runs-on: windows-2019
    strategy:
      matrix:
        compiler: [gcc, clang]
        msystem: [MINGW32, MINGW64]
    defaults:
      run:
        shell: msys2 {0}
    steps:
    - uses: actions/checkout@v2
    - uses: msys2/setup-msys2@v2
      with:
        msystem: ${{ matrix.msystem }}
        install: make
    - run: pacman --noconfirm -S gcc
      if: ${{ matrix.compiler == 'gcc' }}
    - run: pacman --noconfirm -S mingw-w64-x86_64-clang
      if: ${{ (matrix.msystem == 'MINGW64') && (matrix.compiler == 'clang') }}
    - run: pacman --noconfirm -S mingw-w64-i686-clang
      if: ${{ (matrix.msystem == 'MINGW32') && (matrix.compiler == 'clang') }}
    - run: ${{ matrix.compiler }} --version
    - run: make CC=${{ matrix.compiler }}
    - run: file ./hello.exe
    - run: ./hello

That’s all 4 combinations, 32-bit and 64-bit, for each GCC and Clang, in a simple configuration!

Cross-compiling with musl Toolchains

Mon, 22 Jun 2020 05:37:59 -0700

When working on command-line utilities which can be useful for various platforms, from Windows on x86 to Linux on MIPS, the existence of a cross-compilation is highly attractive. A number of different binaries can be constructed conveniently from a single, typically powerful host system.

Alpine Linux popularizes the use of musl a no-frills C standard library for Linux. According to its website:

musl is lightweight, fast, simple, free, and strives to be correct in the sense of standards-conformance and safety.

In addition, thanks to Zach van Rijn, we have a collection of static toolchains based on musl at musl.cc at our disposal. The number of supported systems is rather mind blowing, you got everything from the usual i686 to MIPS to Microblaze and many others. https://github.com/ariya/fastlz/actions

As I search for a viable alternative to the cross-compilation method based on Dockcross (see my previous blog post: Cross Compiling with Docker on WSL 2), musl.cc fits the requirements nicely. I am in the process of migrating the continuous integration of FastLZ, my implementation of byte-aligned LZ77 compression algorithm, to be completely based on musl.cc.

Here is a quick walkthrough. As long as you are on Linux x86-64, you can follow along easily (and yes, this also works great on WSL, Windows Subsystems for Linux). As a reference, we will use the simplest ANSI C/C90 program available at github.com/ariya/hello-c90.

First and foremost, we need QEMU so we can test our binaries not native to x86-64. For convenience, GNU Make is also necessary.

$ sudo apt install -y qemu-users make

After that, let us grab the Hello C90 program:

$ git clone https://github.com/ariya/hello-c90.git
$ cd hello-c90

For a start, let us try to produce MIPS64 binary of our little Hello C90 program. Thus, we ought to grab the toolchains first, weighing at about 90 MB.

$ curl -O https://musl.cc/mips64-linux-musl-cross.tgz
$ tar xzf mips64-linux-musl-cross.tgz

To ensure that this fresh cross-compiler works, do a quick sanity check:

$ ./mips64-linux-musl-cross/bin/mips64-linux-musl-gcc --version
mips64-linux-musl-gcc (GCC) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

This looks good! Now we can compile our Hello C90 program statically:

$ make CC=./mips64-linux-musl-cross/bin/mips64-linux-musl-gcc LDFLAGS=-static
./mips64-linux-musl-cross/bin/mips64-linux-musl-gcc -O -Wall -std=c90 -c hello.c
./mips64-linux-musl-cross/bin/mips64-linux-musl-gcc -static -o hello hello.o

Checking the resulting binary should give the following:

$ file ./hello
./hello: ELF 64-bit MSB executable, MIPS, MIPS-III version 1 (SYSV), statically linked, not stripped

It is exactly what we want! To run the executable:

$ qemu-mips64 ./hello
Hello, world! From C90 with love...

Now, if you are doing this on WSL, or generally have a Windows machine available elsewhere, there is this fun activity of cross-compiling the above app for Windows, without the need for any Windows compiler and SDK. Same steps as before:

$ curl -O https://musl.cc/x86_64-w64-mingw32-cross.tgz
$ tar xzf x86_64-w64-mingw32-cross.tgz
$ make CC=./x86_64-w64-mingw32-cross/bin/x86_64-w64-mingw32-gcc LDFLAGS=-static
$ file ./hello.exe
./hello.exe: PE32+ executable (console) x86-64, for MS Windows

To really test it, just bring hello.exe to Windows and it is going to run as expected.

For more details and elaborated examples, check the collection of workflows YAML files of this Hello C program.

Combined with the continuous integration system of your choice, whether it is DIY via Jenkins or using one of the many services out there (GitHub Actions, Azure Pipelines, Travis CI), creating binaries for various operating systems becomes easier than ever!

Nix Package Manager on Ubuntu or Debian

Sat, 30 May 2020 20:11:45 -0700

Even though Ubuntu/Debian is equipped with its legendary powerful package manager, dpkg, in some cases, it is still beneficial to take advantage of Nix, a purely functional package manager.

The complete manual of Nix does a fantastic job on explaining how to install and use it. But for the impatients among you, here is a quick overview. Note that this also works well when using Ubuntu/Debian under WSL (Windows Subsystem for Linux, both the original and the newest WSL 2.

First, create the /nix directory owned by you (this is the common single-user installation):

$ sudo mkdir /nix
$ sudo chown ariya /nix

And then, run the installation script:

$ sh <(curl -L https://nixos.org/nix/install) --no-daemon

Note that if you use WSL 1, likely you will encounter some error such as:

SQLite database '/nix/var/nix/db/db.sqlite' is busy

This a known issue, the workaround is to create a new file ~/.config/nix/nix.conf with the following content

sandbox = false
use-sqlite-wal = false

and repeat the previous step.

If nothing goes wrong, the script will perform the installation. Grab a cup of tea while waiting for it!

downloading Nix 2.3.4 binary tarball for x86_64-linux
performing a single-user installation of Nix...
copying Nix to /nix/store......................................
replacing old 'nix-2.3.4'
installing 'nix-2.3.4'
unpacking channels...

Note that the last step (unpacking channels) can run for a very long time (no idea why, hope it will be fixed at some point). Just be patient.

To check whether Nix is successfully installed, we use the Hello, world tradition:

$ nix-env -i hello
installing 'hello-2.10'
these paths will be fetched (6.62 MiB download, 31.61 MiB unpacked):
/nix/store/9l6d9k9f0i9pnkfjkvsm7xicpzn4cv2c-libidn2-2.3.0
/nix/store/df15mgn0zsm6za1bkrbjd7ax1f75ycgf-hello-2.10
/nix/store/nwsn18fysga1n5s0bj4jp4wfwvlbx8b1-glibc-2.30
/nix/store/pgj5vsdly7n4rc8jax3x3sill06l44qp-libunistring-0.9.10
$ which hello
/home/ariya/.nix-profile/bin/hello
$ hello
Hello, world!

In the above illustration, hello is a test package that does nothing but to display the famous message. It looks simple, and yet it is very useful!

To get the feeling of packages available at your disposal (almost 29 thousands of them):

$ nix-env -qa  > nix-packages.list
$ wc -l nix-packages.list
28974 nix-packages.list
$ less nix-package.list

While it is not a substitute for a large collection of existing Debian/Ubuntu packages, very often what you get from Nix is more up-to-date. For instance, if you are stuck with a typical Ubuntu 18.04 LTS, it offers git 2.17.1, tmux 2.6.3, jq 1.5, curl 7.58, and Neovim 0.2.2. But, with Nix on that same Ubuntu system, at the time of this writing, you can enjoy git 2.26.2, tmux 3.1b, jq 1.6, curl 7.69, and and Neovim 0.4.3.

The way I use Nix however is not merely as a mechanism to get fresher software and other utilities. Rather, the functional nature of Nix leads to the possibility of multiple working environment, with each distinctive set of applications and tools, and the ability of switching cleanly between them. Those who use nvm (for Node.js) or virtualenv (for Python) probably can appreciate this. Now, imagine nvm/virtualenv but not only for Node.js/Python, and rather applied to an arbitrary set of packages. I have covered this in details before in my previous blog post, Isolated Development Environment using Nix. That blog post talked about Nix on macOS but obviously the experience is very suited for Nix on Debian, Ubuntu, or any other Linux distributions for that matter.

I hope this will inspire you to explore Nix in depth!

Practical Testing of Firebase Projects

Wed, 29 Apr 2020 12:10:04 -0700

Your little Firebase project is getting bigger every day? Never underestimate the need to establish a solid and firm integration tests from the get go.

Once you start to utilize various features of Firebase, from Hosting, Functions, and Firestore, it is imperative to incorporate practical local testing as soon as possible. Not only it will save you from some potential nightmares down the road, it can also facilitate faster iterations and quick(er) turn-around time during refactoring and feature implementation. Here is a few random suggestions to get you started. To follow along, you can also check the git repository containing the sample code at github.com/ariya/hello-firebase-experiment.

First thing that you always need to do is to implement a health check functionality. The name could be as simple as ping. Hence, inside your main Firebase Functions, there should be a block of code that looks like:

exports.ping = functions.https.onRequest((request, response) => {
    response.send('OK');
});

Now if you want to be fancy, it does not hurt to show the timestamp (Unix epoch) which can be valuable to know that this is not a cached or outdated HTTP response. If you wish, feel free to extend it with useful tidbits (but be careful not to reveal sensitive information).

exports.ping = functions.https.onRequest((request, response) => {
    response.send(`OK ${Date.now()}`);
});

In your test code (shown here with Axios to perform an HTTP request, but the concept applies to any library), do a quick sanity check that this /ping is working. This is an important step towards a reliable local testing.

it('should have a working ping function', async function () {
    const res = await axios.get('http://localhost:5000/ping');
    const status = res.data.substr(0, 2);
    const timestamp = res.data.substr(3);
    expect(status).toEqual('OK');
    expect(timestamp).toMatch(/[0-9]+/);
});

Now, the test might fail miserably. If that is the case, you do not have the proper setup yet to use and run Firebase emulators. Using npm, make sure to install all the following packages:

firebase-tools
firebase-functions
firebase-functions-test
firebase-admin
@google-cloud/firestore

And check that your firebase.json looks like the following:

{
  "hosting": {
    "public": "./public"
    "rewrites": [
      {
        "source": "/ping",
        "function": "ping"
      }
    ]
  },
  "emulators": {
    "functions": {
      "port": 5001
    },
    "firestore": {
      "port": 8080
    },
    "hosting": {
      "port": 5000
    }
}

Note the rewrites section. This makes /ping handily available from the main Firebase Hosting domain, instead of the long and cryptic one such as us-central1-YOURFIREBASEPROJECT.cloudfunctions.net/ping.

Before running tests, make sure to launch the emulators for Functions, Firestore, and Hosting:

npm run firebase -- emulators:start --project MYPROJECT

In the above command, npm run firebase works because of the run script definition. Also, substitute the name of your Firebase project accordingly. If the setup is correct, your terminal should show something like:

emulators: Starting emulators: functions, hosting
hub: emulator hub started at http://localhost:4400
functions: functions emulator started at http://localhost:5001
hosting: Serving hosting files from: ./
hosting: Local server: http://localhost:5000
hosting: hosting emulator started at http://localhost:5000
functions[ping]: http function initialized
emulators: All emulators started, it is now safe to connect.

At this point, if you point your browser to localhost:5000/ping, you should get the OK message (followed by the number representing the timestamp as Unix epoch). Of course, running the full tests (npm test) should also yield in a successful run.

When setting up the tests for CI (continuous integration), it might be easier to let the emulators run the test automatically. Here is how it is done:

npm run firebase -- emulators:exec "npm test" --project MYPROJECT

The exec option run the subsequent command, in this case the usual npm test, after starting the emulators. Once the command is completed (whether successfully or not), the emulators are automatically terminated. This is perfect for the CI run!

Next trick on our sleeve: fixtures for Firestore. Let us assume that your application uses this NoSQL datastore via this simple function for illustration (and do not forget to add a new URL rewrite for /answer/):

admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
exports.answer = functions.https.onRequest(async (request, response) => {
    try {
        const doc = await db.collection('universe').doc('answer').get();
        const value = doc.data().value;
        console.log(`Answer is ${value}`);
        response.send(`Answer is ${value}`);
    } catch (err) {
        console.error(`Failed to obtain the answer: ${err.toString()}`);
        response.send(`EXCEPTION: ${err.toString()}`);
    }
});

And the corresponding test:

it('should give a proper answer', async function () {
    const res = await axios.get('http://localhost:5000/answer');
    const answer = res.data;
    expect(answer).toEqual('Answer is 42');
});

Launching the emulators (using the previous instructions) and running the tests however will result in a failure. And if you go to localhost:5000/answer, you fill discover an expected response:

EXCEPTION: TypeError: Cannot read property 'value' of undefined

This should not come as a surprise. When Firebase Emulators launched, its database (for Firestore) is empty. Hence, there is still no proper document, let alone a collection. It will be unnecessarily tedious to populate the database (it works for this simple example but a real-world app might have tons of collections and documents). How do we prepare a fixture for this?

Well, again the Firestore emulators to the rescue! While it is running, and you can perform another steps to populate the database (outside the scope of this blog post, perhaps we will discuss in some other time), you can snapshot the database and save it as the test fixture:

npm run firebase -- emulator:export spec/fixture --project MYPROJECT

Once the fixture is available, rerun the emulator (either as start or through exec) with the import option and the Firestore database will not be empty anymore, as it is populated with the previous snapshot.

npm run firebase -- emulators:start --import spec/fixture --project MYPROJECT

Last but not least, let us run this test as an automation workflow using GitHub Actions. All you need is a file named .github/workflow/test.yml with the following content:

name: Tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js
      uses: actions/setup-node@v1
      with:
        node-version: 10.x
    - run: npm ci
    - run: npm run firebase -- emulators:exec "npm test" --import spec/fixture
      env:
        CI: true

As it turns out, it is not too difficult to set up some practical tests of a Firebase project!

Search Box and Cloud Function

Tue, 31 Mar 2020 23:45:57 -0700

For a blog hosted with Firebase Hosting, it turns out that a little search box is fairly easy to implement by using Cloud Functions for Firebase.

As with the current trend nowadays, this blog is a static site prepared with Hugo and deployed to Firebase (see my previous blog: Static Site with Hugo and Firebase). Some time ago, I realized that since I am using Firebase anyway, might as well take advantage of its Cloud Functions to add a little search functionality to the blog, particularly for its 404 page.

Of course, I am cheating a little bit. Using the above search box actually just redirects the search to my favorite search engine, DuckDuckGo, resulting in the following:

Implementing it is almost trivial. First, we need index.js inside the functions subdirectory with the content as short as this (obviously, for your blog, replace site accordingly):

const functions = require('firebase-functions');
exports.search = functions.https.onRequest((request, response) => {
  const q = request.query.q || '';
  response.redirect(`https://duckduckgo.com/?q=site:ariya.io+${q}`);
});

Once it is properly deployed, the trigger URL will be in the form of us-central1-YOURFIREBASEPROJECT.cloudfunctions.net/search. This is rather ugly. To overcome that, set up a rewrite inside firebase.json so that it looks something like:

{
    "hosting": {
      "rewrites": [{
          "source" : "/search",
          "function": "search"
        }
      ]
    }
  }

and thus, the function is available as the top-level /search of your Firebase Hosting URL, including if it is a custom domain.

After this, inserting the search box is also equally fun:

<form action="/search">
<p><input type="text" name="q" required> <button type="submit">Search</button></p>
</form>

When a visitor uses the search, they will get redirected to DuckDuckGo and be presented with the search result. Fast and easy!

Automatic Merge of Pull Requests

Sat, 29 Feb 2020 23:01:57 -0800

After using Azure DevOps for a while, I am totally sold on its Auto Complete feature for pull requests. While it does not apply universally, I do believe that any development process should be at the level where merging pull requests, or generalizing it, integrating all forms of contribution, should be as automatic and as hassle-free as possible.

If you are not familiar yet with Azure DevOps, it is basically a pay-as-you-go service for code repositories, automatic build runs, task tracker, artifact management, etc. Azure DevOps is pretty much comparable to various other similar services, such as GitHub, GitLab, Bitbucket, and many others. Note that although it bears the name Azure, you do not need to use any other Azure services to be able to take advantage of Azure DevOps offering (similar to how you can use Google Maps but without the need to store your files at Google Drive or host your email with Gmail).

One feature that makes Azure DevOps (at the time of this writing) unique compared to others is its ability to mark a PR (pull request) as Auto Complete. To do this, go to the sidebar and choose Branches (under Repo menu group). Once the branch list is displayed, hover on e.g. master and pick its context menu (rightmost three-dot menu) and choose Branch policies. Pick some settings which suit your need. Make sure to customize the Build validation, this is done by adding a simple build policy.

Now, whenever you create a pull request, there is a noticeable blue button, Set Auto complete, on the pull request page. Basically what it does is the automatic merging of the pull requests of two conditions are fulfilled:

the pull request is approved (by one or more reviewers, per branch policy)
the build succeeds, i.e. as configured with its continuous integration

There are also a few tweaks possible. For instance, you have the option to squash the branch, rebase and fast-forward, etc. Even better, there is an option to automatically delete the branch once it is merged, which can really help to reduce clutter.

Removing the manual step of merging an approved pull request will eliminate one more thing than we, human being, need to be involved with. Who would not enjoy less amount of cognitive load? I hope other services such as GitHub, GitLab, Bitbucket, and many more will follow suit and implement the same feature!

Clang on Windows

Sun, 05 Jan 2020 14:46:09 -0800

Thanks to the MSYS2 project, now there is an easy way to utilize Clang to build C/C++ application on Windows. This works equally well for both 32-bit and 64-bit programs.

MSYS2 is a fantastic (and better) reimagination of Cygwin, it is like taking the best part of a typical modern Unix environment (a familiar shell, a general collection of utilities, a porting layer, a package manager, and so on) while still working on Windows. Bootstrapping into MSYS2 is easy, either install it directly (using the GUI installer) or use Chocolatey: choco install msys2. Once inside its shell, pacman is the go-to, ever-so-powerful package manager, with thousands of packages available at your disposal.

This of course, includes the toolchain. Not only the latest GCC is there, but we also have Clang! To illustrate the concept, let us go back to the simple ANSI C/C90 program covered in the previous blog post. Once we clone the repository, open MSYS2 32-bit shell and try the following:

pacman -S msys/make mingw32/mingw-w64-i686-clang

It is a simple step to install both Make and Clang. Wait a bit and after that, do the usual magic:

CC=clang make

A caveat here, Clang for Windows does not append the .exe suffix for the executable. Thus, a quick rename to the rescue:

ren hello hello.exe

And now you can run, inspect, analyze the executable as usual.

To incorporate it into the continuous integration using Azure Pipelines (again, see the previous blog post), we shall construct a new job. The basic step is as follows.

- job: 'i686_windows_clang'
  pool:
    vmImage: 'vs2017-win2016'
  variables:
    PACMAN_PACKAGES: C:\tools\msys64\var\cache\pacman\pkg
    CC: clang

First, programmatically install MSYS2:

  - script: choco install --no-progress msys2
    displayName: 'Install MSYS2'

After that, perform some pacman maintenances:

  - script: |
      pacman -Sy
      pacman --noconfirm -S pacman-mirrors
    workingDirectory:  C:\tools\msys64\usr\bin\
    displayName: 'Check pacman'

And then, we install the required packages. At the time of this writing, Clang version 9.0 (the latest) will be installed.

  - script:  pacman --noconfirm -S msys/make mingw64/mingw-w64-x86_64-clang
    workingDirectory: C:\tools\msys64\usr\bin\
    displayName: 'Install requirements'

For the x86 architecture (aka, 32-bit Intel/AMD), install a different package:

  - script:  pacman --noconfirm -S msys/make mingw32/mingw-w64-i686-clang
    workingDirectory: C:\tools\msys64\usr\bin\
    displayName: 'Install requirements'

And now, down to the actual build step:

  - script: |
      set PATH=C:\tools\msys64\usr\bin;C:\tools\msys64\mingw64\bin;%PATH%
      make
      ren hello hello.exe
    displayName: 'make'

As a minor tweak, we can also cache pacman downloaded packages. In the above example, it hardly matters since we only install Make and Clang. But if you have a larger application, e.g. requiring Python, Qt, and so on, it is wide to avoid the CI run redownloading the same packages again and again (saving bandwith, and also being nice to those mirrors). We can achieve this by using the Cache task from Azure Pipelines. Simply insert this after MSYS2 installation step.

  - task: Cache@2
    inputs:
      key: pacman
      restoreKeys: pacman
      path: $(PACMAN_PACKAGES)
    displayName: 'Cache pacman packages'

For the complete illustration of such a job, take a look at the actual azure-pipelines.yml for the hello-c90 project.

Clang everywhere, yay!

Continuous Integration of Vanilla C Programs for Intel, ARM, and MIPS Architecture

Mon, 22 Jul 2019 15:33:34 -0700

Developing cross-platform applications presents a major challenge:, how to ensure that every commit does not break some combinations of operating systems and CPU architectures. Fortunately, thanks an array of online services and open-source tools, this challenge becomes easier to tackle.

For this demo, I have the traditional Hello, world program written in ANSI C/C90 at this repository: github.com/ariya/hello-c90 (feel free to take a look). The objective is to verify its automatic build (for the purpose of continuous integration) for a number of different CPU architectures, operating systems, as well as the C/C++ compilers. Supported CPU architectures are (using Debian nomenclatures) are amd64, i386, i686, armhf, arm64, and mips. Among some C/C++ compilers to be tested are GCC, Clang, TCC, Visual C/C++ (as part of Visual Studio 2017 and also 2019), Pelles C, Digital Mars, as well as MinGW. Obviously, some combinations are not available. For instance, there is no such thing (at least, not yet) as Visual C/C++ for Linux or MinGW targeting macOS.

In this particular blog post, we will use Azure Pipelines, a hosted build system supporting all three major OS: Windows, macOS, and Linux. For the DIY among you, the same setup can be achieved by using something like Jenkins, GitLab CI, TeamCity, and many other alternatives, along with the some build agents for the corresponding OS you want to tackle.

The build itself is configured via the YAML file, azure-pipelines.yml. There is a job for each unique combination of (Architecture, Operating System, Compiler). For example, amd64_linux_gcc denotes the build job for binary for Linux on Intel/AMD 64-bit architecture, compiled using GCC. As for now, the total number of those jobs is 16.

The obvious build job is something like this. It is running natively on the hosted agent of Azure Pipelines. We just need to make sure that the right compiler (GCC in this case) is installed. For Linux and macOS, this can be via the package manager, apt and Homebrew, respectively.

- job: 'amd64_linux_gcc'
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - script: sudo apt install -y make gcc
    displayName: 'Install requirements'
  - script: gcc --version
    displayName: 'Verify tools version'
  - script: CC=gcc make
    displayName: 'make'
  - script: file ./hello
    displayName: 'Verify executable'
  - script: ./hello
    displayName: 'Run'

On Windows however, there is no need to do that since the hosted Windows agent is already equipped with Visual Studio. However, because the build is carried out with a Makefile (more specifically, Makefile.win), we need GNU Make which is installed via Chocolatey. Note that a stage in the build job is verifying the executable (useful to know whether it is built correctly or not) using file (Linux and macOS) or dumpbin (Windows).

For two special Windows compilers, Digital Mars and Pelles C (the Windows flavor of a modified TCC), they need to be installed on the fly since they are not available on the Windows hosted agents. Digital Mars is installed with a little dance with curl and unzip. Meanwhile, Pelles C is readily available from Chocolatey.

To target non-Intel CPU architectures, we need to use some cross compilers. Since hosted Linux agent of Azure Pipelines supports Docker, the easiest way to achieve this is to use a Docker-based cross compilation using dockcross. This is explained in-depth in my previous blog post, Cross Compiling with Docker. One of such example is the following build job, for building for Linux running on ARM (32-bit). Note that since the resulting exectable is an ARM binary, we ought to use QEMU to run it.

- job: 'armhf_linux_gcc'
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - script: sudo apt install -y qemu-user
    displayName: 'Install requirements'
  - script: |
      git clone --depth 1 https://github.com/dockcross/dockcross.git
     cd dockcross
      docker run --rm dockcross/linux-armv7 > ./dockcross-linux-armv7
      chmod +x ./dockcross-linux-armv7
    displayName: 'Prepare Dockcross'
  - script: ./dockcross/dockcross-linux-armv7 bash -c '$CC --version'
    displayName: 'Verify tools version'
  - script: ./dockcross/dockcross-linux-armv7 make LDFLAGS=-static
    displayName: 'make'
  - script: file ./hello
    displayName: 'Verify executable'
  - script: qemu-arm ./hello
    displayName: 'Run'

The same approach using Docker and QEMU works well for other CPU architectures such as MIPS, ARM 64-bit, and in fact Intel x86. The last one is quite necessary, since the hosted agent of Azure Pipelines is running in 64-bit mode. Thus, we use this virtualization layer (QEMU) to verify the correct execution of 32-bit binary.

As an illustration, two examples for MingW are illustrated. The first, MinGW is installed on the Windows agent. This is self explanatory.

- job: 'amd64_windows_mingw'
  pool:
    vmImage: 'vs2017-win2016'
  variables:
    CC: 'gcc'
  steps:
  - script: choco install mingw --version 8.1.0
    displayName: 'install MinGW-w64'
  - script: gcc --version
    displayName: 'Verify tools version'
  - script: make
    displayName: 'make'
  - script: file hello.exe
    displayName: 'Verify executable'
  - script: hello.exe
    displayName: 'Run'

For the second example, MinGW is used in a cross compilation fashion. Again, we use the Docker-based dockcross to achieve this. The compiler (GCC) runs inside the Docker container on the hosted Linux agent, however it produces a Windows executable. How do we run the resulting executable? QEMU is not suitable here (since we still need to install or run Windows, remember the host is Linux). But, we have WINE to the rescue!

- job: 'i386_windows_mingw_static'
  pool:
    vmImage: 'ubuntu-16.04'
  steps:
  - script: |
      git clone --depth 1 https://github.com/dockcross/dockcross.git
     cd dockcross
      docker run --rm dockcross/windows-static-x86 > ./dockcross-windows-static-x86
      chmod +x ./dockcross-windows-static-x86
    displayName: 'Prepare Dockcross'
  - script: ./dockcross/dockcross-windows-static-x86 bash -c '$CC --version'
    displayName: 'Verify tools version'
  - script: ./dockcross/dockcross-windows-static-x86 make
    displayName: 'make'
  - script: file ./hello
    displayName: 'Verify executable'
  - script: docker run -v $PWD:/app tianon/wine:32 bash -c "wine /app/hello"
    displayName: 'Run'

In fact, to avoid the hassle of on-the-fly installation/configuration of WINE, we just use the Dockerized WINE.

The whole ordeal of running 16 jobs will take anywhere from 5 minutes to 20 minutes. Obviously, if you are constrainted by the free tier of Azure Pipelines, you can purchase access to more hosted agents or attach your own build agents, which will definitely parallelize and speed things up.

I hope that the idea outlined in this post will inspire to continue to work on more cross-platform apps. Of course, it does not have to be an application written in ANSI C. The concept can be applied to D, Go, Rust, and many other modern compilers.