Code Coverage of QUnit Tests using Istanbul and Karma

QUnit, used by projects like jQuery and jQuery Mobile, is a rather popular JavaScript testing framework. For tests written using QUnit, how do we measure its code coverage? A possible solution which is quite easy to setup is to leverage the deadly combination of Karma and Istanbul.

Just like our previous adventure with Jasmine code coverage, let us take a look at a simple code we need to test. This function My.sqrt is a reimplementation of Math.sqrt which may throw an exception if the input is invalid.

var My = {
  sqrt: function(x) {
    if (x < ) throw new Error("sqrt can't work on negative number");
      return Math.exp(Math.log(x)/2);
  }
};

A very simple QUnit-based test for the above code is as follows.

test("sqrt", function() {
  deepEqual(My.sqrt(4), 2, "square root of 4 is 2");
});

Manually running the test is easy as opening the test runner in a web browser:

For a smoothed development workflow, an automated way to run the tests will be much preferred. This is where Karma becomes very useful. Karma also has the ability to launch a predetermined collection of browsers, or even to use PhantomJS for a pure headless execution (suitable for smoke testing and/or continuous delivery).

Before we can use Karma, installation is necessary:

npm install karma karma-qunit karma-coverage

Karma requires a configuration file. For this purpose, the config file is very simple. As an illustration, the execution is done by PhantomJS but it is easy to include other browsers as well.

module.exports = function(config) {
  config.set({
    basePath: '',
    frameworks: ['qunit'],
    files: [
      '*.js',
      'test/spec/*.js'
    ],
    browsers: ['PhantomJS'],
    singleRun: true,
    reporters: ['progress', 'coverage'],
    preprocessors: { '*.js': ['coverage'] }
  });
};

Now you can start Karma with the above configuration, it would say that the test passes just fine. Should you encounter some problems, you can look at an example repository I have setup github.com/ariya/coverage-qunit-istanbul-karma, it may be useful as a starting point or a reference for your own project. As a convenience, the test in that repository can be executed via npm test.

What is more interesting here is that Karma runs its coverage processor, as indicated by preprocessors in the above configuration. Karma will run Istanbul, a full-featured instrumenter and coverage tracker. Essentially, Istanbul grabs the original JavaScript source and injects extra instrumentation code so that it can gather the execution metrics once the process finishes (read also my previous blog post on JavaScript Code Coverage with Istanbul). In this Karma and Istanbul combo, the generated coverage report is available in the under the subdirectory coverage.

The above report indicates that the single test for My.sqrt is still missing the test for an invalid input, thanks to branch coverage feature of Istanbul. The I indicator next to the conditional statement tells us that the if branch was never taken. Of course, once the issue is known, adding another test which will cover that branch is easy (left as an exercise for the reader).

Now that code coverage is tracker, perhaps you are ready for the next level? It is about setting the hard threshold so that future coverage regression will never happen. Protect yourself and your team from carelessness, overconfidence, or honest mistakes!