Sourcemap + istanbul/isparta code coverage for a webpack + babel (for es6) + mocha (+karma)

Question

I have a test project at https://github.com/danielbush/webpack-babel-karma-sourcemap-coverage.

It consists of two simple source files in src/, one with a deliberate error (src/lib2.js).

I want to webpack and transpile using babel from ES6 to ES5 and run the Mocha tests on this bundle using Karma.

I've already added source mapping and tested this so that I can see the line number of the original file.

For me, the stack trace looks like this:

21 02 2016 16:03:15.445:INFO [karma]: Karma v0.13.21 server started at http://localhost:9876/
21 02 2016 16:03:15.451:INFO [launcher]: Starting browser Chrome
21 02 2016 16:03:16.551:INFO [Chrome 48.0.2564 (Linux 0.0.0)]: Connected on socket /#yRS32ons0_2HGhrwAAAA with id 3072946

START:
  lib1
    ✖ should return 1

Finished in 0.015 secs / 0.001 secs

SUMMARY:
✔ 0 tests completed
✖ 1 test failed

FAILED TESTS:
  lib1
    ✖ should return 1
      Chrome 48.0.2564 (Linux 0.0.0)
    Error: SIMULATED ERROR
        at foo (/home/danb/projects/so-test-project/tests.webpack.js:135:10 <- webpack:///src/lib2.js:13:8)
        at Context.<anonymous> (/home/danb/projects/so-test-project/tests.webpack.js:93:31 <- webpack:///test/index.js:7:11)

On the 2nd last line above, it shows webpack:///src/lib2.js:13 which is the correct location in the original source.

To get that mapping, I did this: https://github.com/danielbush/webpack-babel-karma-sourcemap-coverage/commit/6ea992cae499dccc68488bcb3eaca642ae4ed24e

What I want to do is add coverage using something like istanbul (or isparta which uses istanbul) so that I can generate an HTML coverage report in coverage/ that shows me lines that aren't covered (potentially) -- as well as overall coverage percentages.

But I want to ensure that my stack traces are still source mapped correctly like the above.

I've tried isparta-loader and isparta-instrumenter-loader to transpile and instrument the files in question in webpack, but my line numbers are slightly wrong in the stacktrace (above). There seems to be a number of isparta-like things floating around and the documentation is not super-friendly.

To express all of this a bit more generally: I'm bundling files and tests and then testing this in the browser (karma or by more manual means), and I want my source maps (in the browser) to point to the original lines of code and at the same time I want a coverage report to refer to the original lines and files that haven't been covered.

Answer 1

I don't have a solution for a single karma/webpack configuration. My workaround is to have 2 karma confs - which I'm quite happy with atm. (I have other non-karma setups for running tests as well.)

One conf runs the tests in the background with accurate stack traces for my es6 code, and the other runs coverage using isparta-loader to transpile and instrument my source code. (isparta-loader uses isparta, which uses istanbul).

So my "running tests" karma conf looks a bit like this:

var webpack = require('webpack');
module.exports = function (config) {
  config.set({
    browsers: [ 'PhantomJS' ],
    singleRun: false,
    frameworks: [ 'mocha' ], // Use the mocha test framework
    files: [
      'tests.webpack.js'
    ],
    preprocessors: {
      'tests.webpack.js': [ 'webpack', 'sourcemap' ]
    },
    reporters: [ 'mocha' ],
    webpack: {
      devtool: 'inline-source-map',
      module: {
        loaders: [
          // Use imports loader to hack webpacking sinon.
          // https://github.com/webpack/webpack/issues/177
          {
            test: /sinon\.js/,
            loader: "imports?define=>false,require=>false"
          },
          {
            test: /\.js$/,
            exclude: /node_modules/,
            loader: 'babel-loader',
            query: { presets: ['es2015'] }
          }
        ]
      }
    },
    webpackServer: {
      noInfo: true
    }
  });
}

and my coverage karma conf looks a bit like this:

var webpack = require('webpack'),
    path = require('path');
module.exports = function (config) {
  config.set({
    browsers: [ 'PhantomJS' ],
    singleRun: true,
    frameworks: [ 'mocha' ],
    files: [
      'tests.webpack.js'
    ],
    preprocessors: {
      'tests.webpack.js': [ 'webpack' ]
    },
    reporters: [ 'coverage' ],
    webpack: {
      // *optional* babel options: isparta will use it as well as babel-loader
      babel: {
        presets: ['es2015']
      },
      // *optional* isparta options: istanbul behind isparta will use it
      isparta: {
        embedSource: true,
        noAutoWrap: true,
        // these babel options will be passed only to isparta and not to babel-loader
        babel: {
          presets: ['es2015']
        }
      },
      module: {
        loaders: [
          {
            test: /sinon\.js/,
            loader: "imports?define=>false,require=>false"
          },
          // Perform babel transpiling on all non-source, test files.
          {
            test: /\.js$/,
            exclude: [
              path.resolve('node_modules/'),
              path.resolve('lib/')
            ],
            loader: 'babel-loader',
            query: { presets: ['es2015'] }
          },
          // Instrument source files with isparta-loader (will perform babel transpiling).
          {
            test: /\.js$/,
            include: [
              path.resolve('lib/')
            ],
            loader: 'isparta'
          }
        ]
      }
    },
    webpackServer: {
      noInfo: true
    },
    coverageReporter: {
      reporters: [
        { type: 'html', dir: 'coverage/' },
        { type: 'text' }
      ]
    }
  });
}

This 2nd conf provides a text report that gives immediate coverage summary in the terminal, and the html report which gives nice source file highlighting of untested code. (There's a hack in both for sinon, which isn't related to this issue.) tests.webpack.js uses webpack's require.context to pull in my browser tests written in mocha. And various karma-* plugins are required.

Answer 2

You may have better luck using Istanbul's new official Babel Plugin to instrument your sources for code coverage.

In my experience, compared to isparta, it provided more accurate coverage reports and required basically no configuration apart from adding it to the list of plugins in my Babel configuration.

Sourcemaps appeared to work correctly "out of the box" when I used it on my project.