How to make mocha fail when no expect within it()

Question

Is there an option to set mocha to report test as failing in case no expect provided within it() function?

Idea is workflow to be like this:

1. add one it() with desc and callback function
2. the it() is reported as fail since no expectation is set in the callback
3. expectation is added
4. the it() is still reported as fail as expectation is not met since no implementation
5. implementation is added
6. the it() is reported as success
7. refactor

So main intention is while doing canonical TDD style developing newly added test are reported as failed until expectation is set (or test set as pending without callback or skip()), which is again reported as fail and once implementation is done it is reported as success.

Value I see for it() succeeding without expectation is that once it is added it's failure now proves that it is actually working and proves it is failing. Was it intention or am I missing something?

Also, if anyone have idea how to set that up in karma.conf.js would be great.

Thanks

Answer 1

Mocha does not support what you want to do just by setting a flag. The closest thing is to use it without a callback:

`it("foo")`

Mocha will treat this test as pending and report it as such. It is the same as using it.skip(...). However, the test is not failed, and it does not catch stupid mistakes like having a loop that does not actually iterate:

it("foo", function () {
    var a = something();
    for (var i = 0; i < a.length; ++i) {
        expect(a[i]).to...
    }
});

If it so happens that a is a 0-length array, then you'll not test anything, and the test will pass. In cases like this one I test that the array is not 0-length, but still...

So there is no direct way to do it, and Mocha offers no API for assertion libraries to hook into to tell Mocha that they've actually been used in a test. You can build your own solution though. Here's a proof of concept:

var real_expect = require("chai").expect;

var expect_called = 0;

function expect() {
    expect_called++;
    return real_expect.apply(this, arguments);
}

var real_it = it;

it = function (name, fn) {
    if (!fn.length) {
        // Handle the case where `fn` is declared to be synchronous.
        real_it(name, function () {
            expect_called = 0;
            fn.call(this);
            if (expect_called === 0)
                throw new Error("test did not call expect");
        });
    }
    else {
        // Handle the case where `fn` is declared to be asynchronous.
        real_it(name, function (real_done) {
            expect_called = 0;
            function done () {
                if (expect_called === 0) {
                    done(new Error("test did not call expect"));
                    return;
                }
                real_done();
            }
            fn.call(this, done);
        });
    }
};

it("foo", function () {
    expect(1).to.equal(1);
});

it("foo async", function (done) {
    setTimeout(function () {
        expect(1).to.equal(1);
        done();
    }, 1000);
});

it("bar", function () {});
it("bar 2", function () {});

In the code above, we replace it with our own, which does the check, and we replace expect with our own to flag when it has been called.

A note about asynchronous tests and shared state. Sometimes people think that Mocha will run multiple at the same time if they are marked as asynchronous. This is not normally the case. Mocha waits for one of two things before continuing after an asynchronous test: the test calls its done callback or it times out. You can have code from two tests running at the same time if the earlier test timed out and it so happens that the test that timed out was actually waiting for an asynchronous operation that completes after the time out. In such case, if there is any state that both tests depend on, the timeout can cause cascading test failures (or cascading test successes!). This is a general issue with Mocha. Once the timeout problem is fixed, then the cascading effect will disappear and subsequent tests will succeed or fail on their own merit, without being affected by earlier asynchronous tests that timed out. In the code above, expected_called is a state that all tests depend on. So a timeout may cause cascading effects.

To solve this problem, each test would have to have its own private instance of expect, which would only increment its own private counter. This could be done as follows:

var real_expect = require("chai").expect;

var real_it = it;

it = function (name, fn) {
    if (!fn.length) {
        // Handle the case where `fn` is declared to be synchronous.
        real_it(name, function () {
            var expect_called = 0;

            this.expect = function () {
                expect_called++;
                return real_expect.apply(this, arguments);
            };

            fn.call(this);
            if (expect_called === 0)
                throw new Error("test did not call expect");
        });
    }
    else {
        // Handle the case where `fn` is declared to be asynchronous.
        real_it(name, function (real_done) {
            var expect_called = 0;

            this.expect = function () {
                expect_called++;
                return real_expect.apply(this, arguments);
            };

            function done () {
                if (expect_called === 0) {
                    done(new Error("test did not call expect"));
                    return;
                }
                real_done();
            }

            fn.call(this, done);
        });
    }
};

it("foo", function () {
    this.expect(1).to.equal(1);
});

it("foo async", function (done) {
    var me = this;
    setTimeout(function () {
        me.expect(1).to.equal(1);
        done();
    }, 1000);
});

it("bar", function () {});
it("bar 2", function () {});

The disadvantage though is that you now have to access expect as this.expect, which means writing tests differently than you usually would. You may think that setting the global expect before every test would eliminate the need to use this but this approach would be subject to exactly the same problem as I discussed above. (The global state shared by the tests would be expect itself instead of expect_called.)

Answer 2

I know this is an old question, but in case anyone else is searching for this, there is a much easier way, using Sinon. You would simply require this file at the top of every test file.

let checkForExpectation = [];

// before each test...
beforeEach( () => {

    // ... spy on the expect functions so we will know whether they are called
    checkForExpectation = [
        sinon.spy( chai.expect, 'fail' ),
        sinon.spy( chai, 'expect' )
        // you can also add spies for "assert" here with a loop, but "should" is much harder
    ];

} );

// after each test ...
afterEach( function() { // must use "function()" due to needing `this`

    // ... look for one of the expect functions to have been called
    const called = !!checkForExpectation.find( spy => spy.called );
    checkForExpectation = undefined;

    // ... restore the sinon contexts to their initial state
    sinon.restore();

    // ... create an error for the test that has just ended
    if ( !called && this.currentTest.state !== 'failed' ) {
        this.test.error( new chai.AssertionError( `Test "${this.currentTest.title}" contained no expect statement` ) );
    }

} );

Answer 3

Mocha 9.1 add the --fail-zero option which makes mocha exit with an error code ("non-zero exit code") if no tests are encountered.

Not sure if this is 100% what you're looking for but it's what I was looking for when I came across this Q&A. This might not work when there are some tests defined but not on a per-it() block. I haven't tested.

Answer 4

The solution here has perhaps existed after the original answer, but it is to pass a done callback to the test case. As outlined here: https://blog.cloudboost.io/javascript-asynchronous-testing-gotchas-ac7e5c39257