How to test an exception from a future

Question

Continuing from yesterday question, how would I test that a async method throws an exception.

main(){
  test( "test2", () async {
    expect( await throws(), throwsException);
  });

}

Future throws () async {
  throw new FormatException("hello");
}

Answer 1

This way it works:

import 'package:test/test.dart';
import 'dart:async';

void main() {
  test( "test2", ()  { // with or without `async`
    expect(throws(), throwsA(const TypeMatcher<FormatException>()));
  });
}

Future throws () async {
  throw new FormatException("hello");
}

Basically just remove await. The test framework can deal with futures no matter if they succeed or fail.

Answer 2

There are multiple ways to test errors coming from Future. Gunter's answer will work if "throws without async" method is throwing some exception. Below sample will handle exception coming from future methods.

import 'package:test/test.dart';
import 'dart:async';

void main() {
    test("test with Zone", () {
        runZoned(() {
            throws();
        }, onError: expectAsync((e, s) {
            expect(e, new isInstanceOf<FormatException>());
        }));
    });

    test('test with future catch error', () {
        throws().catchError(expectAsync((e) {
            expect(e, new isInstanceOf<FormatException>());
        }));
    });
}

Future throws() async{
    Completer completer = new Completer();
    completer.complete(new Future(() => throw new FormatException("hello")));
    return completer.future;
}

Answer 3

Use try-catch

The most reliable approach is to use a try-catch block to explicitly catch the exception and ensure the method has finished running.

try {
    await methodWhichThrows();
    fail("exception not thrown");
} catch (e) {
    expect(e, new isInstanceOf<...>());
    // more expect statements can go here
}

This approach also has the advantage that additional checks on the value of the exception can be performed.

Expect with throwsA only works as the last statement

Using expect by itself only works if it is the last statement in the test. There is no control over when the method will throw the exception, so there can be race conditions with the statements (including subsequent calls to expect), if they assume the exception has already been thrown.

expect(methodWhichThrows(), throwsA(new isInstanceOf<...>())); // unreliable unless last

It can be used, but you have to very careful to remember which situations it works in and which it doesn't. So it is safer to stick with the try-catch approach rather than using different approaches for different situations.

Demonstration

The following complete example demonstrates the effect of race conditions on the two approaches:

import 'dart:async';
import 'package:test/test.dart';

//----------------------------------------------------------------
/// This approach to expecting exceptions is reliable.
///
Future reliableApproach(int luck) async {
  expect(await setValueAndReturnsHalf(42), equals(21));
  expect(state, equals(Evenness.isEven));

  try {
    await setValueAndReturnsHalf(3);
    fail("exception not thrown");
  } catch (e) {
    expect(e, new isInstanceOf<ArgumentError>());
  }

  // Expect value to be odd after execption is thrown.

  await shortDelay(luck); // in my experience there's no such thing called luck
  expect(state, equals(Evenness.isOdd));
}

//----------------------------------------------------------------
/// This approach to expecting exceptions is unreliable.
///
Future unreliableApproach(int luck) async {
  expect(await setValueAndReturnsHalf(42), equals(21));
  expect(state, equals(Evenness.isEven));

  expect(setValueAndReturnsHalf(3), throwsA(new isInstanceOf<ArgumentError>()));

  // Expect value to be odd after execption is thrown.

  await shortDelay(luck); // luck determines if the race condition is triggered
  expect(state, equals(Evenness.isOdd));
}

//----------------------------------------------------------------

enum Evenness { isEven, isOdd, inLimbo }

int value = 0;
Evenness state = Evenness.isEven;

/// Sets the [value] and [state].
///
/// If the [newValue] is even, [state] is set to [Evenness.isEven] and half of it
/// is returned as the Future's value.
///
/// If the [newValue] is odd, [state] is set to [Evenness.isOdd] and an exception
/// is thrown.
///
/// To simulate race conditions, this method takes 2 seconds before it starts
/// processing and 4 seconds to succeed or throw an exception. While it is
/// processing, the [state] is set to [Evenness.inLimbo].
///
Future<int> setValueAndReturnsHalf(int newValue) async {
  await shortDelay(2);

  state = Evenness.inLimbo;

  await shortDelay(2);

  value = newValue;

  if (newValue % 2 != 0) {
    state = Evenness.isOdd;
    throw new ArgumentError.value(newValue, "value", "is not an even number");
  } else {
    state = Evenness.isEven;
    return value ~/ 2;
  }
}

/// Delays used to simulate processing and race conditions.
///
Future shortDelay(int seconds) {
  var c = new Completer();
  new Timer(new Duration(seconds: seconds), () => c.complete());
  return c.future;
}

/// Examples of the reliable and unreliable approaches.
///
void main() {
  test("Correct operation when exception is not thrown", () async {
    expect(await setValueAndReturnsHalf(42), equals(21));
    expect(value, equals(42));
  });

  group("Reliable approach:", () {
    test("works when there is bad luck", () async {
      // 1 second = bad luck, future returning function not started processing yet
      await reliableApproach(1);
    });

    test("works when there is more bad luck", () async {
      // 3 second = bad luck, future returning function still processing
      await reliableApproach(3);
    });

    test("works when there is good luck", () async {
      // 5 seconds = good luck, future returning function definitely finished
      await reliableApproach(5);
    });
  });

  group("Unreliable approach:", () {
    test("race condition encountered by bad luck", () async {
      // 1 second = bad luck, future returning function not started processing yet
      await unreliableApproach(1);
    });

    test("race condition encountered by more bad luck", () async {
      // 3 second = bad luck, future returning function still processing
      await unreliableApproach(3);
    });

    test("race condition avoided by good luck", () async {
      // 5 seconds = good luck, future returning function definitely finished
      await unreliableApproach(5);
    });
  });
}

Answer 4

The easiest and shortest answer is:

expect(throws(), throwsException)

To test exception/error type:

expect(throws(), throwsA(predicate((e) => e is MyException)));

Answer 5

After too many tries and errors I found that this one works as expected:

test('fetch SHOULD throw exception WHEN api fail with exception', () {
  when(clientMock.get(uri)).thenAnswer((_) async => throw Exception());
  expect(() => sut.fetch(), throwsA(isInstanceOf < Exception > ()));
});

Answer 6

The official document have good examples by using expectLater with throwsA for that as following examples.

void functionThatThrows() => throw SomeException();

void functionWithArgument(bool shouldThrow) {
  if (shouldThrow) {
    throw SomeException();
  }
}

Future<void> asyncFunctionThatThrows() async => throw SomeException();

expect(functionThatThrows, throwsA(isA<SomeException>()));

expect(() => functionWithArgument(true), throwsA(isA<SomeException>()));

var future = asyncFunctionThatThrows();
await expectLater(future, throwsA(isA<SomeException>()));

await expectLater(
    asyncFunctionThatThrows, throwsA(isA<SomeException>()));

Answer 7

I wasn't satisfied with any of the other answers; too verbose.

So the short version is:

test('ArgumentError is thrown when throwIt is true', () {
   expectLater(() => myAsyncFunctionThatTakesAnArgument(throwIt: true), throwsA(isA<ArgumentError>()));
});