How to compare floating point in catch2

Question

I am using Catch v2.13.1

What is the correct way to compare float values. I thought the below would fail, but both pass.

REQUIRE(1147332687.7189338 == Approx(1147332688.4281545).margin(0.0001));
REQUIRE(1147332687.7189338 == Approx(1147332688.4281545));

However this fails as expected

REQUIRE(abs(1147332687.7189338 - 1147332688.4281545) <= Approx(0).margin(0.0001));

I don't understand why the first two statements wouldn't work

Answer 1

There are a couple of things to consider in the posted example.

REQUIRE(1147332687.7189338 == Approx(1147332688.4281545));

This will pass, "unexpectedly". The reason can be found in the documentation (assertions - floating point comparisons).

Approx is constructed with defaults that should cover most simple cases. For the more complex cases, Approx provides 3 customization points:

• epsilon - epsilon serves to set the coefficient by which a result can differ from Approx's value before it is rejected. By default set to std::numeric_limits<float>::epsilon()*100.
• [...]

In the posted example, the two numbers differ by a cofficient near 6.2e-10, while the default is (given a 32-bit float) near 1.2e-5.

The following test wouldn't pass.

CHECK( a == Approx(b).epsilon(1e-12) );

---

The other tests involve margin, which is described in the documentation as

• margin - margin serves to set the the absolute value by which a result can differ from Approx's value before it is rejected. By default set to 0.0.

The caveat, though, can be found in issue#1507.

This is because of the default value for epsilon in the Approx class and the fact that Approx::equalityComparisonImpl will pass if the value is in the range of the margin OR the epsilon values.

So, this test wouldn't pass:

CHECK( a == Approx(b).margin(0.0001).epsilon(1e-12) );

Note that this "issue" seems to be marked as resolved - not a bug:

So, I don't think this is a bug.

The reason for this is that it is very hard (well, impossible), to determine the user's intent, thus it is better to assume that the user has set up both checks correctly -- after all, if the user does not want a relative comparison, they can always set the epsilon to zero. In fact, I think that using both tolerances and taking the most forgiving one is the superior option to implement things like the Approx Matcher (#1499).

Answer 2

Since you are using a Catch version >= 2.10, you might want to switch to Matchers which is the recommended way of dealing with floating point comparison in Catch (the doc advises against using Approx in new code). They make for a much improved, foolproof test API.

The downside to Approx is that it has a couple of issues that we cannot fix without breaking backwards compatibility. Because Catch2 also provides complete set of matchers that implement different floating point comparison methods, Approx is left as-is, is considered deprecated, and should not be used in new code.

#include <catch2/matchers/catch_matchers_floating_point.hpp>
REQUIRE_THAT(1147332687.7189338, Catch::Matchers::WithinAbs(1147332688.4281545, 0.0001)); // fails