Apache math3 erfInv fails on very small values, while Excel NORM.S.INV does it right…

Question

I have to compute in Java the same result as Excel NORM.S.INV.

^{(actually using LOI.NORMALE.STANDARD.INVERSE in French, that's NORM.S.INV in English)}

I use org.apache.commons.math3.special.Erf.erfInv this way:

return Math.sqrt(2) * Erf.erfInv(2 * p - 1);

But when p is small enough (around 1e-17), erfInv returns Infinity, because when p is that small, 2 * p -1 is equal to -1.

So I created a modified erfInv function, preventing this kind of problem by turning the equations around, and it works well, well, up to a point.

---

public double inverseCumulativeProbability(final double p) throws OutOfRangeException {

    if (p < 0.0 || p > 1.0) {
        throw new OutOfRangeException(Double.valueOf(p), Double.valueOf(0), Double.valueOf(1));
    }
    final boolean precisionError = p > 0 && Double.compare(1 - 2 * p, 1) == 0;
    return FastMath.sqrt(2) * (precisionError ? this.erfInvModified(2 * p) : Erf.erfInv(2 * p - 1));
}

/**
 * Rewrite of {@link Erf#erfInv(double)} using a parameter which has yet to be subtracted 1, which has not been
 * done because of the error ε -1 = -1
 *
 * @param ε a very small value, which has yet to be subtracted 1
 * @return the correct value
 */
private double erfInvModified(final double ε) {

    // the original writing [ (1 - (ε - 1)) × (1 + ( ε - 1)) ] gave a result of 0 instead of  [ 2 × ε ]
    double w = -FastMath.log(2 * ε);
    double p = this.__originalCode(w);

    // rounding equivalent to the original writing [ p × (ε - 1) ]
    return -p;
}

/**
 * This part is copied unmodified from the original erfInv method
 */
private double __originalCode(double w) {

    double p;

    if (w < 6.25) {
        w = w - 3.125;
        p = -3.6444120640178196996e-21;
        p = -1.685059138182016589e-19 + p * w;
        p = 1.2858480715256400167e-18 + p * w;
        p = 1.115787767802518096e-17 + p * w;
        p = -1.333171662854620906e-16 + p * w;
        p = 2.0972767875968561637e-17 + p * w;
        p = 6.6376381343583238325e-15 + p * w;
        p = -4.0545662729752068639e-14 + p * w;
        p = -8.1519341976054721522e-14 + p * w;
        p = 2.6335093153082322977e-12 + p * w;
        p = -1.2975133253453532498e-11 + p * w;
        p = -5.4154120542946279317e-11 + p * w;
        p = 1.051212273321532285e-09 + p * w;
        p = -4.1126339803469836976e-09 + p * w;
        p = -2.9070369957882005086e-08 + p * w;
        p = 4.2347877827932403518e-07 + p * w;
        p = -1.3654692000834678645e-06 + p * w;
        p = -1.3882523362786468719e-05 + p * w;
        p = 0.0001867342080340571352 + p * w;
        p = -0.00074070253416626697512 + p * w;
        p = -0.0060336708714301490533 + p * w;
        p = 0.24015818242558961693 + p * w;
        p = 1.6536545626831027356 + p * w;
    }
    else if (w < 16.0) {
        w = FastMath.sqrt(w) - 3.25;
        p = 2.2137376921775787049e-09;
        p = 9.0756561938885390979e-08 + p * w;
        p = -2.7517406297064545428e-07 + p * w;
        p = 1.8239629214389227755e-08 + p * w;
        p = 1.5027403968909827627e-06 + p * w;
        p = -4.013867526981545969e-06 + p * w;
        p = 2.9234449089955446044e-06 + p * w;
        p = 1.2475304481671778723e-05 + p * w;
        p = -4.7318229009055733981e-05 + p * w;
        p = 6.8284851459573175448e-05 + p * w;
        p = 2.4031110387097893999e-05 + p * w;
        p = -0.0003550375203628474796 + p * w;
        p = 0.00095328937973738049703 + p * w;
        p = -0.0016882755560235047313 + p * w;
        p = 0.0024914420961078508066 + p * w;
        p = -0.0037512085075692412107 + p * w;
        p = 0.005370914553590063617 + p * w;
        p = 1.0052589676941592334 + p * w;
        p = 3.0838856104922207635 + p * w;
    }
    else if (!Double.isInfinite(w)) {
        w = FastMath.sqrt(w) - 5.0;
        p = -2.7109920616438573243e-11;
        p = -2.5556418169965252055e-10 + p * w;
        p = 1.5076572693500548083e-09 + p * w;
        p = -3.7894654401267369937e-09 + p * w;
        p = 7.6157012080783393804e-09 + p * w;
        p = -1.4960026627149240478e-08 + p * w;
        p = 2.9147953450901080826e-08 + p * w;
        p = -6.7711997758452339498e-08 + p * w;
        p = 2.2900482228026654717e-07 + p * w;
        p = -9.9298272942317002539e-07 + p * w;
        p = 4.5260625972231537039e-06 + p * w;
        p = -1.9681778105531670567e-05 + p * w;
        p = 7.5995277030017761139e-05 + p * w;
        p = -0.00021503011930044477347 + p * w;
        p = -0.00013871931833623122026 + p * w;
        p = 1.0103004648645343977 + p * w;
        p = 4.8499064014085844221 + p * w;
    }
    else {
        p = Double.POSITIVE_INFINITY;
    }
    return p;
}

---

This produces accurate results, very near to those given by NORM.S.INV, but they slowly diverge as p is becoming smaller:

p        Excel    modified erfInv
1E-30    11.464   11.458
1E-40    13.311   11.642
1E-50    14.933   -49.694 
1E-75    18.377   -19628.2
1E-100   21.273   -705825.1

I see no error in my logic, the only source of error I can think of is the third polynomial part in apache library when w is getting too high… !

I tried using BigDecimal, but the divergence is the same, it is not a rounding problem…

What advice can you give me ? An other library to use maybe ?

Thanks for your help…