Turing.jl estimators show proposal will be rejected due to numerical errors

Question

Model in Turing.jl seems to be stuck in errors with

Warning: The current proposal will be rejected due to numerical error(s).
│   isfinite.((θ, r, ℓπ, ℓκ)) = (true, false, false, false)

for NUTS(), HMCDA() and sometimes HMC() sampling methods. I don't really understand what's causing these errors (what's θ?), but it makes NUTS and HMCDA unusable as sampling methods while HMC has around 2/3 of samples rejected. I looked at similar questions on here and on the forums, but no one seems to have a fix for this so far.

Answer 1

From AdvancedHMC GitHub https://github.com/TuringLang/AdvancedHMC.jl/blob/beeb37b418992a3280fc3e59d01d2a124639507e/src/hamiltonian.jl

θ::T  # Position variables / model parameters.
r::T  # Momentum variables
ℓπ::V # Cached neg potential energy for the current θ.
ℓκ::V # Cached neg kinect energy for the current r.

Answer 2

θ is the current parameter vector, while the other three will depend on the target posterior distribution and are parameters to the physics engine used by HMC to explore the posterior.

If you are getting this warning for almost all of the proposed samples, your model is probably misspecified, but it's impossible to tell without the Turing model code of course.

It's not a big deal if you get this message a few dozen times while fitting model, especially if its during the warmup stage. I frequently get this message if there are parts of parameter space where the likelihood is not defined - for example if the model depends on the solution to a differential equation that returns NaN with the current parameters.