A .NET Framework error occurred during execution of user-defined routine or aggregate "No extended information available." calling geography

Question

I'm trying to pull lat/lon coordinates from one table and see if they intersect a particular area using geometry::STIntersect. However, I get the error in the title after the query has started running and producing results. This only happens intermittently, though. Below is a pseudocode example of what I'm trying to do:

SELECT *
FROM table_with_lat_lons
INNER JOIN(SELECT id,1 include 
    FROM table_with_lat_lons 
    INNER JOIN table_with_geog_boundaries 
    ON table_with_geog_boundaries.geog.STIntersects(geography::Point(ISNULL(CAST(y_Coord as float),0.0), ISNULL(Cast(x_Coord as float),0.0), 4326)) = 1) bounds 
    ON table_with_lat_lons.id =bounds.id
    WHERE bounds.include=1

Wrapping everything into a with...then will fix the error, but that's not a viable solution for this issue for me