The problem: running inference in tensorflow in GPU gives correct result, while running the same code in CPU gives as output only NANs.
I have tried: 1 checking that the input does not contain NANs, I tried using an array of ones and get the same problem. 2 The mean min and max of the input are the same in GPU and CPU 3 The weights of the model are the same, I have not check if there are nans in the CPU when loading the model (how to find the tensors that contain NANs in tensorflow?)
Does anyone know first what could be the problem and how to fix it? A possible problem may come from bach norm as is the only operation doing division.