I have been getting the below error. I have already trained the model and faced no issues while doing so, but receiving this error while making predictions. How to resolve this?
OOM when allocating tensor with shape[224,128,128,128] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
[[{{node model_1/time_distributed_4/conv2d_transpose_2/conv2d_transpose}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.
[Op:__inference_predict_function_4260]
While training I set the batch size to 16 and trained the model, and it trained fine (for 80,000 frames). What to do while making predictions?