How can I evaluate the performance of my TensorFlow model on specific slices of a large dataset?

Question

How can I evaluate the performance of my TensorFlow model on specific slices (segments) of a large evaluation dataset?

Answer 1

Use TensorFlow Model Analysis (TFMA) which is an open-source library that combines TensorFlow and Apache Beam to compute and visualize evaluation metrics. It is designed for this use case, and allows you to evaluate your models on large amounts of data in a distributed fashion, using the same metrics defined in your TensorFlow trainer. These metrics can also be computed over different slices of data, and the results can be visualized in Jupyter Notebooks. TFMA uses Apache Beam to do a full pass over your specified evaluation dataset. This not only allows more accurate calculation of metrics, but also scales up to massive evaluation datasets, since Beam pipelines can be run using distributed processing back-ends.

See https://github.com/tensorflow/model-analysis for more information.