A TFX equivalent for Pytorch

Question

I'v recently worked with TensorFlow Extended(TFX) platform. Since my development background is on Pytorch stack, I'm looking for well-compatible alternatives of TFX for Pytorch.

While searching for this alternative, I came across torchx and torchserve. But torchx is a newly developed library and not so comprehensive as well as TFX. On the other hand, torchserve focus is on deployment and software engineering side of Machine Learning Operation(MLOps) and doesn't have many rich features as good as TFX for data pipeline.

Is there any other alternative for Pytorch stack?

Answer 1

This might be sort of self-serving, as I'm one of the co-creators of the proposed framework, but as the question asked directly for recommendations, I think its fair with this disclaimer up front.

How about using ZenML. It is a TFX-like framework for all frameworks including PyTorch (indeed uses ml-pipelines-sdk in the background - although it will be removed soon).

Here are the docs for your convenience and a pytorch example to get your started. Here is a recent blog about converting simple PyTorch code to ZenML pipelines.

Cheers and feel free to leave a comment that this answer is too market-y and I'll delete it (It isnt meant to be, I just stumbled upon this question today).