How to create a setuptools sdist .tar.gz package from a Conda project?

Question

I have a PyTorch training project where dependencies are managed using conda. I need to package my project as a setuptools software distribution (sdist) in a .tar.gz file so that I can run it as a custom job in Google Cloud Platform's Vertex.AI Training service.

I've found an official guide that explains how to generate a bdist package using Conda but I don't think I can use that in Vertex.AI.

Is there a Conda command for generating an sdist distribution? Or are there any other tools I can use to get this done automatically?

Please note: I know I can manually recreate the list of dependencies in my setuptools setup.py file but I would rather not do that because manually maintaining two copies of the same list could link to sync issues and errors.

Answer 1

You can use python setuptools to achieve it. On vertex ai documentation about using Setuptools to build a source distribution to use with Vertex AI there is a complete guide on how to achieve it.

Its provide you with this steps to consider when dealing with your own packages:

• Follow the instructions on Before you begin and Application structure
• Create your setup.
• Add dependencies ( Could be Python or Custom dependencies - If any )
• Setup Python Modules ( If any )
• Upload your source distribution to Cloud Storage

Note: Vertex AI suggest some alternatives to it as described on Alternatives to creating a source distribution.