Build python package for CLI script

Question

I'm trying to build my first python package and publish it to PyPi. My goal is to provide a certain feature via CLI so the final user can install it via pip like pip3 install my-package and use it directly from the terminal like my-package -fa first_argument -sa second_argument.

Let's say my package is called fp-test-package. I made a mockup of my project folder structure that you can find here https://github.com/fabiopipitone/fp-test-package.

In the mockup I also inserted as requirements one of the packages required in the real project (tqdm) and keep the exact same convention (hyphen on external folder and github repo, underscores for internal packages, same position and import for helpers and utils and so on). This way, if it works on the mockup it must work on the real project.

Here's the directory strucure of the project

fp-test-package
├── fp_test_package
│   ├── __init__.py
│   ├── fp_test_package.py
│   ├── helpers
│   │   ├── arguments_checkers.py
│   │   ├── csv_handlers.py
│   │   └── utility_functions.py
│   └── utils
│       ├── __init__.py
│       ├── CustomLogger.py
│       └── TqdmLoggingHandler.py
├── LICENSE
├── README.rst
├── requirements.txt
├── setup.py
├── docs
└── tests

Here's the setup.py

from setuptools import setup, find_packages
with open("README.rst", "r") as fh:
  long_description = fh.read()

setup( 
  name ='fp-test-package', 
  version ='0.0.1', 
  description='Simple test building a CLI tool package',
  long_description=long_description,
  long_description_content_type='text/x-rst', 
  license ='GPLv2', 
  packages = find_packages(), 
  entry_points ={ 
    'console_scripts': [ 
      'fp_test_package = fp_test_package.py:main'
    ] 
  }, 
  classifiers =( 
    "Programming Language :: Python :: 3", 
    "License :: OSI Approved :: GNU General Public License v2", 
    "Operating System :: Linux", 
  ), 
  keywords ='test packaging fabiopipitone', 
  install_requires = ['tqdm>=4.49.0'], 
  zip_safe = False
) 

I tried following couple of howtos about how to build and publish a CLI python package, then I tried with sudo python3 setup.py install from inside the fp-test-package directory (same level of the setup.py). It seems to install the package (I can find the entry fp-test-package==0.0.1 in the pip3 freeze) but if I try fp-test-package in the terminal it returns fp-test-package: command not found, if I try fp_test_package it returns:

Traceback (most recent call last):
  File "/usr/local/bin/fp_test_package", line 11, in <module>
    load_entry_point('fp-test-package==0.0.1', 'console_scripts', 'fp_test_package')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'fp_test_package.py'

Then I went with sudo pip3 uninstall fp_test_package, which successfully uninstalled the package. I deleted the previously created build, dist and fp_test_package.egg-info directories and tried with pip3 install -e . (again from inside the fp-test-package). It created the fp_test_package.egg-info directory but again, running fp_test_package in the console it returns

Traceback (most recent call last):
  File "/usr/local/bin/fp_test_package", line 11, in <module>
    load_entry_point('fp-test-package==0.0.1', 'console_scripts', 'fp_test_package')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'fp_test_package.py'

Now, what am I missing? I'd like to simply package and install it so that when the user type fp-test-package or fp_test_package it returns the error about the --export_path argument being required (means the script's correctly started) like when I call it from the console:

$ python3 fp_test_package/fp_test_package.py
                                
usage: fp_test_package.py [-h] -ep EXPORT_PATH [-sa SECOND_ARGUMENT] [-ta THIRD_ARGUMENT]
fp_test_package.py: error: the following arguments are required: -ep/--export_path

How can I do it?

EDIT: I noticed what @Dustin then pointed out about the setup.py. In fact, a colleague of mine PRed the dummy repo changing the console_scripts part. Now, on my colleague machine everything seems to work as expected. On mine, after pulling the repo and rerun the python3 setup.py install, when calling fp_test_package from the console, it returns the following:

Traceback (most recent call last):
  File "/usr/local/bin/fp_test_package", line 11, in <module>
    load_entry_point('fp-test-package==0.0.1', 'console_scripts', 'fp_test_package')()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 490, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2854, in load_entry_point
    return ep.load()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2445, in load
    return self.resolve()
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 2451, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/bin/fp_test_package.py", line 4, in <module>
    __import__('pkg_resources').run_script('fp-test-package==0.0.1', 'fp_test_package.py')
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 667, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1452, in run_script
    raise ResolutionError(
pkg_resources.ResolutionError: Script 'scripts/fp_test_package.py' not found in metadata at '/usr/local/lib/python3.8/dist-packages/fp_test_package-0.0.1-py3.8.egg/EGG-INFO'

Any ideas?

Answer 1

Ok, I eventually found the problem on my specific case. What caused that cryptic error when installing with python3 setup.py install

pkg_resources.ResolutionError: Script 'scripts/fp_test_package.py' not found in metadata at '/usr/local/lib/python3.8/dist-packages/fp_test_package-0.0.1-py3.8.egg/EGG-INFO'

or the equivalent error when installing with pip3 install .

pkg_resources.ResolutionError: Script 'scripts/fp_test_package.py' not found in metadata at '/home/fabio/Desktop/test_python_package/fp-test-package/fp_test_package-0.0.1-py3.8.egg/EGG-INFO'

was the absence in the $PATH variable of the required path to properly launch the script.

Summing up, what I did was:

• editing the setup.py file according to the Dustin suggestions about the console_scripts
• building and installing the package running pip3 install . (or pip3 install -e . if you want the editable version) from inside the fp-test-package folder.
• checking where the binary of the script was created with which fp_test_package (in my case /home/fabio/.local/bin/fp_test_package)
• adding that path to the $PATH env var (in my .bashrc - and .zshrc since I use zsh - I added a line export PATH="$HOME/.local/bin:$PATH")

Now everything works as expected. I won't delete the correct version of the repo, so if anyone wants to use a working skeleton to start a project, it can be found there.

Answer 2

Your setup.py has:

  entry_points ={ 
    'console_scripts': [ 
      'fp_test_package = fp_test_package.py:main'
    ] 
  }, 

This is the equivalent of the following import:

from fp_test_package.py import main as fp_test_package

The problem is that you don't have a fp_test_package.py module, you have a fp_test_package module (directory with an __init__.py file) that contains a fp_test_package submodule (module names don't include the extension).

Assuming your fp_test_package.py file defines a main function, you could change this to:

  entry_points ={ 
    'console_scripts': [ 
      'fp_test_package = fp_test_package.fp_test_package:main'
    ] 
  }, 

Or you could move your main function to fp_test_package/__init__.py, so it could be:

  entry_points ={ 
    'console_scripts': [ 
      'fp_test_package = fp_test_package:main'
    ] 
  },