Python: Private Types in Type Hints?

Question

I like type hints, especially for my method parameters. In my current script one function should retrieve a parameter of type argparse._SubParsersAction. As one can see from the underscore, this is a private type by convention. I guess this is why PyCharm complains with the error message Cannot find reference '_SubParsersAction' in 'argparse.pyi' when trying to import it (although it's there).

The script runs but it feels wrong. The error message seems reasonable to me as private types are meant to be... well, private. My first question is therefore why the public method ArgumentParser.add_subparsers() returns an object of a private type in the first place.

I've looked for a public super class or interface and _SubParsersAction does indeed extend from argparse.Action, but that doesn't help me as Action does not define _SubParsersAction's add_parser() method (which I need).

So my next questions are: Can I use type hints with the argparse API? Or is it only partially possible because the API was designed long before type hints were introduced? Or does my idea of typing not fit to Python's type system?

Here is my affected code snippet. It creates an argument parser with sub arguments as described in the documentation (https://docs.python.org/dev/library/argparse.html#sub-commands).

main.py

from argparse import ArgumentParser

import sub_command_foo
import sub_command_bar
import sub_command_baz


def main():
    parser = ArgumentParser()
    sub_parsers = parser.add_subparsers()

    sub_command_foo.add_sub_parser(sub_parsers)
    sub_command_bar.add_sub_parser(sub_parsers)
    sub_command_baz.add_sub_parser(sub_parsers)

    args = parser.parse_args()
    args.func(args)


if __name__ == '__main__':
    main()

sub_command_foo.py

from argparse import _SubParsersAction, Namespace


def add_sub_parser(sub_parsers: _SubParsersAction):
    arg_parser = sub_parsers.add_parser('foo')
    
    # Add arguments...

    arg_parser.set_defaults(func=run)


def run(args: Namespace):
    print('foo...')

The problem lies in sub_command_foo.py. PyCharm shows the error message Cannot find reference '_SubParsersAction' in 'argparse.pyi' on the first line from argparse import _SubParsersAction, Namespace.

Answer 1

I don't use pycharm and haven't done much with type hints so can't help your there. But I know argparse well. The bulk of this module was written in before 2010, and it's been modified since then at a snail's pace. It's well organized in the OOP sense, but the documentation is more of glorified tutorial than a formal reference. That is, it focuses on the functions and methods users will needed, and doesn't try to formally document all classes and the methods.

I do a lot of my testing in an interactive ipython session where I can look at the objects returned by commands.

In Python the distinction between public and private classes and methods is not as formal as other languages. The '_' prefix does mark 'private' things. They aren't usually documented, but they are still accessible. Sometimes a '*' import will import all object that don't start with it, but argparse has a more explicit __all__ list.

argparser.ArgumentParser does create an object instance. But that class inherits from 2 'private' classes. The add_argument method creates an Action object, and puts it on the parser._actions list. It also returns it to the user (though usually that reference is ignored). The action is actually a subclass (all of which are 'private'). add_subparsers is just a specialized version of this add_argument.

The add_parser method creates a ArgumentParser object

It feels to me that type hinting that requires a separate import of 'private' classes is counter productive. You shouldn't be explicitly referencing those classes, even if your code produces them. Some people (companies) fear they can be changed without notification and thus break their code. Knowing how slowly argparse gets changed I wouldn't too much about that.