How to use type hints in python 3.6?

Question

I noticed Python 3.5 and Python 3.6 added a lot of features about static type checking, so I tried with the following code (in python 3.6, stable version).

from typing import List

a: List[str] = []
a.append('a')
a.append(1)
print(a)

What surprised me was that, Python didn't give me an error or warning, although 1 was appended to a list which should only contain strings. Pycharm detected the type error and gave me a warning about it, but it was not obvious and it was not shown in the output console, I was afraid sometimes I might miss it. I would like the following effects:

1. If it's obvious that I used the wrong type just as shown above, throw out a warning or error.
2. If the compiler couldn't reliably check if the type I used was right or wrong, ignore it.

Is that possible? Maybe mypy could do it, but I'd prefer to use Python-3.6-style type checking (like a: List[str]) instead of the comment-style (like # type List[str]) used in mypy. And I'm curious if there's a switch in native python 3.6 to achieve the two points I said above.

Answer 1

Is that possible? Maybe mypy could do it, but I'd prefer to use Python-3.6-style type checking (like a: List[str]) instead of the comment-style (like # type: List[str]) used in mypy. And I'm curious if there's a switch in native python 3.6 to achieve the two points I said above.

There's no way Python will do this for you; you can use mypy to get type checking (and PyCharms built-in checker should do it too). In addition to that, mypy also doesn't restrict you to only type comments # type List[str], you can use variable annotations as you do in Python 3.6 so a: List[str] works equally well.

With mypy as is, because the release is fresh, you'll need to install typed_ast and execute mypy with --fast-parser and --python-version 3.6 as documented in mypy's docs. This will probably change soon but for now you'll need them to get it running smoothly

Update: --fast-parser and --python-version 3.6 aren't needed now.

After you do that, mypy detects the incompatibility of the second operation on your a: List[str] just fine. Let's say your file is called tp_check.py with statements:

from typing import List

a: List[str] = []
a.append('a')
a.append(1)
print(a)

Running mypy with the aforementioned arguments (you must first pip install -U typed_ast):

python -m mypy --fast-parser --python-version 3.6 tp_check.py

catches the error:

tp_check.py:5: error: Argument 1 to "append" of "list" has incompatible type "int"; expected "str"

As noted in many other answers on type hinting with Python, mypy and PyCharms' type-checkers are the ones performing the validation, not Python itself. Python doesn't use this information currently, it only stores it as metadata and ignores it during execution.

Answer 2

Type annotations in Python are not meant to be type-enforcing. Anything involving runtime static-type dependency would mean changes so fundamental that it would not even make sense to continue call the resulting language "Python".

Notice that the dynamic nature of Python does ALLOW for one to build an external tool, using pure-python code, to perform runtime type checking. It would make the program run (very) slowly, but maybe it is suitable for certain test categories.

To be sure - one of the fundamentals of the Python language is that everything is an object, and that you can try to perform any action on an object at runtime. If the object fails to have an interface that conforms with the attempted operation, it will fail - at runtime.

Languages that are by nature statically typed work in a different way: operations simply have to be available on objects when tried at run time. At the compile step, the compiler creates the spaces and slots for the appropriate objects all over the place - and, on non-conforming typing, breaks the compilation.

Python's typechecking allows any number of tools to do exactly that: to break and warn at a step prior to actually running the application (but independent from the compiling itself). But the nature of the language can't be changed to actually require the objects to comply in runtime - and veryfying the typing and breaking at the compile step itself would be artificial.

Although, one can expect that future versions of Python may incoroporate compile-time typechecking on the Python runtime itself - most likely through and optional command line switch. (I don't think it will ever be default - at least not to break the build - maybe it can be made default for emitting warnings)

So, Python does not require static type-checking at runtime because it would cease being Python. But at least one language exists that makes use both of dynamic objects and static typing - the Cython language, that in practice works as a Python superset. One should expect Cython to incorporate the new type-hinting syntax to be actual type-declaring very soon. (Currently it uses a differing syntax for the optional statically typed variables)

Answer 3

Type hints are entirely meant to be ignored by the Python runtime, and are checked only by 3rd party tools like mypy and Pycharm's integrated checker. There are also a variety of lesser known 3rd party tools that do typechecking at either compile time or runtime using type annotations, but most people use mypy or Pycharm's integrated checker AFAIK.

In fact, I actually doubt that typechecking will ever be integrated into Python proper in the foreseable future -- see the 'non-goals' section of PEP 484 (which introduced type annotations) and PEP 526 (which introduced variable annotations), as well as Guido's comments here.

I'd personally be happy with type checking being more strongly integrated with Python, but it doesn't seem the Python community at large is ready or willing for such a change.

The latest version of mypy should understand both the Python 3.6 variable annotation syntax and the comment-style syntax. In fact, variable annotations were basically Guido's idea in the first place (Guido is currently a part of the mypy team) -- basically, support for type annotations in mypy and in Python was developed pretty much simultaneously.

Answer 4

Starting in python3.7 you could take advantage of the type_enforced module. Nested type checking is supported in type_enforced for python3.9+.

While it does not address your explicit example, it does support type checking for functions and methods.

As an example:

pip install type_enforced

import type_enforced
from typing import List

@type_enforced.Enforcer
def my_fn(a:List[str]):
    a.append('a')

a=[]
b={}
my_fn(a)
my_fn(a)
print(a) # => ['a','a']
my_fn(b) # Raises error - see below

Outputs:

['a', 'a']
Traceback (most recent call last):
  File "/Users/conmak/development/personal/type_enforced/test.py", line 13, in <module>
    my_fn(b) # Raises error
    ^^^^^^^^
  File "/Users/conmak/development/personal/type_enforced/type_enforced/enforcer.py", line 171, in __call__
    self.__check_type__(assigned_vars.get(key), value, key)
  File "/Users/conmak/development/personal/type_enforced/type_enforced/enforcer.py", line 188, in __check_type__
    self.__exception__(
  File "/Users/conmak/development/personal/type_enforced/type_enforced/enforcer.py", line 129, in __exception__
    raise TypeError(f"({self.__fn__.__qualname__}): {message}")
TypeError: (my_fn): Type mismatch for typed variable `a`. Expected one of the following `[<class 'list'>]` but got `<class 'dict'>` instead.

Answer 5

The pydantic package has a validate_arguments decorator that checks type hints at runtime. You can add this decorator to all functions or methods where you want type hints enforced.

I wrote some code to help automate this for an entire module, so that I could enable runtime checks for my test suite to help with debugging, but then have them off for code that uses the library so there's no performance impact.

import sys
import inspect
import types
from pydantic import validate_arguments

class ConfigAllowArbitraryTypes:
    """allows for custom classes to be used in type hints"""
    
    arbitrary_types_allowed = True

def add_runtime_type_checks(module):
    """adds runtime typing checks to the given module/class"""

    if isinstance(module, str):
        module = sys.modules[module]
    
    for attr in module.__dict__:
        obj = getattr(module, attr)
        
        if isinstance(obj, types.FunctionType):
            setattr(module, attr, validate_arguments(obj, config=ConfigAllowArbitraryTypes))
        elif inspect.isclass(obj):
            # recursively add decorator to class methods
            add_runtime_type_checks(obj)

In my test suite I then add decorators by calling add_runtime_type_checks with the name of the module.

import mymodule

def setup_module(module):
    """executes once"""
    
    add_runtime_type_checks('mymodule')

def test_behavior():
   ...

Note that with pydantic it might do some unexpected conversions when type checking, so if the desired type is an int and you pass the function 0.2, it will cast it to 0 silently rather than failing. In principle, you could do almost the same thing with the typen library's enforce_type_hints decorator, but that does not do recursive checks (so you can't use types like list[int], only list).