I'm still very much learning Python and started using Mypy from quite early on to avoid forgetting to use correct types and check for things missing in my code. Up to now most things played well and I was astonished what the checker reads from the code and not to complain about.
Today I found some situation that puzzles me. With the following very simplified example code
from typing import Optional
myvar: Optional[int] = None
myvar = 42
if myvar < 1:
print("Error")
Mypy has nothing to complain because it obviously knows myvar
cannot be None
in the if
statement.
Strange enough if I use a TypedDict
this does not hold true as within the following piece:
from typing import Optional, TypedDict
class MyDictType(TypedDict):
mykey: Optional[int]
mydict1: MyDictType = {'mykey': None}
mydict1['mykey'] = 42
if mydict1['mykey'] < 1:
print("Error")
Here Mypy complains Unsupported operand types for > ("int" and "None")
and Left operand is of type "Optional[int]"
. It "forces" me to directly check like
if mydict1['mykey'] is not None and mydict1['mykey'] < 1:
print("Error")
Therefore my question is: Why doesn't Mypy recognize the very same thing (that mydict1['mykey']
can't be None
) with TypedDict
?
Your two examples are not identical and thus results in the different behavior, caused by assignment rules (and the lack of true
Optional
type for assignments in Python, more on this later) governing Python and thus Mypy. In the first example you have replacedmyvar
with a completely new assignment42
which has the typeint
, notOptional[int]
. In the second example there lacks a completely new assignment tomydict1
(as type checking is ultimately done against assignments at the underlying name - as the assignment of a new value to key'mykey'
only affects the opaque inner value insidemydict1
, without a material change to the assignment tomydict1
. This however only answers part of the question, as Mypy will correctly flag type mismatch issues where you to assign'forty-two'
tomyvar
, so what did Mypy really saw?To find out, let's use
reveal_type
helper from Mypy. Add it to the line immediately after the affected assignments:Running the above through Mypy (
mypy 1.8.0
)Note Mypy simply assume
myvar
became anint
in this instance and thus will not flag the simple comparison as an error (despite the fact that the underlying type annotation has not changed one bit, this may be verified by checkinglocals()['__annotations__']
at the relevant lines in the example scripts).As for the second example, do the same
reveal_type
check:The check will note that
mydict1['mykey']
maintains the typeUnion[builtins.int, None]
without dropping down tobuiltins.int
.Do note that
pyright
will have not flag any issues (pyright 1.1.351
) with either examples as it assumes the final values in both cases to beLiteral[42]
which is abuiltins.int
. I personally think that this behavior in both type checkers are incorrect as it may result in brittleness as a subtle change in the assignment with an actualOptional[int]
will reveal the lack of the guard againstNone
, which completely defeats the whole point of type checking in the first place, as you undoubtedly noted. To demonstrate this, instead of assigning a static value, let's change the assignment to the result of a function that's annotated to return anOptional[int]
(to emulate a subsequent code change later in the development cycle):This will then result in the expected error:
Should the return type of that function be changed to simply
int
(e.g.def answer() -> int:
), Mypy will once again consider the type assigned tomyvar
be simply just anint
, as Python does not have a realOptional
type like other languages where the assignment of a non-None
value must be encapsulated to be marked, so doingmyvar = 42
will be a syntax error. For example, your intended example with the conditional check has the equivalent code in Rust:Otherwise doing
myvar = 42
will simply result in "expected `Option<i64>`, found integer" - this is how an actual strongly typed languages deal with distinction between bareint
vsOptional[int]
(while also allowing comparison between different optional values explicitly without extra guard againstNone
whenever not necessary). This example unfortunately demonstrates how Python's type hinting currently is still quite brittle and limiting, and how this can potentially result in unexpected outcomes when code might be refactored later.