Why is 'pip3 install netifaces' failing on Debian 10 Buster?

Question

I setup a new Debian 10 (Buster) instance on AWS EC2, and was able to install a pip3 package that depended on netifaces, but when I came back to it the next day the package is breaking reporting an error in netifaces. If I try to run pip3 install netifaces I get the same error:

~$ pip3 install netifaces
Collecting netifaces
  Using cached https://files.pythonhosted.org/packages/0d/18/fd6e9c71a35b67a73160ec80a49da63d1eed2d2055054cc2995714949132/netifaces-0.10.9.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 20, in <module>
        from setuptools.dist import Distribution, Feature
      File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 35, in <module>
        from setuptools.depends import Require
      File "/usr/lib/python3/dist-packages/setuptools/depends.py", line 7, in <module>
        from .py33compat import Bytecode
      File "/usr/lib/python3/dist-packages/setuptools/py33compat.py", line 55, in <module>
        unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
    AttributeError: 'HTMLParser' object has no attribute 'unescape'

Answer 1

HTMLParser().unescape was removed in Python 3.9. Compare the code in Python 3.8 vs Python 3.9.

The error seems to be a bug in setuptools. Try to upgrade setuptools. Or use Python 3.8.

Answer 2

I had python3.6 and related packages through deb management. Needed python3.9 for side project and the solution to fix pip and AttributeError: 'HTMLParser' object has no attribute 'unescape' was to update pip for python3.9 locally for one user:

python3.9 -m pip install --upgrade pip

now installing python3.9 version of the pip-packages work:

python3.9 -m pip install --target=~/.local/lib/python3.9/site-packages numpy

Answer 3

Downgrading to any older python3 version is not the solution and most of the time upgrading setuptools won't fix the issue. The proper solution that worked for me to work with pip using python3.9 is the following on Ubuntu18: locate /usr/lib/python3/dist-packages/setuptools/py33compact.py33 and change

# unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)  # comment out this line
unescape = getattr(html, 'unescape', None)
if unescape is None:
    # HTMLParser.unescape is deprecated since Python 3.4, and will be removed
    # from 3.9.
    unescape = html_parser.HTMLParser().unescape

Answer 4

I was facing this issue in PyCharm 2018. Apart from upgrading setuptools as mentioned above, I also had to upgrade to PyCharm 2020.3.4 to solve this issue. Related bug on PyCharm issue tracker: https://youtrack.jetbrains.com/issue/PY-39579

Hope this helps someone avoid spending hours trying to debug this.