Aws Airflow 2.7.2 installing paramiko 3.3.1 error

Question

I'm upgrading my airflow from 1.10.12 to 2.7.2, python 3.6 to 3.11.7 and trying to install my libraries I get an error on Paramiko

enter image description here

but I have already tried to install bcrypt as it says dependencies compatibility but I get everytime this error for every version of it:

ERROR: bcrypt-3.2.0-cp36-abi3-musllinux_1_1_x86_64.whl is not a supported wheel on this platform.

ERROR: bcrypt-4.0.1-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl is not a supported wheel on this platform.

my requirements.txt file is like this:

enter image description here

I have tried changing the version, the system of the whl (Version of the operating system: 4.14.336-253.554.amzn2.x86_64) but always I get the same error that is not a supported wheel on this platform.

Answer 1

The command line interface (CLI) utility replicates an Amazon Managed Workflows for Apache Airflow environment locally.

The CLI builds a Docker container image locally that’s similar to an Amazon MWAA production image. This allows you to run a local Apache Airflow environment to develop and test DAGs, custom plugins, and dependencies before deploying to Amazon MWAA.

To run the CLI, see the aws-mwaa-local-runner on GitHub.

Using the local runner with this file.

--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.7.2/constraints-3.11.txt"
paramiko==3.3.1
xmlschema==3.0.1
elementpath==4.1.5
et_xmlfile==1.1.0
openpyxl==3.1.2
jmespath==0.10.0
s3transfer==0.6.2
six==1.16.0
botocore==1.31.17
boto3==1.28.17
bcrypt==4.0.1

test-requirements command builds successfully

--constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.7.2/constraints-3.11.txt"
Installing requirements.txt
Collecting paramiko==3.3.1
  Downloading paramiko-3.3.1-py3-none-any.whl (224 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 224.8/224.8 kB 3.3 MB/s eta 0:00:00
Collecting xmlschema==3.0.1
  Downloading xmlschema-3.0.1-py3-none-any.whl (405 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 405.2/405.2 kB 5.9 MB/s eta 0:00:00
Collecting elementpath==4.1.5
  Downloading elementpath-4.1.5-py3-none-any.whl (216 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 216.4/216.4 kB 9.5 MB/s eta 0:00:00
Collecting et_xmlfile==1.1.0
  Downloading et_xmlfile-1.1.0-py3-none-any.whl (4.7 kB)
Collecting openpyxl==3.1.2
  Downloading openpyxl-3.1.2-py2.py3-none-any.whl (249 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 250.0/250.0 kB 8.8 MB/s eta 0:00:00
Requirement already satisfied: jmespath==0.10.0 in ./.local/lib/python3.11/site-packages (from -r /usr/local/airflow/requirements/requirements.txt (line 7)) (0.10.0)
Requirement already satisfied: s3transfer==0.6.2 in ./.local/lib/python3.11/site-packages (from -r /usr/local/airflow/requirements/requirements.txt (line 8)) (0.6.2)
Requirement already satisfied: six==1.16.0 in ./.local/lib/python3.11/site-packages (from -r /usr/local/airflow/requirements/requirements.txt (line 9)) (1.16.0)
Requirement already satisfied: botocore==1.31.17 in ./.local/lib/python3.11/site-packages (from -r /usr/local/airflow/requirements/requirements.txt (line 10)) (1.31.17)        
Requirement already satisfied: boto3==1.28.17 in ./.local/lib/python3.11/site-packages (from -r /usr/local/airflow/requirements/requirements.txt (line 11)) (1.28.17)
Collecting bcrypt==4.0.1
  Downloading bcrypt-4.0.1-cp36-abi3-manylinux_2_28_x86_64.whl (593 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 593.7/593.7 kB 9.7 MB/s eta 0:00:00
Requirement already satisfied: cryptography>=3.3 in ./.local/lib/python3.11/site-packages (from paramiko==3.3.1->-r /usr/local/airflow/requirements/requirements.txt (line 2)) (41.0.4)
Collecting pynacl>=1.5
  Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 856.7/856.7 kB 11.6 MB/s eta 0:00:00
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in ./.local/lib/python3.11/site-packages (from botocore==1.31.17->-r /usr/local/airflow/requirements/requirements.txt (line 10)) (2.8.2)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in ./.local/lib/python3.11/site-packages (from botocore==1.31.17->-r /usr/local/airflow/requirements/requirements.txt (line 10)) (1.26.17)
Requirement already satisfied: cffi>=1.12 in ./.local/lib/python3.11/site-packages (from cryptography>=3.3->paramiko==3.3.1->-r /usr/local/airflow/requirements/requirements.txt (line 2)) (1.16.0)
Requirement already satisfied: pycparser in ./.local/lib/python3.11/site-packages (from cffi>=1.12->cryptography>=3.3->paramiko==3.3.1->-r /usr/local/airflow/requirements/requirements.txt (line 2)) (2.21)
Installing collected packages: et_xmlfile, elementpath, bcrypt, xmlschema, pynacl, openpyxl, paramiko
Successfully installed bcrypt-4.0.1 elementpath-4.1.5 et_xmlfile-1.1.0 openpyxl-3.1.2 paramiko-3.3.1 pynacl-1.5.0 xmlschema-3.0.1

aws mwaa best-practices-dependencies