Unable to run pytest command in a lambda function on AWS

Question

I have the following lambda function on AWS

import os
import sys
sys.path.insert(0, '/opt')


def tc1(event, context):
    print("in tc1")
    
    os.system("pytest /tests/Test_preRequisites.py -s -v")

    os.system("python -m pytest /tests/Test_preRequisites.py -s -v")

when I run this function, the following error is displayed

Response:
null

Request ID:
"8e8738b7-9h28-4379-b814-688da8c31d58"

Function logs:
START RequestId: 8e8738b7-9h28-4379-b814-688da8c31d58 Version: $LATEST
in tc1
sh: pytest: command not found
/var/lang/bin/python: No module named pytest
END RequestId: 8e8738b7-9h28-4379-b814-688da8c31d58
REPORT RequestId: 8e8738b7-9h28-4379-b814-688da8c31d58  Duration: 38.46 ms  Billed Duration: 100 ms Memory Size: 2048 MB    Max Memory Used: 57 MB  Init Duration: 123.66 ms    

I can understand that the lambda function is unable to find the pytest module from these errors sh: pytest: command not found and /var/lang/bin/python: No module named pytest

I have tried to run the pytest command and also the python -m pytest command, both both give the same error.

However, I have already added a zip file as a layer and added that layer to this lambda function.

I installed pytest on my local machine to a folder by the command pip install pytest -t C:\Users\admin\dependencies and then zipped the contents of that folder and uploaded it to the layer on AWS. Still I am unable to access the pytest module.

This works perfectly fine on my local machine on local environment. This issue is occurring for AWS lambda only, so the script is working fine.

Can anyone please let me know what needs to be added or modified here to get this working. Thanks.

Answer 1

Place your dependencies in a 'python' directory for Python layers, like this:

pip install pytest -t C:\Users\admin\dependencies\python

then zip up the contents of the 'dependencies' folder as before. The zip file will contain a single directory, 'python' with your dependencies under it.

Answer 2

This is because there's no entry point in the Lambda environment. When you install pytest normally, you get a pytest script due to the project's options.entry_points value in its setup.cfg ( found here: https://github.com/pytest-dev/pytest/blob/main/setup.cfg )

If you install the package into a virtualenv and navigate to the /bin directory, you'll see a pytest script sitting in there. That's what's normally being executed when you invoke the pytest command on the CLI. Your Lambda needs a version of that script, if you want to be able to shell out to it.

For reference, here's what's in that script:

#!/path/to/venv/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from pytest import console_main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(console_main())

I have not verified it myself, but I suspect that changing the shebang to #!/usr/bin/env python in this script would cause it to work from within the Lambda. Also, note that since your dependencies typically end up dumped into the same directory as your code in a Lambda package, you may need to use a different name for the script (because the name pytest is already used by a directory)