How do I properly import modules from custom packages into an AWS Lambda function?

Question

I am new to AWS and need some advice. I have a project with several AWS Lambda functions. In my case, there are four and they are located in the functions folder. They all need to use a database connection. So I decided to put the code related to databases in a separate package called databases and use it in AWS Lambda Layers. As you can see, the databases package has two modules. As I understand from the documentation, I need to archive my package and put it in AWS Lambda Layers. However, the address of this package changes. Since AWS puts it in the /opt directory by default. I'm a little confused. How do I properly import modules from custom packages into an AWS Lambda function so that it works locally and in production?

The structure of my project looks like this:

src
   functions
      create_user_information
         __init__.py
         lambda_function.py
      update_user_information
         __init__.py
         lambda_function.py
      get_user_information
         __init__.py
         lambda_function.py
      delete_user_information
         __init__.py
         lambda_function.py
   layers
      databases
         __init__.py
         cassandra.py
         postgresql.py
         requirements.txt
template.yaml
venv
   bin
      ...
   include
   lib
      ...

lambda_function.py:

from src.layers.databases import cassandra


cassandra_db_session = None
cassandra_db_username = 'your-username'
cassandra_db_password = 'your-password'
cassandra_db_endpoints = ['your-endpoint']
cassandra_db_port = 9142


def lambda_handler(event, context):
    global cassandra_db_session
    if not cassandra_db_session:
        cassandra_db_session = cassandra.create_session(
            cassandra_db_username,
            cassandra_db_password,
            cassandra_db_endpoints,
            cassandra_db_port
        )
    # Some business logic.
    # ...
    return "AWS Lambda function finished."

template.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: User Information Service
Globals:
    Function:
        Timeout: 10
Resources:
    Databases:
        Type: AWS::Serverless::LayerVersion
        Properties:
            LayerName: Databases
            Description:
            ContentUri:
            CompatibleRuntimes:
                - python3.8
            LicenseInfo: MIT
            RetentionPolicy: Retain
    GetUserInformation:
        Type: AWS::Serverless::Function
        Properties:
            FunctionName: GetUserInformation
            Description:
            CodeUri: src/functions/get_user_information
            Handler: lambda_function.lambda_handler
            Runtime: python3.8
            Layers:
                - !Ref Databases
    CreateUserInformation:
        Type: AWS::Serverless::Function
        Properties:
            FunctionName: CreateUserInformation
            Description:
            CodeUri: src/functions/create_user_information
            Handler: lambda_function.lambda_handler
            Runtime: python3.8
            Layers:
                - !Ref Databases
    UpdateUserInformation:
        Type: AWS::Serverless::Function
        Properties:
            FunctionName: UpdateUserInformation
            Description:
            CodeUri: src/functions/update_user_information
            Handler: lambda_function.lambda_handler
            Runtime: python3.8
            Layers:
                - !Ref Databases
    DeleteUserInformation:
        Type: AWS::Serverless::Function
        Properties:
            FunctionName: DeleteUserInformation
            Description:
            CodeUri: src/functions/delete_user_information
            Handler: lambda_function.lambda_handler
            Runtime: python3.8
            Layers:
                - !Ref Databases
Outputs:
    DatabasesARN:
        Value: !Ref Databases
        Description: Databases ARN
        Export:
            Name: databases-arn

Answer 1

Let's assume you've packaged databases and installed it correctly in the layer, then your import is going to be as simple as:

# assume 'databases' is the installation name of your pkg
from databases import cassandra  

1. Your package must be "installable" for the code above to work, which it doesn't see to. There are lots of references on the web about how to do it right.

It's important to notice you don't need to follow this approach (besides being recommended). You can package the whole src folder and upload as your lambda function, as a consequence you'd be able to use as follows:

# don't use relative imports in lambda because the change to opt/
from src.layers.databases import cassandra  

The current problem is not caused by any sort of "directory change" but instead by incorrect packaging.