Lambda function AttributeError: module 'os' has no attribute 'add_dll_directory'

Question

I am encountering an issue with my AWS Lambda function with runtime Python 3.9. The function uses Spacy (version 3.7.2) and is set up as a layer in AWS Lambda. During execution, I'm facing the following error.

{
  "errorMessage": "module 'os' has no attribute 'add_dll_directory'",
  "errorType": "AttributeError",
  "requestId": "",
  "stackTrace": [
    "  File \"/var/lang/lib/python3.9/importlib/__init__.py\", line 127, in import_module\n    return _bootstrap._gcd_import(name[level:], package, level)\n",
    "  File \"<frozen importlib._bootstrap>\", line 1030, in _gcd_import\n",
    "  File \"<frozen importlib._bootstrap>\", line 1007, in _find_and_load\n",
    "  File \"<frozen importlib._bootstrap>\", line 986, in _find_and_load_unlocked\n",
    "  File \"<frozen importlib._bootstrap>\", line 680, in _load_unlocked\n",
    "  File \"<frozen importlib._bootstrap_external>\", line 850, in exec_module\n",
    "  File \"<frozen importlib._bootstrap>\", line 228, in _call_with_frames_removed\n",
    "  File \"/var/task/lambda_function.py\", line 3, in <module>\n    import thinc\n",
    "  File \"/opt/python/thinc/__init__.py\", line 2, in <module>\n    import numpy\n",
    "  File \"/opt/python/numpy/__init__.py\", line 126, in <module>\n    _delvewheel_patch_1_5_1()\n",
    "  File \"/opt/python/numpy/__init__.py\", line 114, in _delvewheel_patch_1_5_1\n    os.add_dll_directory(libs_dir)\n"
  ]
}

I've already attempted the following troubleshooting steps:

1. Updated Spacy to the latest version using pip install --upgrade spacy.

2. Disabled GPU support for Thinc in the Lambda function code using import thinc; thinc.require_gpu(False).

Despite these efforts, the issue persists. I suspect it might be related to the Lambda environment or the way dependencies are packaged.

Here are some specific questions I have:

1. Is there a known compatibility issue between Spacy 3.7.1 and Python 3.9 in AWS Lambda?

2. Are there specific configurations or dependencies that need to be addressed for Spacy in the AWS Lambda environment?

3. Could the "Unknown application error" be a result of the AttributeError related to os.add_dll_directory?

How to resolve this issue? If there are specific Lambda configurations or Spacy settings that need attention, please provide guidance.