TechQA.

python-oracledb module fails to use instantclient in a docker container

364 views Asked by At

I'm trying to build a docker image to access an oracle database at runtime, I'm having the following error message: DPI-1047: Cannot locate a 64-bit Oracle Client library: "/oracle/instantclient/libclntsh.so: cannot open shared object file: No such file or directory".

Inside the container, there actually is a symbolic link at /oracle/instantclient/libclntsh.so

Dockerfile:

FROM python:3.12-slim

RUN apt-get update && \
    apt-get install -y wget unzip libaio1

ARG ORACLE_HOME=/oracle
ARG ORACLE_CLIENT_HOME=${ORACLE_HOME}/instantclient

# Download and install Oracle instantclient
RUN mkdir /tmp/oracle && \
    wget https://download.oracle.com/otn_software/linux/instantclient/1920000/instantclient-basic-linux.x64-19.20.0.0.0dbru.zip -P /tmp/oracle && \
    unzip /tmp/oracle/instantclient-basic-* -d /tmp/oracle && \
    mkdir ${ORACLE_HOME} && \
    mv /tmp/oracle/instantclient_* ${ORACLE_CLIENT_HOME}

ENV LD_LIBRARY_PATH="${ORACLE_CLIENT_HOME}"

RUN pip install --upgrade pip && \
    pip install pipenv
ENV PIPENV_VENV_IN_PROJECT=1

WORKDIR /app

ADD main.py Pipfile Pipfile.lock ./

RUN pipenv sync

ENTRYPOINT ["./.venv/bin/python", "main.py"]
CMD [""]

main.py:

import os

import oracledb


def print_db_version(db_config):
    params = oracledb.ConnectParams(host=db_config['host'], port=db_config['port'], service_name=db_config['name'])
    with oracledb.connect(user=db_config['username'], password=db_config['password'], params=params) as conn:
        print(f'Database version: {conn.version}')
        conn.close()


if __name__ == '__main__':
    # Both calls below fail...
    # oracledb.init_oracle_client()
    oracledb.init_oracle_client(os.environ['LD_LIBRARY_PATH'])

    db_config = {
        'host': os.environ['DB_HOST'],
        'port': os.environ['DB_PORT'],
        'name': os.environ['DB_NAME'],
        'username': os.environ['DB_USERNAME'],
        'password': os.environ['DB_PASSWORD'],
    }

    print_db_version(db_config)

Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
oracledb = "1.4.2"

command lines (last one allows to explore the container):

docker build -t my-version .
docker run my-version
docker run -it --entrypoint "" my-version bash

I can not figure out why this error pops up while the library is actually installed in my container... any ideas?

EDIT

I tried Anthony Tuininga suggestions and had the following output:

ODPI [00001] 2023-10-21 19:13:32.206: ODPI-C 5.0.1
ODPI [00001] 2023-10-21 19:13:32.206: debugging messages initialized at level 64
ODPI [00001] 2023-10-21 19:13:32.206: Context Parameters:
ODPI [00001] 2023-10-21 19:13:32.206:     Oracle Client Lib Dir: /oracle/instantclient
ODPI [00001] 2023-10-21 19:13:32.206: Environment Variables:
ODPI [00001] 2023-10-21 19:13:32.206:     LD_LIBRARY_PATH => "/oracle/instantclient"
ODPI [00001] 2023-10-21 19:13:32.206: load in parameter directory
ODPI [00001] 2023-10-21 19:13:32.206: load in dir /oracle/instantclient
ODPI [00001] 2023-10-21 19:13:32.206: load with name /oracle/instantclient/libclntsh.so
ODPI [00001] 2023-10-21 19:13:32.206: load by OS failure: /oracle/instantclient/libclntsh.so: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-21 19:13:32.206: load with name /oracle/instantclient/libclntsh.so.19.1
ODPI [00001] 2023-10-21 19:13:32.207: load by OS failure: /oracle/instantclient/libclntsh.so.19.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-21 19:13:32.207: load with name /oracle/instantclient/libclntsh.so.18.1
ODPI [00001] 2023-10-21 19:13:32.207: load by OS failure: /oracle/instantclient/libclntsh.so.18.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-21 19:13:32.207: load with name /oracle/instantclient/libclntsh.so.12.1
ODPI [00001] 2023-10-21 19:13:32.207: load by OS failure: /oracle/instantclient/libclntsh.so.12.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-21 19:13:32.207: load with name /oracle/instantclient/libclntsh.so.11.1
ODPI [00001] 2023-10-21 19:13:32.207: load by OS failure: /oracle/instantclient/libclntsh.so.11.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-21 19:13:32.207: load with name /oracle/instantclient/libclntsh.so.20.1
ODPI [00001] 2023-10-21 19:13:32.207: load by OS failure: /oracle/instantclient/libclntsh.so.20.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-21 19:13:32.207: load with name /oracle/instantclient/libclntsh.so.21.1
ODPI [00001] 2023-10-21 19:13:32.207: load by OS failure: /oracle/instantclient/libclntsh.so.21.1: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/app/main.py", line 18, in <module>
['libocci.so.19.1', 'libnnz19.so', 'adrci', 'libipc1.so', 'xstreams.jar', 'libclntsh.so.11.1', 'libclntsh.so.18.1', 'genezi', 'libocci.so.12.1', 'network', 'libocci.so.10.1', 'libocci.so', 'libociei.so', 'libclntsh.so', 'libclntsh.so.12.1', 'libocci.so.18.1', 'libclntsh.so.19.1', 'ucp.jar', 'BASIC_LICENSE', 'libocijdbc19.so', 'ojdbc8.jar', 'BASIC_README', 'libmql1.so', 'liboramysql19.so', 'libocci.so.11.1', 'libclntshcore.so.19.1', 'libclntsh.so.10.1', 'uidrvci']
    oracledb.init_oracle_client(os.environ['LD_LIBRARY_PATH'])
  File "src/oracledb/impl/thick/utils.pyx", line 476, in oracledb.thick_impl.init_oracle_client
  File "src/oracledb/impl/thick/utils.pyx", line 500, in oracledb.thick_impl.init_oracle_client
  File "src/oracledb/impl/thick/utils.pyx", line 421, in oracledb.thick_impl._raise_from_info
oracledb.exceptions.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "/oracle/instantclient/libclntsh.so: cannot open shared object file: No such file or directory". See https://python-oracledb.readthedocs.io/en/latest/user_guide/initialization.html for help

It shows that:

  • LD_LIBRARY_PATH is properly set to /oracle/instantclient
  • the instant client is actually searched in this directory
  • /oracle/instantclient actually contains libclntsh.so file (it's actually a symbolic link to libclntsh.so.19.1

It sounds pretty weird to me...

I made the source code available here: https://github.com/galak75/python-oracle-img

EDIT 2

I tried to use the bare call to init_oracle_client(), and then it looks like LD_LIBRARY_PATH variable is not used: instant client is search in my python virtualenv:

ODPI [00001] 2023-10-22 14:48:34.681: ODPI-C 5.0.1
ODPI [00001] 2023-10-22 14:48:34.681: debugging messages initialized at level 64
ODPI [00001] 2023-10-22 14:48:34.681: Context Parameters:
ODPI [00001] 2023-10-22 14:48:34.681: Environment Variables:
ODPI [00001] 2023-10-22 14:48:34.681:     LD_LIBRARY_PATH => "/oracle/instantclient"
ODPI [00001] 2023-10-22 14:48:34.681: check module directory
ODPI [00001] 2023-10-22 14:48:34.681: module name is /app/.venv/lib/python3.12/site-packages/oracledb/thick_impl.cpython-312-aarch64-linux-gnu.so
ODPI [00001] 2023-10-22 14:48:34.681: load in dir /app/.venv/lib/python3.12/site-packages/oracledb
ODPI [00001] 2023-10-22 14:48:34.681: load with name /app/.venv/lib/python3.12/site-packages/oracledb/libclntsh.so
ODPI [00001] 2023-10-22 14:48:34.681: load by OS failure: /app/.venv/lib/python3.12/site-packages/oracledb/libclntsh.so: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.681: load with OS search heuristics
ODPI [00001] 2023-10-22 14:48:34.681: load with name libclntsh.so
ODPI [00001] 2023-10-22 14:48:34.682: load by OS failure: libaio.so.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.682: load with name libclntsh.so.19.1
ODPI [00001] 2023-10-22 14:48:34.682: load by OS failure: libaio.so.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.682: load with name libclntsh.so.18.1
ODPI [00001] 2023-10-22 14:48:34.682: load by OS failure: libaio.so.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.682: load with name libclntsh.so.12.1
ODPI [00001] 2023-10-22 14:48:34.683: load by OS failure: libaio.so.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.683: load with name libclntsh.so.11.1
ODPI [00001] 2023-10-22 14:48:34.683: load by OS failure: libaio.so.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.683: load with name libclntsh.so.20.1
ODPI [00001] 2023-10-22 14:48:34.683: load by OS failure: libclntsh.so.20.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.683: load with name libclntsh.so.21.1
ODPI [00001] 2023-10-22 14:48:34.683: load by OS failure: libclntsh.so.21.1: cannot open shared object file: No such file or directory
ODPI [00001] 2023-10-22 14:48:34.683: check ORACLE_HOME
Traceback (most recent call last):
  File "/app/main.py", line 19, in <module>
    oracledb.init_oracle_client()
  File "src/oracledb/impl/thick/utils.pyx", line 476, in oracledb.thick_impl.init_oracle_client
LD_LIBRARY_PATH = /oracle/instantclient
['libocci.so.19.1', 'libnnz19.so', 'adrci', 'xstreams.jar', 'libclntsh.so.11.1', 'libclntsh.so.18.1', 'genezi', 'libocci.so.12.1', 'network', 'libocci.so.10.1', 'libocci.so', 'libociei.so', 'libclntsh.so', 'libclntsh.so.12.1', 'libocci.so.18.1', 'libclntsh.so.19.1', 'ucp.jar', 'BASIC_LICENSE', 'libocijdbc19.so', 'ojdbc8.jar', 'BASIC_README', 'liboramysql19.so', 'libocci.so.11.1', 'libclntshcore.so.19.1', 'libclntsh.so.10.1', 'uidrvci']
  File "src/oracledb/impl/thick/utils.pyx", line 500, in oracledb.thick_impl.init_oracle_client
  File "src/oracledb/impl/thick/utils.pyx", line 421, in oracledb.thick_impl._raise_from_info
oracledb.exceptions.DatabaseError: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libaio.so.1: cannot open shared object file: No such file or directory". See https://python-oracledb.readthedocs.io/en/latest/user_guide/initialization.html for help

EDIT 3

I added libaio1 package installation to my code, since it was clearly the next issue (after the target OS platform issue), as pointed out by Christopher's answer

python oracle docker instantclient python-oracledb
Original Q&A
3

There are 3 answers

2
Géraud On BEST ANSWER

I finally figured it out, after so many hours struggling!

I'm running on a MacBook with an M1 chip (arm64 architecture), while I'm installing an instantclient distribution for amd64 architectures...

then I made it work by forcing the target docker platform to amd64:

docker build -t my-version . --platform amd64
docker run --env-file .local/env my-version

Note: I also had to fix few issues like installing libaio1 debian package, and removing the conn.close() python statement

EDIT: Another solution is to download the right instant client package depending on the target OS platform:

RUN mkdir /tmp/oracle && \
    if [ "`uname -m`" = "aarch64" ] || [ "`uname -m`" = "arm64" ]; \
      then echo "ARM64 architecture detected" && wget https://download.oracle.com/otn_software/linux/instantclient/1919000/instantclient-basic-linux.arm64-19.19.0.0.0dbru.zip -P /tmp/oracle; \
    elif [ "`uname -m`" = "amd64" ] || [ "`uname -m`" = "x86_64" ] ; \
      then echo "AMD64 architecture detected" && wget https://download.oracle.com/otn_software/linux/instantclient/1920000/instantclient-basic-linux.x64-19.20.0.0.0dbru.zip -P /tmp/oracle; \
    else  \
      echo "target platform was not recognized : `uname -a`" && exit 1; \
    fi; \
    unzip /tmp/oracle/instantclient-basic-* -d /tmp/oracle && \
    mkdir -p ${ORACLE_CLIENT_HOME} && \
    mv /tmp/oracle/instantclient_*/* ${ORACLE_CLIENT_HOME}
3
Anthony Tuininga On

If you do not require thick mode you can avoid the error completely by simply removing the call to init_oracle_client().

If you do require it because of limitations in thin mode, set the environment variable DPI_DEBUG_LEVEL to the value 64 before running your script. That may help you diagnose the issue. If not, include the output in your question and I'll take a look.

[UPDATE: with the output provided it is clear that the aio library is missing as also suggested by Chris in his answer. Add that library and your issues should disappear!]

It may also be helpful to list the files found in the directory using os.listdir(os.environ["LD_LIBRARY_PATH"]).

Note that the library directory cannot be specified on Linux so you must use the bare call to init_oracle_client().

3
Christopher Jones On

For reference, see Docker for Oracle Database Applications in Node.js and Python.

When I tried your Dockerfile I got a fairly clear message that the libaio package wasn't installed.

The next issue (other than the missing lock file!) was that main.py is using a context manager for the connection but also explicitly closing the connection.

This Dockerfile works for me:

FROM python:3.12-slim

RUN apt-get update && \
    apt-get install -y wget unzip libaio1

ARG ORACLE_CLIENT_HOME=/opt/oracle/instantclient

WORKDIR /opt/oracle

# Download and install Oracle instantclient
RUN mkdir /tmp/oracle && \
    wget https://download.oracle.com/otn_software/linux/instantclient/1921000/instantclient-basic-linux.x64-19.21.0.0.0dbru.zip -P /tmp/oracle && \
    unzip /tmp/oracle/instantclient-basic-* -d /tmp/oracle && \
    mv /tmp/oracle/instantclient_* ${ORACLE_CLIENT_HOME} && \
    cd ${ORACLE_CLIENT_HOME} && rm -f *jdbc* *occi* *mysql* *jar uidrvci genezi adrci && \
    echo ${ORACLE_CLIENT_HOME} > /etc/ld.so.conf.d/oracle-instantclient.conf && \
    ldconfig

RUN pip install --upgrade pip && pip install oracledb

WORKDIR /app
ADD main.py ./

CMD ["python", "main.py"]

Note I removed the overloaded use of ORACLE_HOME, since this has a specific meaning in Oracle. I also just used python directly, since presumably the container is only going to run one app environment.

I also prefer ldconfig to LD_LIBRARY_PATH since the latter is prone to various issues.

main.py is:

import os

import oracledb


def print_db_version(db_config):
    params = oracledb.ConnectParams(host=db_config['host'], port=db_config['port'], service_name=db_config['name'])
    with oracledb.connect(user=db_config['username'], password=db_config['password'], params=params) as conn:
        print(f'Database version: {conn.version}')

if __name__ == '__main__':
    oracledb.init_oracle_client()

    db_config = {
        'host': 'whatever',
        'port': 1521,
        'name': 'orclpdb1',
        'username': 'cj',
        'password': 'cj',
    }

    print_db_version(db_config)

Follow Anthony's advice about not passing lib_dir to init_oracle_client() on Linux.

Output is like:

$ docker run my-version
Database version: 21.3.0.0.0

Finally you should revisit which user you are running in the container.

Related Questions in PYTHON

Related Questions in ORACLE

Related Questions in DOCKER

Related Questions in INSTANTCLIENT

Related Questions in PYTHON-ORACLEDB

Popular Questions

Popular Tags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json python-3.x ruby-on-rails .net sql-server swift django angular objective-c pandas excel

Trending Questions