A conda activated virtual enviroment is not running the Python file when called from a flask webservice

Question

I have created a virutal environment using conda, and am trying to run a python script in it using a flask end point call. The code is not executing , whereas if I run the file from terminal by getting inside the virtual environment it runs in less than a minute. Any idea what am I missing?

My venv name: py311

My calls are:

import flask
import requests
import json
import urllib3
import urllib.parse
from flask import request, jsonify,make_response

import subprocess

app = flask.Flask(__name__)
app.config["DEBUG"] = True


@app.route('/vl/<startdate>/<enddate>', methods=['GET'])
def gen2(startdate, enddate):
     result_err={}
     k=subprocess.getoutput(' /home/azureuser/anaconda3/bin/conda run -n py311 && python  Ekal_Visit_Listing_Between_Dates_A.py ' +  startdate  +"  " +enddate)
     result_err['fname']=k
     return make_response(jsonify(result_err),200)

if __name__ == "__main__":
    app.run(host='0.0.0.0')

Error in browser is :

{
  "fname": "\n# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<\n\n    Traceback (most recent call last):\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/exception_handler.py\", line 17, in __call__\n        return func(*args, **kwargs)\n               ^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/cli/main.py\", line 83, in main_subshell\n        exit_code = do_call(args, parser)\n                    ^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/cli/conda_argparse.py\", line 196, in do_call\n        result = getattr(module, func_name)(args, parser)\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/cli/main_run.py\", line 96, in execute\n        script, command = wrap_subprocess_call(\n                          ^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/utils.py\", line 448, in wrap_subprocess_call\n        raise Exception(\"No compatible shell found!\")\n    Exception: No compatible shell found!\n\n`$ /home/azureuser/anaconda3/bin/conda run -n py311`\n\n  environment variables:\n                 CIO_TEST=<not set>\n               CONDA_ROOT=/home/azureuser/anaconda3\n           CURL_CA_BUNDLE=<not set>\n               LD_PRELOAD=<not set>\n                     PATH=/home/azureuser/anaconda3/envs/py311/bin\n       REQUESTS_CA_BUNDLE=<not set>\n            SSL_CERT_FILE=<not set>\n\n     active environment : None\n       user config file : /var/www/.condarc\n populated config files : \n          conda version : 24.1.2\n    conda-build version : 24.1.2\n         python version : 3.11.7.final.0\n                 solver : libmamba (default)\n       virtual packages : __archspec=1=broadwell\n                          __conda=24.1.2=0\n                          __glibc=2.27=0\n                          __linux=5.4.0=0\n                          __unix=0=0\n       base environment : /home/azureuser/anaconda3  (read only)\n      conda av data dir : /home/azureuser/anaconda3/etc/conda\n  conda av metadata url : None\n           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64\n                          https://repo.anaconda.com/pkgs/main/noarch\n                          https://repo.anaconda.com/pkgs/r/linux-64\n                          https://repo.anaconda.com/pkgs/r/noarch\n          package cache : /home/azureuser/anaconda3/pkgs\n                          /var/www/.conda/pkgs\n       envs directories : /var/www/.conda/envs\n                          /home/azureuser/anaconda3/envs\n               platform : linux-64\n             user-agent : conda/24.1.2 requests/2.31.0 CPython/3.11.7 Linux/5.4.0-1109-azure ubuntu/18.04.5 glibc/2.27 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.6\n                UID:GID : 33:33\n             netrc file : None\n           offline mode : False\n\n\nAn unexpected error has occurred. Conda has prepared the above report.\nIf you suspect this error is being caused by a malfunctioning plugin,\nconsider using the --no-plugins option to turn off plugins.\n\nExample: conda --no-plugins install <package>\n\nAlternatively, you can set the CONDA_NO_PLUGINS environment variable on\nthe command line to run the command without plugins enabled.\n\nExample: CONDA_NO_PLUGINS=true conda install <package>\n"

Curl call to this is also failing:

curl  'https://afitraining.ekalarogya.org/vl/2024-01-01/2024-03-01'
{
  "fname": "\n# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<\n\n    Traceback (most recent call last):\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/exception_handler.py\", line 17, in __call__\n        return func(*args, **kwargs)\n               ^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/cli/main.py\", line 83, in main_subshell\n        exit_code = do_call(args, parser)\n                    ^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/cli/conda_argparse.py\", line 196, in do_call\n        result = getattr(module, func_name)(args, parser)\n                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/cli/main_run.py\", line 96, in execute\n        script, command = wrap_subprocess_call(\n                          ^^^^^^^^^^^^^^^^^^^^^\n      File \"/home/azureuser/anaconda3/lib/python3.11/site-packages/conda/utils.py\", line 448, in wrap_subprocess_call\n        raise Exception(\"No compatible shell found!\")\n    Exception: No compatible shell found!\n\n`$ /home/azureuser/anaconda3/bin/conda run -n py311`\n\n  environment variables:\n                 CIO_TEST=<not set>\n               CONDA_ROOT=/home/azureuser/anaconda3\n           CURL_CA_BUNDLE=<not set>\n               LD_PRELOAD=<not set>\n                     PATH=/home/azureuser/anaconda3/envs/py311/bin\n       REQUESTS_CA_BUNDLE=<not set>\n            SSL_CERT_FILE=<not set>\n\n     active environment : None\n       user config file : /var/www/.condarc\n populated config files : \n          conda version : 24.1.2\n    conda-build version : 24.1.2\n         python version : 3.11.7.final.0\n                 solver : libmamba (default)\n       virtual packages : __archspec=1=broadwell\n                          __conda=24.1.2=0\n                          __glibc=2.27=0\n                          __linux=5.4.0=0\n                          __unix=0=0\n       base environment : /home/azureuser/anaconda3  (read only)\n      conda av data dir : /home/azureuser/anaconda3/etc/conda\n  conda av metadata url : None\n           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64\n                          https://repo.anaconda.com/pkgs/main/noarch\n                          https://repo.anaconda.com/pkgs/r/linux-64\n                          https://repo.anaconda.com/pkgs/r/noarch\n          package cache : /home/azureuser/anaconda3/pkgs\n                          /var/www/.conda/pkgs\n       envs directories : /var/www/.conda/envs\n                          /home/azureuser/anaconda3/envs\n               platform : linux-64\n             user-agent : conda/24.1.2 requests/2.31.0 CPython/3.11.7 Linux/5.4.0-1109-azure ubuntu/18.04.5 glibc/2.27 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.6\n                UID:GID : 33:33\n             netrc file : None\n           offline mode : False\n\n\nAn unexpected error has occurred. Conda has prepared the above report.\nIf you suspect this error is being caused by a malfunctioning plugin,\nconsider using the --no-plugins option to turn off plugins.\n\nExample: conda --no-plugins install <package>\n\nAlternatively, you can set the CONDA_NO_PLUGINS environment variable on\nthe command line to run the command without plugins enabled.\n\nExample: CONDA_NO_PLUGINS=true conda install <package>\n"

whereas if I run this from inside venv this runs in less than a min:

(base) myhost:~$ source activate py311
(py311) myhost:~$ python script.py '2024-01-01' '2024-03-01'
myurl/reports/19_03_2024_10_27_56_product_listing.xlsx

Answer 1

The basic usage for the conda run command is:

conda run -n ENVIRONMENT COMMAND [arguments]

The && in this line /home/azureuser/anaconda3/bin/conda run -n py311 && python .. chains the commands:

1. /home/azureuser/anaconda3/bin/conda run -n py311
2. python ...

The first command exits with an error because the command isn't properly constructed and the second command never runs.

To run the script in the conda environment without activating it, use:

    k = subprocess.getoutput(
        f'/home/azureuser/anaconda3/bin/conda run -n py311 ' 
        f'python Ekal_Visit_Listing_Between_Dates_A.py '  
        f'{startdate} {enddate}'
    )