How to pip install extension modules in Azure web jobs?

Question

I'm trying to schedule a python script that uses an extension module in an Azure web job:

import sys

sitepackage = "D:\home\site\wwwroot\env\Lib\site-packages"
sys.path.append(sitepackage)

try:
    from bs4 import BeautifulSoup
    print("!!! BEAUTIFUL SOUP !!!")
except ImportError as e:
    print(e)

I have all the appropriate extension modules pip installed in my (venv) inside of my 'site-packages' folder:

But it fails to run because it cannot import beautifulsoup4 from bs4:

error: "No module named bs4"

Answer 1

Okay, so I figured it out here's my solution and I'll explain each step in detail down below.

1. Make sure you have an extension for python in your App service.
2. Create and zip a folder for 3 items: your_file_name.py, run.bat, and requirements.txt
3. Create a new Web Job with the new zipped folder

STEP 1 - Make sure you have python site extension in your App Service:

1. Navigate to your App Service in Azure
2. Go to advance tools
3. Click on site extensions
4. Install the python extension that you'd like to use

STEP 2 - Create and zip a folder for 3 items: your_file_name.py, run.bat, and requirements.txt

• your_file_name.py is just your python script that you want to run
• run.bat is your batch file to call your executable files. This file should contain these commands: the first will pip install your 3rd party dependencies specified in your requirements.txt and the second will execute your script. (edit path & filenames to match yours)

    D:\home\python364x86\python.exe -m pip install --upgrade -r D:\home\site\wwwroot\App_Data\jobs\triggered\webjobname\zippedfoldername\requirements.txt
    D:\home\python364x86\python.exe your_file_name.py

• requirements.txt is where you want to specify the extension modules that you'd like to use and even the version like so. (more about requirements.txt here)

beautifulsoup4==4.9.3
bs4==0.0.1
soupsieve==2.2
urlopen==1.0.0

STEP 3 - Create a new Web Job with the new zipped folder