Schedule a python file to run automatically every week

Question

My python code involves picking up a file from Azure Blob storage, making changes to the file and then send it back to the Azure storage. I have a working code to do this task. Now I need to schedule it to run every week at a specific time. What is the best tool to use, where I can simply upload my ipynb file and schedule it? I tried using Power Automate but it has no option to upload ipynb files. Can someone help? Also, can I get steps on how to use that tool to achieve what I want? I tried to research all my options but not finding perfect tutorials for my task. Looked into Azure Data Functions using VS Code, but don't know the steps.

Answer 1

You can make use of Azure Batch Service, As your python code picks file from the blob storage makes changes and send it back to the same storage.

Add your Python Code file in the Storage account from where our Azure batch service will pick it as a resource and run it inside a pool of your choice that you can deploy. And then you can use Job scheduler to schedule your Job that runs the code as per your time. Azure batch service uses less compute to run your code.

Add your python script in the Storage account Container and create a SAS token URL for the uploaded python script blob:-

Link this Storage account to your Batch Service:-

Create a Pool with Linux OS and then create Job Schedules:-

In schedule select the time at which you want to run your job:-

Select Job configuration (update) > Add the Pool Id from the Pool created above. And in Command line add the command below:-

/bin/bash -c "pip install azure-storage-blob && python scheduler.py"

In the Resource select Http url > in Value add the Storage blob with SAS Token URL > In File path add scheduler.py

scheduler.py

from azure.storage.blob import BlobServiceClient

# Azure Storage account details
account_name = 'silistrg89'
account_key = 'xxxxxxxx'
container_name = 'data'
blob_name = 'Hello test.txt'

# Connect to the Azure Blob service
blob_service_client = BlobServiceClient(account_url=f"https://{account_name}.blob.core.windows.net", credential=account_key)

# Get a specific blob
blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)

# Download the blob content
blob_data = blob_client.download_blob()
blob_text = blob_data.readall()

# Modify the content (for example, appending a string)
modified_text = blob_text + b"\nModified content appended."

# Upload the modified content back to Azure Blob storage
blob_client.upload_blob(modified_text, overwrite=True)

print("File modified and uploaded successfully.")

Jobs schedule created:-

If you want to implement this in Azure functions, You need to create a Timer Trigger Function refer here. In Timer Trigger you need to use CRON expression to specific time once per week.

And use this code in your function_app.py:-

import logging
import azure.functions as func
from azure.storage.blob import BlobServiceClient

app = func.FunctionApp()

@app.schedule(schedule="0 * * * * *", arg_name="myTimer", run_on_startup=True,
              use_monitor=False) 
def timer_trigger(myTimer: func.TimerRequest) -> None:
    # Azure Storage account details
    account_name = 'silistrg89'
    account_key = 'GLw7xxxxxxxxxx'
    container_name = 'data'
    blob_name = 'Hello test.txt'

    # Connect to the Azure Blob service
    blob_service_client = BlobServiceClient(account_url=f"https://{account_name}.blob.core.windows.net", credential=account_key)

    # Get a specific blob
    blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)

    # Download the blob content
    blob_data = blob_client.download_blob()
    blob_text = blob_data.readall()

    # Modify the content (for example, appending a string)
    modified_text = blob_text + b"\nModified content appended."

    # Upload the modified content back to Azure Blob storage
    blob_client.upload_blob(modified_text, overwrite=True)

    print("File modified and uploaded successfully.")

Local Output:-

Deploy this in your Azure Function App and it the Function will trigger automatically in the time Specified by the CRON exprsssion:-