apscheduler BackgroundScheduler() process is not running in the background

Question

I am working on a project. Below are the files from which the problem resides.

cli.py

import click
from apscheduler.schedulers.background import BackgroundScheduler

scheduler = BackgroundScheduler(daemon=True)

def func():
    print("scheduler running with interval....")

@click.command()
@click.option('--interval', type=int, default=5, help='Interval for the scheduler in seconds')
def start_scheduler(interval):
    scheduler.add_job(func, 'interval', seconds=interval)
    scheduler.start()
    try:
        # Keep the script running (use Ctrl+C to stop it)
        while True:
            pass
    except (KeyboardInterrupt, SystemExit):
        scheduler.shutdown()


@click.command()
def stop_scheduler():
        if scheduler.running:
            scheduler.shutdown()
            print("scheduler shut down")
        else:
            print("scheduler is not running")
        
@click.command()
def status_scheduler():
    if scheduler.running:
        print("Scheduler is running.")
    else:
        print("Scheduler is not running.")

setup.py

""" This file is used for packaging the source code and command line interface"""

from setuptools import setup

setup(
    name='test_package',
    version='0.1.0',
    packages=['my_pack'],
    install_requires=['click','APScheduler'],
    entry_points={
        'console_scripts': [
            'start = my_pack.cli:start_scheduler'
            'status = my_pack.cli:status_scheduler',
            'stop = my_pack.cli:stop_scheduler',
        ],
    },
)

Output i am getting :

when i run start command on terminal it's not running in the background. to exit from the the script need to hit ctl+C i am using virtual environment in Ubuntu.

Expected output :

when i give start command it should return me shell to type another commands and should run in the background until it's stopped manually.

$ start
$ 

python version : Python 3.10.12

Platform : #40~22.04.1-Ubuntu

Answer 1

You should probably make this into a systemd unit. That lets you start and stop the scheduler process at will (using systemctl start myservice or systemctl stop myservice, respectively), in addition to allowing you to start it at system boot time.

One alternative would be to use systemd timers instead, if you just want to run a Python script every minute and it doesn't have a lot of startup overhead. Then you wouldn't need APScheduler either.

Answer 2

As you mentioned in the comment your motto :

However, my requirement is to run a specific function every 1 minute in the background on Linux using Python and i must be able to stop that process from command line using another python function command.

To be an answer for that : You can use subprocess to fulfill your expectation by piping out your current python file. So the another file will be still running in background. I just re-created your tree path and explained below,

setup.py

import subprocess
process = subprocess.Popen(["python3", "my_pack/cli.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)

Note : Above code will run your cli.py file as subprocess and pipe out the output or error into background.

cli.py

import time
while True:
    print('Running in background..')
    time.sleep(5*60)

Note : Above code as cli.py file made for example to run every 5 minutes a function (print function shown as example) as OP asked for.

Result,

In the result picture, you can see, before running setup.py there is no cli.py running in background. After running setup.py checking running process using same command ps aux | grep python3 you can notice cli.py is running in background, at the same time setup.py is already end its process.

command used to check all python running process,

ps aux | grep python3

ps : process status
a : all users on system
u : display detailed information from user
x : add extra information other than terminal process
grep : used to search for particular word
python3 : keyword searching for (process what we looking for)

To Kill the process

kill <process_id> kill 9349

Simply call this bash line from your python code to kill your cli.py using its process id. Process id : 9349 is from my side, it will be vary from your end. Look the result image carefully for better understanding, Hope your expectation fulfilled.

Note : Focus on 3rd line in process status from result image above to identify cli.py process running in background after running setup.py.