DjangoJobStore Randomly Deletes Jobs

Question

I have apscheduler running with a DjangoJobStore from a management command in django, but every so often some (not all) of the jobs will be deleted from the database.

The scheduler still runs, still gets other jobs, but doesnt find the specific one in the code below. When i check the admin, its deleted.

Restarting the command fixes it as it adds the job back for a random amount of time before deleting again.

I added a listener for EVENT_JOB_REMOVED and it is never fired, and i added breakpoints in all the delete methods for DjangoJob and none of them ever hit.

the function must_reply simply checks an email inbox and sends a reply back over smtp, nothing in there accesses the DjangoJob table, nor does anything in the rest of the code.

must_reply is decorated with @close_old_connections

removing DjangoJobStore as a job store fixes the problem, but obviously I would like to use it haha

Has anyone experienced this? And what are some solutions or ideas to try for fixing this.

It is truly strange.

class Command(BaseCommand):
    help = "Runs APScheduler."

    def handle(self, *args, **options):

            executors = {
                'default': ThreadPoolExecutor(max_workers=10)
            }

            job_defaults = {
                'coalesce': True,
                'misfire_grace_time': 10

            }

            scheduler = BlockingScheduler(timezone=settings.TIME_ZONE, executors=executors, job_defaults=job_defaults)

            scheduler.add_jobstore(DjangoJobStore(), "default")

            scheduler.add_job(
                must_reply,
                IntervalTrigger(
                    seconds=10
                ),
                id="must_reply",
                max_instances=1,
                replace_existing=True,
            )

            try:
                scheduler.start()
            except:
   
                scheduler.shutdown()