Django backend with celery workers - memory threshold

Question

I have a system with some periodic tasks. Each periodic task reads in files and creates objects out of them in the RAM.

To prevent data loss, the cached file content is immediately stored from the peridoic task into the database after creating the object out of it. Then the file is deleted from the source.

The object is queued into a RabbitMQ and the Celery workers take them for further processing and concurrency.

Before caching the next file, the memory usage of the system is fetched with psutil. If it exceeds a threshld, the periodic task stops to read in files and continues with it, when executed the next time.

My question is

1. If this is generally a feasible approach if no data must be lost (the files are either stored in the database or still in the source folder)

2. If it is a good approach to keep the synchronisation effort low (files are deleted after reading, the worker operates on objects and does not delete files and there is no sync needed with the periodic task)

3. Which memory threshold is "safe" and fast enough. The file size can be up to 40 MB.