How to synchronize a python dictionary across a flask app and a background thread

Question

Here is my problem. I have a Flask application which does some predictions on some datasets. The users can upload multiple datasets to the server and operate on any of the dataset.

Here, I need to clean the user data directory which is

static/files/userdata_<ip_of_client>/

I am trying to remove the entire directory if user does not connect for 30 seconds. FYI, the client side javascript sends a POST request using fetch every 10 seconds, so if it misses 3 window the files will be cleared. Thats the idea.

The POST request triggers an API having code like this

@app.route("/api/ping", methods=["POST"])
def ping():
  try:
     client_ip = request.remote_addr
     with data_lock:
       folder_access_times[client_ip] = datetime.datetime.now()

Here data_lock is a globally declared Lock() from threading module

In another thread I am checking if any client did not connect for 30 sec like this

with data_lock:
    curtime = datetime.datetime.now()
    for key,val in folder_access_times.items():
        diff = (curtime-val).seconds
        if diff > 30:
            shutil.rmtree(...)

The problem is that the lock is not working properly. noticing very strange beheavior like

Even if I am using Lock here and the ping API successfully modifies the dictionary, but I am noticing and the background thread sees the old data even long after the API updated it.

The data is very inconsistent as current date also reverts back to previous in next iteration of background process ( I am using while loop in the background process) which is not intended in case of Lock.

Don't understand what am I missing. Any help will be appreciated.