Mongodb TTL expires documents early

Question

I am trying insert a document into a Mongo database and have it automatically expire itself after a predetermine time. So far, my document get inserted but always get deleted from the database from 0 - 60 seconds even though I set the 'expireAfterSeconds' to much longer. I know mongodb deletes expired documents about every 60 seconds so it seems the 'expredAfterSeconds' variable is not working.

I followed the documentation here: Mongodb TTL Docs

Here is my test code that should expire (delete) the document after 3 minutes (but it does it under a minute):

import pymongo
import datetime

mongo_con = pymongo.Connection('localhost', 27017)
mongo_db = mongo_con.Mongo_database
mongo_col = mongo_db.my_TTL_collection

timestamp = datetime.datetime.now()

mongo_col.ensure_index("date", expireAfterSeconds=3*60)                     

mongo_col.insert({'_id': 'login_session', "date": timestamp, "session": "test session"})

Does anybody have any ideas what the problem is?

Answer 1

Your problems come from using naive timestamps in your local timezone. The FAQ of pymongo has an entry which includes a warning not to use datetime.datetime.now(). Using utcnow, the ttl-setting works as expected:

import pymongo
import datetime

mongo_con = pymongo.Connection('localhost', 27017)
mongo_db = mongo_con.Mongo_database
mongo_col = mongo_db.my_TTL_collection

timestamp = datetime.datetime.now()
utc_timestamp = datetime.datetime.utcnow()

mongo_col.ensure_index("date", expireAfterSeconds=3*60)                     

mongo_col.insert({'_id': 'session', "date": timestamp, "session": "test session"})
mongo_col.insert({'_id': 'utc_session', "date": utc_timestamp, "session": "test session"})
# the utc_session will be deleted after around 3 minutes, 
# the other depending on your timezone

Answer 2

For Pymongo 3 this is the updated Syntax.

mongo_collection.create_index("date", expireAfterSeconds=3*60)