Query with $in operator and large list of Ids

Question

I have a pretty large number of document Ids to iterate through (say 5k-10k). The $in operator doesn't limit that number starting from mongodb version 2.6. Earlier versions had a combinatorial limit of 4 mio.

That said, does it make sense at all to do something like that in mongodb or is it an anti-pattern with performance penalties and I should iterate manually in the application layer?

Answer 1

It's somewhat of an anti-pattern, but sometimes there's no other choice.

If you can change the schema and make that query redundant then you should. If you can't, doing it yourself will surly be slower than letting MongoDB do it.

However, there is another limit you need to consider. Each document in MongoDB is limited to 16MB and each query is sent as a document so with enough items you may reach that limit and get an exception.