Can I tell Lucene.net not to create files over a certain size?

Question

I have an upload limit on file sizes of 100MB but my index files are coming out larger than that. Is there an option somewhere I can set so that at a specific file size the index will spill out into a secondary file?

Answer 1

Sort of. From Java Lucene FAQ:

Is there a way to limit the size of an index?

This question is sometimes brought up because of the 2GB file size limit of some 32-bit operating systems.

This is a slightly modified answer from Doug Cutting:

The easiest thing is to use IndexWriter.setMaxMergeDocs().

If, for instance, you hit the 2GB limit at 8M documents set maxMergeDocs to 7M. That will keep Lucene from trying to merge an index that won't fit in your filesystem. It will actually effectively round this down to the next lower power of Index.mergeFactor.

So with the default mergeFactor set to 10 and maxMergeDocs set to 7M Lucene will generate a series of 1M document indexes, since merging 10 of these would exceed the maximum.