Is it possible to speedup FileProviderEnumerator for many files?

168 views Asked by At

I'm creating FileProvider extension for MacOS. In my case I have a remote folder with a lots of small files (~50k).

If I trying to open file directly by known name (for example with a cat in terminal), FileProvider asks listing of directory first — invokes enumerator, and only then fetchContents.

It's wouldn't be a problem if enumerateItems will be fast. If enumerate all 50k items, macOS kills extension. So I did batching. And it works, but extremely slow. My system suggests 200 items in a batch (observer's suggestedPageSize property), enumerateItems invokes about once per second. More than 4 minutes to get a list files in directory! Without any network requests, because list is cached.

Is it possible to speedup NSFileProviderEnumerator? Or tell FileProvider to invoke fetchContents without parent enumerating? Maybe any hacks?

let pageSize: Int64 = Int64(observer.suggestedPageSize ?? 256)
let sliceFrom: Int64 = page.toInt64()
var sliceTo: Int64 = 0
var nextPage: NSFileProviderPage? = nil
var items: [FileProviderItem] = []
var content = item.getChildItems() ?? []  // cached
    
if content.count >= sliceFrom + pageSize {
    sliceTo = sliceFrom + pageSize - 1
    nextPage = NSFileProviderPage(sliceTo + 1)
}
else {
    sliceTo = Int64(content.count - 1)
}

content.enumerated().filter({ $0.offset >= sliceFrom && $0.offset <= sliceTo}).forEach {
    (_, file) in
    let itemIdentifier = FileProviderItem.fileToItemIdentifier(object: file)
    let childItem = FileProviderItemRepository.shared.getItem(identifier: itemIdentifier)
    items.append(childItem)
}

observer.didEnumerate(items)
observer.finishEnumerating(upTo: nextPage)
0

There are 0 answers