Alternative for Virtual file system (VFS) kernel extension on macOS M1

Question

we have developed a kernel extension (KEXT) for a virtual file system (VFS) on macOS to integrate our software with external programs like Adobe InDesign or Microsoft Word. Our software and the KEXT are used by many of our customers.

As it looks like KEXTs are deprecated and may be removed completely in future versions of macOS, particularly on Apple Silicon based computers. See e.g. Apple's announcement in its security guide:

"This is why developers are being strongly encouraged to adopt system extensions before kext support is removed from macOS for future Mac computers with Apple silicon"

Therefore we are currently investigating in possible alternatives.

Apple suggests to migrate to System Extensions instead of KEXTs. However, the only VFS related API we found is to implement a File Provider that is based on an NSFileProviderReplicatedExtension.

Unfortunately that NSFileProviderReplicatedExtension has several flaws:

1. Files can either be in the cloud or downloaded. It is not possible to download/read only a portion of a file. This is a big performance problem for us, since we work with large images (> 1GB). The programs we integrate with typically only read a part of the image, e.g. the embedded preview. The API does not offer a way to access selected blocks of a file (random access file).
2. The File Provider learns about the file system content via enumerators. So everything that is inside a folder must be enumerated (listed) first. Otherwise it cannot be accessed. However, we cannot enumerate our VFS. Most of the content of our VFS is fully dynamic. It only exists when it is accessed by a client the first time. Such dynamic content also includes dynamic parameters like the client's locale or the size of a box where the image will be placed. Since we do not know those parameters in advance, we cannot enumerate the VFS's content in advance.

This means, an NSFileProviderReplicatedExtension in its current state isn't a replacement for a "real" VFS and therefore cannot be used by us as a replacement for our current VFS KEXT.

My questions:

1. Will Apple allow kernel extensions also in future versions of (Apple Silicon/M1 based) operating systems? Or is there at least a clear deadline?
2. If not, what is Apple's officially suggested replacement for KEXT based VFS solutions?
3. Will the API of an NSFileProviderReplicatedExtension be improved to behave like a "real" file system so that above mentioned flaws will no longer be an issue?

Many thanks for any answers or comments!

Best regards,

Michael

Answer 1

Will Apple allow kernel extensions also in future versions of (Apple Silicon/M1 based) operating systems? Or is there at least a clear deadline?

Apple doesn't really give timelines, and they also occasionally break promises of support.

However, this sort of hard API deprecation and removal usually is done as part of a major release, so you will typically get deprecation notice for it at WWDC one year, users might start seeing deprecation notices when the .0 of the OS release ships at the earliest, and sometimes the .3 or .4 revision. Then you'll typically be told at the next WWDC that the API is blocked in the upcoming release, so by that point you should have implemented a replacement.

If not, what is Apple's officially suggested replacement for KEXT based VFS solutions?

As far as I'm aware, NSFileProviderReplicatedExtension is currently the only one.

Will the API of an NSFileProviderReplicatedExtension be improved to behave like a "real" file system so that above mentioned flaws will no longer be an issue?

Other than via beta SDKs, Apple generally doesn't pre-announce future APIs. Update for macOS 13: Some of Apple's own file system drivers, including "msdos" (FAT) are now implemented in user space. The kernel API this uses is thus far not public.

My advice:

• File issues for each of the file provider shortcomings you are hitting using Feedback Assistant. (Radar)
• File an "enhancement request" feedback issue with Apple for a "real" file system API replacement for the VFS KPI.
• If your vfs kext is critical to your business/product, I suggest additionally asking Apple's DTS via a TSI what they recommend for your situation. Reference the feedback IDs of the issues filed, otherwise they will recommend that you file issues.

Answer 2

Regarding your obstacles:

1. There is NSFileProviderPartialContentFetching which allows partial fetching of an item's content.
2. Enumeration only is telling the system of files and folders just like when you browse the local file system with Finder. What you tell the system is under your control and you can pass on metadata you retrieved with a parameter like your client's locale code. Maybe you can elaborate how this is an obstacle. Based on your description it should still work out.

I is odd to ask for the NSFileProviderReplicatedExtension to behave like a "real" file system because it actually provides metadata of remote storage to macOS which then reconstructs that state in the local, actual file system. So in the end you browse the actual file system in ~/Library/CloudStorage/<whatever-domain> with (optional) "placeholder" files (the unmaterialized items).