I have large indexes in memory for quick search in a huge dataset. It takes forever to build the indexes and it likewise take a long time to just deserialize a snapshot of them from disk to memory. And they use up a lot of precious/expensive RAM. I did make an attempt to use the indexes straight from disk with a kind of custom search-reader that just deserialized the required bits on-the-fly. This worked surprisingly well but did have some serious drawbacks in a multi-threaded environment regarding locks, concurrency, versioning etc. And it was not superfast of course.
Anyhow, I was wondering if "persistent RAM" (NVMe, Optane..?) could help me out here. I'm not sure I understand how they are supposed to be used exactly, but it seems they fill a gap between HDD/SSD and DDR-RAM? They cost less than RAM but are faster than SSD, and are persistent.
So what I was hoping for is to somehow run a .NET-program straight up on persistent RAM, with my indexes, and without having to serialize it back and forth to SSD between restarts. That would be a dream come true!
If not putting an entire .NET-program in persistent RAM, perhaps there are other ways of interfacing with it from .NET? Even if it's only writing and reading bytes?