How to write a Faiss index to memory?

4.7k views Asked by At

I want to write a faiss index to back it up on the cloud. I can write it to a local file by using faiss.write_index(filename, f). However, I would rather dump it to memory to avoid unnecessary disk IO.

I tried passing in a StringIO or ByteIO stream, but I'm getting these error from the swig layer.

TypeError: Wrong number or type of arguments for overloaded function 'write_index'.
  Possible C/C++ prototypes are:
    faiss::write_index(faiss::Index const *,char const *)
    faiss::write_index(faiss::Index const *,FILE *)
    faiss::write_index(faiss::Index const *,faiss::IOWriter *)

Any idea on how to write_index to memory?

2

There are 2 answers

0
Lizozom On

Found this here

chunk = faiss.serialize_index(index)
0
Will Smith R On

From Github issues

You can use the following:

faiss.write_index(index, filename)