Getting started with Bleve using BoltDB

Question

I am trying to wrap my head around Bleve and I understand everything that is going on in the tutorials, videos and documentation. I however get very confused when I am using it on BoltDB and don't know how to start.

Say I have an existing BoltDB database called data.db populated with values of struct type Person

type Person struct {
   ID int          `json:"id"`             
   Name string     `json:"name"` 
   Age int         `json:"age"`
   Sex string      `json:"sex"`
}

How do I index this data so that I can do a search? How do I handle the indexing of data that will be stored in the database in the future?

Any help will be highly appreciated.

Answer 1

Bleve uses BoltDB as one of several backend stores and is separate from where you store your application data. To index your data in Bleve, simply add your Index:

index.Index(person.ID, person)

That index exists separately from your application data (whether it's in Bolt, Postgres, etc).

To retrieve your data, you'll need to construct a search request using bleve.NewSearchRequest(), then call Index.Search(). This will return a SearchResult which includes a Hits field where you can retrieve the ID for your object. You can use this to look up the object in your application data store.

Disclaimer: I am the author of BoltDB.

Answer 2

How you index your data depends on how you want to query for it.

If you want to query by any arbitrary fields, like {Age:15, Name:"Bob"} then BoltDB isn't an awesome fit for your problem.

BoltDB is simply a key value store with fast access to sequential keys and efficient prefix seeking. It's not really a replacement for general use databases.

You likely want something more like a document store (ie: MongoDB) or RDBMS (ie: PostgreSQL).

If you just wanted something that uses simple files and is embedded, you could also use SQlite with the Go module

If you want to search by only a single field, like ID or Name, then use that as the key.

If lookup speed doesn't matter at all, I guess you can use Bolt to just iterate over the entire db, parse the json and check the fields. But that's probably the worst approach you could take.