Basic GROUP BY statement using OPA MongoDB high level API

Question

My question is quite simple: I'd like to perform a GROUP BY like statement with MongoDB using the OPAlang high level database API. But I don't think that is possible (?)

If I do want to perform a mongodb $group operation, do I necessarily need to use the low-level API (using stdlib.apis.mongo ?)

Finally, can I use both low-level and high-level APIs to communicate with my MongoDB ?

Thanks.

Answer 1

I am afraid that, taking into account latest published Opa compiler code, no aggregation is supported :( See the thread in Opa forum. Also note the comment of Quentin about the using both low- and high-level API-s:

"You can use this [low level] library and built-in [hight level] library together, [...]"

See the auto-increment implementation advices by the guys from the MLstate in this thread. Note the high level DB field /next_id definition and initialization with low level read and increment.

Answer 2

I just got different idea.

All MongoDB commands (eg. the "group" command you are using) are accessible with the virtual collection named $cmd. You just ask the server to find the document {command_name: command_parameter, additional: "options", are: ["listed", "here"]}. You should be able to use every fancy feature of your MongoDB server, not supported yet with the Opa API, with single find query. This includes the aggregation framework introduced in version 2.2 and the full-text searching still in beta since version 2.4.

For example, I want to use new text command to search in full-text index for collecion coll_name the query string query. I am currently using the code (where oncuccess is the function to parse the answer and get the id-s of the documents found):

{ search: query, project: {_id:0, id:1}, }
|> Bson.opa2doc
|> MongoCommands.simple_str_command_opts(ll_db, db_name, "text", coll_name, opts)
|> MongoCommon.outcome_map(_, onsuccess, onfailure)

And if you take a look at the source code of the API, simple_str_command_opts is implemented as a findOne() to the Mongo.

But instead I could use the high level DB support:

/test/`$cmd`[{text: coll_name, search: query, project: {_id: 0, id: 1}}]

What you have to do, is to declare the high-level DB collection with the type including:

1. all the fields that you use to make the query,
2. all the fields that you can get in possible answer.

For the text command:

type commands = {
  // command
  string text,

  // query
  string search,
  {
    int _id,
    int id,
  } project,

  // result of executing command "text"
  string queryDebugString,
  string language,
  list({
      float score,
      {int id} obj,
    }) results,
  {
    int nscanned,
    int nscannedObjects,
    int n,
    int nfound,
    int timeMicros,
  } stats,
  int ok,
  // in case of failure (`ok: 0`)
  string errmsg,
}

Unfortunately, it is not working :( During the application start-up Opa run-time DB support tries to create the unique index for the primary key of the set (for following example {text, search, project}):

database test {
  article /article[{id}]
  commands /`$cmd`[{text, search, project}]
}

Using primary key is necessary, since you have to use findOne(), not find(). Creating an index for virtual collection $cmd is not allowed and DB initialization fails.

If you find the way to stop Opa from creating index, you will be able to use all the fancy features of Mongo using no more then high-level API ;)