Testing ElasticSearch with Node. Force update index after populating MongoDB

Question

I'm populating MongoDB and adding river-mongodb in a before hook in my tests. But it looks like Elasticsearch hasn't indexed the data I have in MongoDB when I query it. Am I doing it wrong, or should I force an update somehow?

  function setupMongoDBRiver() {
      var url = 'http://127.0.0.1:9200/_river/mongodb/_meta';
      var requestData = {
          "type": "mongodb",
          "mongodb": {
              "db": "harvester-test",
              "collection": "entries"
          },
          "index": {
              "name": "harvester-test",
              "type": "entries"
          }
      };
      request({
          url: url,
          method: 'PUT',
          body: JSON.stringify(requestData)
      },
  }



 describe('Entries Test Parameters', function() {
    before(function(done) {
        DatabaseHelper.restoreAll(mongoUrl, done);
        deleteTestIndex();
        setupMongoDBRiver();
        testQuery();  
       // ^ curl http://127.0.0.1:9200/harvester-test/entries/_search?pretty=true&q=*:*
    }

testQuery Does not return what I put in MongoDB. Could be that ES needs more time to index the data? Do I have to specify the ES index, or is that taken care of by river-mongodb?

Cheers, Martin

Answer 1

There are a few things to note about your before() function which seem to suggest that you haven't fully grasped how asynchronous function calling works on Node.

Let me walk through the steps:

  DatabaseHelper.restoreAll(mongoUrl, done);

I assume that this function resets or restores your MongoDB to a predefined state. You're calling Mocha's done callback when it has finished (which signals to Mocha that the before() is finished), yet continue to call other functions. You should call done when everything is ready, not just the first function.

  deleteTestIndex();
  setupMongoDBRiver();
  testQuery();

These are all asynchronous functions. In other words, you need to explicitly wait for their completion before continuing on to the next step (you first want to delete the test index, then set up the MongoDB river, then run the test query).

A good module to coordinate asynchronous functions is async. You can use async.series (or perhaps async.waterfall) to coordinate your function calls, but it would require some rewriting (for instance, your setupMongoDBRiver() function doesn't call a completion callback).

Assuming that you rewrite your code to use completion callbacks, your before() would look something similar to this:

before(function(done) {
  async.series([
    function(next) {
      DatabaseHelper.restoreAll(mongoUrl, next);
    },
    function(next) {
      deleteTestIndex(next);
    },
    function(next) {
      setupMongoDBRiver(next);
    },
    function(next) {
      testQuery(next);
    }
  ], function(err) {
    if (err) return done(err);
    done();
  });
});

(NB: this can be written more concisely but I used verbose notation for clarity)