Count query with PHP Elastica and Symfony2 FosElasticaBundle

Question

I'm on a Symfony 2.5.6 project using FosElasticaBundle (@dev).

In my project, i just need to get the total hits count of a request on Elastic Search. That is, i'm querying Elastic Search with a normal request, but through the special "count" URL:

localhost:9200/_search?search_type=count

Note the "search_type=count" URL param.

Here's the example query:

{
  "query": {
    "filtered": {
      "query": {
        "match_all": []
      },
      "filter": {
        "bool": {
          "must": [
            {
              "terms": {
                "media.category.id": [
                  681
                ]
              }
            }
          ]
        }
      }
    }
  },
  "sort": {
    "published_at": {
      "order": "desc"
    }
  },
  "size": 1
}

The results contains a normal JSON response but without any documents in the hits part. From this response i easily get the total count:

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "hits": {
        "total": 81,
        "max_score": 0,
        "hits": [ ]
    }
}

Okay, hits.total == 81.

Now, i couldn't find any solution to do the same through FOSElasticaBundle, from a repository.

I tried this:

$query = (...) // building the Elastica query here
$count = $this->finder->findPaginated(
    $query,
    array(ES::OPTION_SEARCH_TYPE => ES::OPTION_SEARCH_TYPE_COUNT)
)->getNbResults();

But i get an Attempted to load class "Pagerfanta". I don't want Pagerfanta.

Then this:

$count = $this->finder->createPaginatorAdapter(
    $query,
    array(ES::OPTION_SEARCH_TYPE => ES::OPTION_SEARCH_TYPE_COUNT)
)->getTotalHits();

But it would always give me 0.

Would be easy if i had access to the Elastica Finder service from the repository (i could then get a ResultSet from the query search, and this ResultSet has a correct getTotalHits() method). But services from repository... you know.

Thank you for any help or clue!

Answer 1

Ok, so, here we go: it is not possible.

You cannot, as of version 3.1.x-dev (2d8903a), get the total matching document count returned by elastic search from FOSElasticaBundle, because this bundle does not expose this value.

The RawPaginatorAdapter::getTotalHits() method contains this code:

return $this->query->hasParam('size')
    ? min($this->totalHits, (integer) $this->query->getParam('size'))
    : $this->totalHits;

which prevents to get the correct $this->totalHits without actually requiring any document. Indeed, if you set size to 0, to tell elasticsearch not to return any document, only meta information, RawPaginatorAdapter::getTotalHits() will return 0.

So FosElasticaBundle doesn't provide a way to know this total hits count, you could only do that through the Elastica library directly. Of course with the downisde that Elastica finders are natively available in \FOS\ElasticaBundle\Repository. You'd had to make a new service, do some injection, and inovke your service instead of the FOSElasticaBundle one for repositories... ouch.

I chose another path, i forked https://github.com/FriendsOfSymfony/FOSElasticaBundle and changed the method code as follow:

/**
 * Returns the number of results.
 *
 * @param boolean $genuineTotal make the function return the `hits.total`
 * value of the search result in all cases, instead of limiting it to the
 * `size` request parameter.
 * @return integer The number of results.
 */
public function getTotalHits($genuineTotal = false)
{
    if ( ! isset($this->totalHits)) {
        $this->totalHits = $this->searchable->search($this->query)->getTotalHits();
    }

return $this->query->hasParam('size') && !$genuineTotal
    ? min($this->totalHits, (integer) $this->query->getParam('size'))
    : $this->totalHits;
}

$genuineTotal boolean restores the elasticsearch behaviour, without introducing any BC break. I could also have named it $ignoreSize and use it the opposite way.

I opened a Pull Request: https://github.com/FriendsOfSymfony/FOSElasticaBundle/pull/748

We'll see! If that could help just one person i'd be happy already!

Answer 2

I faced the same challenge, getting access to the searchable interface from inside the repo. Here's what I ended up with:

Create AcmeBundle\Elastica\ExtendedTransformedFinder. This just extends the TransformedFinder class and makes the searchable interface accessible.

<?php

namespace AcmeBundle\Elastica;

use FOS\ElasticaBundle\Finder\TransformedFinder;

class ExtendedTransformedFinder extends TransformedFinder
{

    /**
     * @return \Elastica\SearchableInterface
     */
    public function getSearch()
    {
        return $this->searchable;
    }

}

Make the bundle use our new class; in service.yml:

parameters:
      fos_elastica.finder.class: AcmeBundle\Elastica\ExtendedTransformedFinder

Then in a repo use the getSearch method of our class and do what you want :)

class SomeSearchRepository extends Repository
{
    public function search(/* ... */)
    {
        // create and set your query as you like
        $query = Query::create();

        // ...

        // then run a count query
        $count = $this->finder->getSearch()->count($query);

    }
}

Heads up this works for me with version 3.1.x. Should work starting with 3.0.x.

Answer 3

While, you can get the index instance as a service (fos_elastica.index.INDEX_NAME.TYPE_NAME) and ask for count() method.

Joan