CouchDB Replication does not replicate old revision documents

Question

I'm working on "one database per user" system using the CouchDB replication with a selector to filter my data based on the user configuration.

It works pretty well, until the day when i noticed an issue with the replication, it is difficult for me to describe it so I will do it with an example:

I have my main database "mainDB" which i'm using as the "source" database for the replication, and i decide to create a sub database "subDB" for a user which will be the "target" for the replication.

I create my replication doc with my selector to filter the data from my "mainDB" and nothing happen, my "subDB" is empty, the replication state is marked as "Running" but 0 pending changes.

And as soon as i update a doc from the "mainDB" (doc that is supposed to be replicated to my "subDB"), the "_rev" of this doc will change, the replication really start and replicate my doc to the "subDB".

In brief, CouchDB filtred replication based on a selector will not replicate any doc until we update "_rev" of each doc that is supposed to be replicated.

App version Apache CouchDB v. 3.2.2

EDIT 1 The selector looks like this:

{
   "selector": {
        "$or": [
            {
                "date_debut": {
                    "$lte": "#end_date#"
                },
                "typedoc": "ActiviteDocument",
                "date_fin": {
                    "$gte": "#start_date#"
                },
                "id": {
                    "$in": [
                        #array_of_integer_A#
                    ]
                }
            },
            {
                "typedoc": "IndividuDocument",
                "id": {
                    "$in": [
                       #array_of_integer_B#
                    ]
                }
            },
    (JSON too long to full parse here, but other part of the $or use same logical)
    ...

}

EDIT 2 : I changed the selector logical by using $or and $and

"selector": {
        "$or": [
               {
                "$and": [
                    {
                        "typedoc": "ActiviteDocument"
                    },
                    {
                        "date_debut": {
                            "$lte": "#end_date#"
                        }
                    },
                    {
                        "date_fin": {
                            "$gte": "#statt_date#"
                        }
                    },
                    {
                        "id": {
                            "$in": [#array_of_integer_A#]
                        }
                    }
                ]
            },
            {
                "$and": [
                    {
                        "typedoc": "IndividuDocument"
                    },
                    {
                        "id": {
                            "$in": [#array_of_integer_B#]
                        }
                    }
                ]
            },

EDIT 3 : i changed my replication doc by removing selector and using "doc_ids", the replication will not replicate my docs except if i update one of them so the "_rev" change and the replication detect that and start working

{
  "_id": "replicationmaster-1123",
  "source": "mysource",
  "target": "mytarget",
  "doc_ids": [
    "ActiviteDocument_335765",
    "ActiviteDocument_351882",
    "ActiviteDocument_421350",
    "ActiviteDocument_423684",
    "ActiviteDocument_428304",
    "ActiviteDocument_440523",
    "ActiviteDocument_442048",
    "ActiviteDocument_443727"
  ],
  "continuous": true,
  "create_target": false,
  "owner": "admin"
}

EDIT 4 : demo https://youtu.be/OqJA0fDQqy8

Answer 1

The problem is that in your selector JSON the $or parameter needs to be an array of objects, each one being an individual condition. The way you have it, this parameter is an array with a single object that has all the conditions in it.

Here is a complete replicator document based on your conditions, with the correct syntax:

{
  "_id": "abc12357",
  "source": "https://username:[email protected]/db1",
  "target": "https://username:[email protected]/db2",
  "selector": {
    "$or": [
      {
        "start": {
          "$lte": "2022-10-27"
        }
      },
      {
        "typedoc": "ActiviteDocument"
      },
      {
        "end": {
          "$gte": "2022-09-29"
        }
      },
      {
        "id": {
          "$in": [
            65993,
            63938,
            87265,
            312112,
            64885,
            64277
          ]
        }
      }
    ]
  }
}