/var/lib/postgresql/data blind mount folder data for timescale in docker are not updated despite of explicit insert and update rows to it

Question

We have a docker container for timescale that explicitly uses the volume blind mount set up like this:

timescaledb:
    restart: always
    image: timescale/timescaledb:latest-pg11
    user: root
    container_name: timescaledb
    logging:
      driver: syslog
      options:
        syslog-address: "tcp://${DOCKER_HOST_IP}:5006"
        tag: "{{.Name}}/{{.ID}}"
    ports:
      - 5432:5432
    volumes:
      - ./postgresql/postgresql.conf:/etc/postgresql/postgresql.conf
      - /mnt/postgresql/data:/var/lib/postgresql/data
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - TIMESCALEDB_TELEMETRY=off
      - POSTGRES_PASSWORD=*was there but removed*
    command: postgres -c 'config_file=/etc/postgresql/postgresql.conf'
    depends_on:
      - logstash

so please note /mnt/postgresql/data:/var/lib/postgresql/data here.

After updating some of existing rows I check for the last modified file at the host in /mnt/postgresql/data folder using the approach from here: https://stackoverflow.com/a/7448828/1759063

find $1 -type f -exec stat --format '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head

And I haven't fount any recent file updates there. So where actually the timescale data is stored and what we actually should mount in order not to lose data on docker-compose down and so on?

Output is like that:

find $1 -type f -exec stat --format '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head
2020-10-12 00:33:25.027351283 +0000 ./data/pg_stat/global.tmp
2020-10-12 00:33:18.714856683 +0000 ./data/pg_stat_tmp/global.stat
2020-10-12 00:33:18.714856683 +0000 ./data/pg_stat_tmp/db_0.stat
2020-10-12 00:33:14.770545499 +0000 ./data/pg_stat_tmp/db_13117.stat
2020-10-12 00:01:09.337593127 +0000 ./data/pg_wal/00000001000000240000002A
2020-10-12 00:01:02.329038233 +0000 ./data/global/pg_control
2020-10-12 00:01:02.061017012 +0000 ./data/pg_logical/replorigin_checkpoint
2020-10-12 00:01:01.724990408 +0000 ./data/base/13117/18130
2020-10-12 00:01:01.436967605 +0000 ./data/pg_xact/0018
2020-10-11 23:59:16.664673521 +0000 ./data/pg_stat_tmp/pgss_query_texts.stat

Today is

 date
Wed Oct 21 17:54:06 UTC 2020

So what is the actual place of the folder within container where the updated data were stored?

The most intriguing part there is that the last updated files according to how they are observed from within container and from the host on the same folder (i.e. mapped as a volume blind mount)

From container, looking at the mapped folder:

find $1 -type f -exec stat -c '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head
2020-10-21 19:08:46.000000000 ./data/pg_stat_tmp/global.stat
2020-10-21 19:08:46.000000000 ./data/pg_stat_tmp/db_13117.stat
2020-10-21 19:08:46.000000000 ./data/pg_stat_tmp/db_0.stat
2020-10-21 12:51:21.000000000 ./data/pg_stat_tmp/pgss_query_texts.stat
2020-10-21 10:36:06.000000000 ./data/pg_wal/00000001000000080000001A
2020-10-21 10:36:00.000000000 ./data/pg_xact/0002
2020-10-21 10:36:00.000000000 ./data/pg_logical/replorigin_checkpoint
2020-10-21 10:36:00.000000000 ./data/global/pg_control
2020-10-21 10:36:00.000000000 ./data/base/13117/16539
2020-10-13 18:22:56.000000000 ./data/base/13117/50258

The same folder from the host:

:/mnt/postgresql/data#  find $1 -type f -exec stat -c '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head
2020-10-12 00:33:25.027351283 +0000 ./pg_stat/global.tmp
2020-10-12 00:33:18.714856683 +0000 ./pg_stat_tmp/global.stat
2020-10-12 00:33:18.714856683 +0000 ./pg_stat_tmp/db_0.stat
2020-10-12 00:33:14.770545499 +0000 ./pg_stat_tmp/db_13117.stat
2020-10-12 00:01:09.337593127 +0000 ./pg_wal/00000001000000240000002A
2020-10-12 00:01:02.329038233 +0000 ./global/pg_control
2020-10-12 00:01:02.061017012 +0000 ./pg_logical/replorigin_checkpoint
2020-10-12 00:01:01.724990408 +0000 ./base/13117/18130
2020-10-12 00:01:01.436967605 +0000 ./pg_xact/0018
2020-10-11 23:59:16.664673521 +0000 ./pg_stat_tmp/pgss_query_texts.stat

when trying to reach the file, that was changed according to the docker container at 2020-10-21 19:08:46.000000000 but according to the host it was changed about week earlier only:

:/mnt/postgresql/data# ls -l pg_stat_tmp/global.stat
-rw------- 1 70 70 607 Oct 12 00:33 pg_stat_tmp/global.stat

Can a volume blind mount folder contain the different files in container and host view?

The only unusual thing here is that on the same docker we have another container running not the blind mount, but the volume with the same Destination, as we have in our blind mount:

"Type": "volume",
>                 "Name": "9a241364bdb2c9f1f1ac5ce30effeca15d1c7477f7b99eec42acf40bee5e8ba2",
>                 "Source": "/var/lib/docker/volumes/9a241364bdb2c9f1f1ac5ce30effeca15d1c7477f7b99eec42acf40bee5e8ba2/_data",
>                 "Destination": "/var/lib/postgresql/data",
>                 "Driver": "local",
>                 "Mode": "",
>                 "RW": true,
>                 "Propagation": ""
>             }

Answer 1

You can get the information about the connected volume location

$ docker inspect <container_name>

From the output, you can get the "Source".

The reference folder contains the files from the location /var/log of the respective container

$ ls -lah /var/lib/docker/volumes/b2.....1a/_data