I'm running a dockerized version of ElasticSearch and using volumes to map ElasticSearch directories to outside of docker.
My issue is that the size of the docker container (in /var/lib/docker/containers/ ) is growing at the same speed as the files that are in my /path/to/outside/docker/container directory. I have confirmed that contents inside of the docker container are the same as the contents of the /path/to... directory.
Is this duplication of data normal?
Docker info attached:
Server Version: 1.12.1
Storage Driver: devicemapper
Pool Name: docker-253:6-3151143-pool
Pool Blocksize: 65.54 kB
Base Device Size: 10.74 GB
Backing Filesystem: xfs
Data file: /dev/loop0
Metadata file: /dev/loop1
Data Space Used: 1.123 GB
Data Space Total: 107.4 GB
Data Space Available: 9.204 GB
Metadata Space Used: 2.966 MB
Metadata Space Total: 2.147 GB
Metadata Space Available: 2.145 GB
Thin Pool Minimum Free Space: 10.74 GB
Udev Sync Supported: true
Deferred Removal Enabled: false
Deferred Deletion Enabled: false
Deferred Deleted Device Count: 0
Data loop file: /var/lib/docker/devicemapper/devicemapper/data
WARNING: Usage of loopback devices is strongly discouraged for production use. Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.
Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
Library Version: 1.02.107-RHEL7 (2016-06-09)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 3.10.0-327.36.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.687 GiB
Name: bos-qa-elk-01
ID: CFHQ:37BL:JKQM:E5LC:IYHO:ZO3T:USKV:SGUW:XFYV:3IYJ:K5QD:U5PN
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Insecure Registries:
127.0.0.0/8
The filesystem inside your docker container consists of a couple things. The first is the root filesystem from the image- this is mounted from a copy on write filesystem supplied by the Docker storage volume. Inside of this, various "volumes" can be mounted. A volume is different because it does not use the copy on write filesystem, it uses the base filesystem of your system, and is mounted inside the rootfs at the location you specified. If you choose a host location for your volume, that directory/file will be mounted inside the container filesystem- if you do not, docker will create a directory for you and mount that.
So.
You created a volume on your host system and are storing data in it. The data actually exists at that location on your host. Inside the container, the volume directory is mounted, so you see the same data there as well. This doesn't mean that the data is duplicated, just that you can see it in multiple places.