After setting up a full production CI pipeline for a docker / ghost.org blog based site, I am attempting to setup a local development environment to more quickly develop themes, however with Ghost running in "Development Mode" and while passing "NODE_ENV = development" changes are not visible on browser refresh.
I am running the official Ghost Docker container (https://hub.docker.com/_/ghost/) locally, with Ghost properly in Development mode, changes made to the local host code (which were piped into the Docker Container via the -v volumes tag) are not visible on browser refresh.
The Back Story
Since I was at first running an NGinx reverse proxy in-front of the ghost container I started by attempting to tune my nginx.conf file since I assumed that the issue was cache based.
I added the following to my nginx.conf to attempt to disable all caching on my local ghost.org blog to make sure I was not caching my pages:
expires off;
I about Docker / NGinx reverse proxies having some issues regarding sendfile related to virtualbox (Docker restart not showing the desired effect), so I set sendfile off:
sendfile off;
When the above had no effect I completely removed the Reverse Proxy for my local development setup (to try to narrow down the possible issues). I assumed that this would solve the problem and allow a browser refresh to show my local changes, it did not.
After NGinx Reverse Proxy was Removed
At this point I was running ONLY the official ghost docker image (with no reverse proxy out front).
Obviously the issue was with ghost so my first impulse was that Ghost was running in "Production Mode" even though Ghost said it was running in "development mode" at the end of its startup. To make SURE it was running in development mode I removed the "Production" block options from my config.js ghost configuration file (which would cause ghost to error out if it indeed was in production mode).
I also added an echo statement to the beginning of the config file so that when Ghost spins up it will Echo out the current value of NODE_ENV (to make sure that is set properly to development).
Removed the container with:
docker rm ghost -f
And re-ran it successfully, ghost echoed "NODE_ENV = development" and booted, but again changes were not visible.
Are Code changes changing inside the container?
My next thought was that my changes might not be doing anything / being passed through into my docker container. Therefore I docker exec -it'd into the container with:
docker exec -it ghost bash
I installed vim inside of the container and then opened up one of the ghost files to verify the source before exiting VIM without saving. I then modified the file on my local host and saved it.
When I re-open that same file from within the Ghost container I was able to successfully view my changes therefore proving that volume based files on my local host are properly finding their way into the Ghost container and Should be able to be served / viewed / updated with a browser refresh since Ghost is in development mode.
Try a Different Container?
Next I tried a different container from Docker Hub (even though changing this would mean I have to make changes to my production pipeline to sync everything back up).
I searched Docker hub and found two of the most popular alternative Ghost.org docker containers, then pulled those down to test:
docker pull ptimof/ghost/ (2.9k pulls)
and
docker pull gold/ghost/ (1.9k pulls)]
Obviously MOST people use the official ghost docker image with 1 million + pulls so I was not confident that this would have any impact, however upon looking at the Dockerfile(s) for gold/ghost and ptimof/ghost both referenced a modification to permissions in the following line:
RUN chown -R user $GHOST_SOURCE/content
The above appeared to reference the production ghost blog and then modify the permissions for the content sub-directory that should then contain the themes folder and my code changes. I was a little hopeful at this point, but alas, after replacing the official container with each alternative container (even though the Dockerfiles appear identical) the problem persisted.
Another version of Ghost?
The last thing I tried was to spec out an older version of ghost since I suppose it could be an issue in the latest released container, I tried two previous versions but this had no impact.
StackOverflow to the rescue?
Since I couldn't think of anything else to try I thought I would post here before continuing my search on my own in the hopes that someone with more ghost experience, more docker experience (hopefully both) might stumble onto an answer.
The docker run command I am using is;
docker run --env NODE_ENV=development --name ghost -d -p 80:2368 -v ~/blog/ghost-letsencrypt/config/config dev.js:/usr/src/ghost/config.example.js -v ~/blog/ghost-letsencrypt/themes/:/usr/src/ghost/content/themes ghost:0.11.3
To re-state the problem: Everything spins up properly and my local ghost docker container successfully connects to my remote mysql DB as intended. My site is visible and correctly displayed in by browser but when I make changes to my local code neither a container restart or a browser refresh shows any of those changes.
Changes to the local code base are not properly displayed on browser refresh because the volume is mapping to the /src/ (Source) directory as opposed to the /var/lib/ghost working directory.
Ghost File Structure
Basically the ghost blog (inside a container or otherwise) has two identical file structures that can both contain theme / blog content. The Source directory and Working Directory.
The first one is located within the following directory: /usr/src/ghost/
This directory contains the base source ghost files that are only used / copied to the working directory during blog start-up if no additional theme / config files are found in the working directory.
Default Config would be in: /usr/src/ghost/config.example.js
Default Themes would be in: /usr/src/ghost/content/themes
Essentially the /src Source Directory is like a template that exists within every ghost installation for backup / reversion purposes only. It is in this directory that we copy / overwrite the default configuration file since when ghost spins up the first time it will then pull our MODIFIED default config into the working ghost directory.
This is where I (and maybe you) might have gone wrong since many docker containers copy their config file into the /src directory.
Working Ghost Files
The second file structure is where Ghost looks to find the source files which it will actually interact with when serving / creating static or dynamically created pages. The working directory is at the following location within the Ghost container: /var/lib/ghost
Themes that are currently being used by the working ghost server would be at: /var/lib/ghost/themes
So, to compare the Themes for the working directory with the source directory: /usr/src/ghost/content/themes /var/lib/ghost/themes
As you can see there is no "content" sub-directory within the /var/lib/ghost path while that "content" sub-directory does exist within the src / source file structure.
Our problem was that we were referencing the source directory / path from our docker volume as opposed to the working ghost directory. Therefore the blog would spin up properly (since the theme files DID exist on ghost launch, and would be successfully copied from there into our working ghost directory) but then would not be able to be modified since Ghost only looks at the source ONCE when starting for the first time.
That caused refreshing the browser or re-starting the containers to have no effect since ghost was already started for the first time. Ie. Ghost only copies from the source /src file structure ONCE when it first starts and then completely forgets / ignores that file structure from then on.
Any changes to files within the source file structure have no impact on Ghost once it is running so changes we made to files that were referenced by our volumes were not monitored by ghost and therefore were not displayed by the server / visible within the browser.