TechQA.

Kamal deployment: error "Access denied for user" with mysql

250 views Asked by At

I'm trying to kamal setup my app by following this tutorial.

I set up 2 servers (one for the Ruby on Rails 7.1.2 app and one for the MySQL 8.0.31 database, both are Linux Ubuntu 23.10) and a NodeBalancer running on Linode.

My deploy.yml looks like this (I just changed sensitive references and removed comments):

service: my-app

image: myusername/my-app

servers:
  web:
    hosts:
      - 1.1.1.79

registry:
  username: myusername
  password:
    - KAMAL_REGISTRY_PASSWORD

env:
  clear:
    DB_HOST: 1.1.1.172
  secret:
    - RAILS_MASTER_KEY
    - MYSQL_ROOT_PASSWORD

accessories:
  db:
    image: mysql:8.0
    host: 1.1.1.172
    port: 3306
    env:
      clear:
        MYSQL_ROOT_HOST: '%'
      secret:
        - MYSQL_ROOT_PASSWORD
    files:
      # - config/mysql/production.cnf:/etc/mysql/my.cnf
      - db/production.sql:/docker-entrypoint-initdb.d/setup.sql
    directories:
      - data:/var/lib/mysql

My Dockerfile looks like this (sensitive references changed):

# syntax = docker/dockerfile:1

# Make sure RUBY_VERSION matches the Ruby version in .ruby-version and Gemfile
ARG RUBY_VERSION=3.2.2
FROM registry.docker.com/library/ruby:$RUBY_VERSION-slim as base

# Rails app lives here
WORKDIR /var/www/my-app.com/public

# Set production environment
ENV RAILS_ENV="production" \
    BUNDLE_DEPLOYMENT="1" \
    BUNDLE_PATH="/usr/local/bundle" \
    BUNDLE_WITHOUT="development"

# Install packages needed to run the app: OS-level dependencies
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y libmariadb-dev nodejs && \
    rm -rf /var/lib/apt/lists/* /var/cache/apt/archives


# Throw-away build stage to reduce size of final image
FROM base as build_stage

# Install packages needed to build gems
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential git libvips pkg-config

# Install application gems
COPY Gemfile Gemfile.lock ./
RUN bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \
    bundle exec bootsnap precompile --gemfile

# Copy application code
COPY . .

# Precompile bootsnap code for faster boot times
RUN bundle exec bootsnap precompile app/ lib/

# We precompile assets locally. So, we disable assets precompile task.
## Precompiling assets for production without requiring secret RAILS_MASTER_KEY
#RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile


# Final stage for app image
FROM base

# Install packages needed for deployment
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y curl libsqlite3-0 libvips && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Copy built artifacts: gems, application
COPY --from=build_stage /usr/local/bundle /usr/local/bundle
COPY --from=build_stage /var/www/my-app.com/public /var/www/my-app.com/public

# Run and own only the runtime files as a non-root user for security
RUN useradd rails --create-home --shell /bin/bash && \
    chown -R rails:rails db log storage tmp
USER rails:rails

# Entrypoint prepares the database.
ENTRYPOINT ["/var/www/my-app.com/public/bin/docker-entrypoint"]

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD ["./bin/rails", "server"]

In the .env file I've:

KAMAL_REGISTRY_PASSWORD=secret_kamal_password
RAILS_MASTER_KEY=secret_master_key
MYSQL_ROOT_PASSWORD=<%= ENV['MYSQL_ROOT_PASSWORD'] %>

# Note: `ENV['MYSQL_ROOT_PASSWORD']` comes from setting environment variables using credentials (see https://dev.to/dalezak/rails-environment-variables-using-credentials-mh7)
# I also tried the following without success (please read bottom notes):
# MYSQL_ROOT_PASSWORD=secret_mysql_password (an explicit password)
# MYSQL_ROOT_PASSWORD=

In the config/database.yml file I've:

default: &default
  adapter: mysql2
  encoding: utf8mb4
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  username: root
  password:
  socket: /tmp/mysql.sock
  variables:
    sql_mode: TRADITIONAL

# ...

production:
  <<: *default
  host: <%= ENV["DB_HOST"] %>
  database: my_app_production
  username: root
  password: <%= ENV['MYSQL_ROOT_PASSWORD'] %>
  adapter: mysql2
  encoding: utf8mb4
  reconnect: false
  socket: /var/run/mysqld/mysqld.sock
  variables:
    sql_mode: TRADITIONAL

I also setup routes get 'up' => 'rails/health#show', :as => :rails_health_check, and many other things in app and on servers... please, ask for more info.

Now, I feel like I'm nearing the end of setup in seeing my app finally deployed.

When in the console I run kamal setup then I get:

$ kamal setup
  INFO [e7ea88de] Running /usr/bin/env mkdir -p .kamal on 1.1.1.79
  INFO [e7ea88de] Finished in 0.617 seconds with exit status 0 (successful).
Acquiring the deploy lock...
Ensure Docker is installed...
  INFO [bd598bad] Running docker -v on 1.1.1.79
  INFO [5cfc802f] Running docker -v on 1.1.1.172
  INFO [bd598bad] Finished in 0.188 seconds with exit status 0 (successful).
  INFO [8595d5d3] Running /usr/bin/env mkdir -p .kamal on 1.1.1.79
  INFO [8595d5d3] Finished in 0.166 seconds with exit status 0 (successful).
  INFO [5cfc802f] Finished in 0.825 seconds with exit status 0 (successful).
  INFO [4e3cde99] Running /usr/bin/env mkdir -p .kamal on 1.1.1.172
  INFO [4e3cde99] Finished in 0.174 seconds with exit status 0 (successful).
Push env files...
  INFO [7bd3548a] Running /usr/bin/env mkdir -p .kamal/env/roles on 1.1.1.79
  INFO [7bd3548a] Finished in 0.167 seconds with exit status 0 (successful).
  INFO Uploading .kamal/env/roles/my-app-web.env 100.0%
  INFO [6b8a9939] Running /usr/bin/env mkdir -p .kamal/env/traefik on 1.1.1.79
  INFO [6b8a9939] Finished in 0.164 seconds with exit status 0 (successful).
  INFO Uploading .kamal/env/traefik/traefik.env 100.0%
  INFO [f3ae511b] Running /usr/bin/env mkdir -p .kamal/env/accessories on 1.1.1.172
  INFO [f3ae511b] Finished in 0.090 seconds with exit status 0 (successful).
  INFO Uploading .kamal/env/accessories/my-app-db.env 100.0%
  INFO [52493b0b] Running /usr/bin/env mkdir -p $PWD/my-app-db/data on 1.1.1.172
  INFO [52493b0b] Finished in 0.160 seconds with exit status 0 (successful).
  INFO [97cd379e] Running /usr/bin/env mkdir -p my-app-db/docker-entrypoint-initdb.d on 1.1.1.172
  INFO [97cd379e] Finished in 0.168 seconds with exit status 0 (successful).
  INFO Uploading /Users/MyUser/Sites/my-app/db/production.sql 100.0%
  INFO [6222be4b] Running /usr/bin/env chmod 755 my-app-db/docker-entrypoint-initdb.d/setup.sql on 1.1.1.172
  INFO [6222be4b] Finished in 0.161 seconds with exit status 0 (successful).
  INFO [4612e419] Running docker login -u [REDACTED] -p [REDACTED] on 1.1.1.172
  INFO [4612e419] Finished in 1.566 seconds with exit status 0 (successful).
  INFO [66336809] Running docker run --name my-app-db --detach --restart unless-stopped --log-opt max-size="10m" --publish 3306:3306 --env-file .kamal/env/accessories/my-app-db.env --volume $PWD/my-app-db/docker-entrypoint-initdb.d/setup.sql:/docker-entrypoint-initdb.d/setup.sql --volume $PWD/my-app-db/data:/var/lib/mysql --label service="my-app-db" mysql:8.0 on 1.1.1.172
  INFO [66336809] Finished in 0.880 seconds with exit status 0 (successful).
Log into image registry...
  INFO [b7e078b2] Running docker login -u [REDACTED] -p [REDACTED] as MyUser@localhost
  INFO [b7e078b2] Finished in 1.761 seconds with exit status 0 (successful).
  INFO [d894d0e4] Running docker login -u [REDACTED] -p [REDACTED] on 1.1.1.79
  INFO [d894d0e4] Finished in 1.014 seconds with exit status 0 (successful).
Build and push app image...
  INFO [222fcd63] Running docker --version && docker buildx version as MyUser@localhost
  INFO [222fcd63] Finished in 0.209 seconds with exit status 0 (successful).
The following paths have uncommitted changes:
 M Dockerfile
  INFO [c4cfd7ed] Running docker buildx build --push --platform linux/amd64,linux/arm64 --builder kamal-my-app-multiarch -t lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 -t lapser/my-app:latest --label service="my-app" --file Dockerfile . as MyUser@localhost
 DEBUG [c4cfd7ed] Command: docker buildx build --push --platform linux/amd64,linux/arm64 --builder kamal-my-app-multiarch -t lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 -t lapser/my-app:latest --label service="my-app" --file Dockerfile .
 DEBUG [c4cfd7ed]   #0 building with "kamal-my-app-multiarch" instance using docker-container driver
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #1 [internal] load build definition from Dockerfile
 DEBUG [c4cfd7ed]   #1 transferring dockerfile: 3.58kB done
 DEBUG [c4cfd7ed]   #1 DONE 0.0s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #2 [auth] docker/dockerfile:pull token for registry-1.docker.io
 DEBUG [c4cfd7ed]   #2 DONE 0.0s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #3 resolve image config for docker.io/docker/dockerfile:1
 DEBUG [c4cfd7ed]   #3 DONE 1.1s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #4 docker-image://docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021
 DEBUG [c4cfd7ed]   #4 resolve docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31be0021 done
 DEBUG [c4cfd7ed]   #4 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #5 [linux/arm64 internal] load metadata for registry.docker.com/library/ruby:3.2.2-slim
 DEBUG [c4cfd7ed]   #5 DONE 1.1s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #6 [linux/amd64 internal] load metadata for registry.docker.com/library/ruby:3.2.2-slim
 DEBUG [c4cfd7ed]   #6 DONE 1.1s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #7 [internal] load .dockerignore
 DEBUG [c4cfd7ed]   #7 transferring context: 782B 0.0s done
 DEBUG [c4cfd7ed]   #7 DONE 0.0s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #8 [linux/amd64 base 1/4] FROM registry.docker.com/library/ruby:3.2.2-slim@sha256:14eba677236d3360f2b66595c7fa0f2440ed6e33c519befe4d11c1242a8815a8
 DEBUG [c4cfd7ed]   #8 resolve registry.docker.com/library/ruby:3.2.2-slim@sha256:14eba677236d3360f2b66595c7fa0f2440ed6e33c519befe4d11c1242a8815a8 done
 DEBUG [c4cfd7ed]   #8 DONE 0.0s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #9 [linux/arm64 base 1/4] FROM registry.docker.com/library/ruby:3.2.2-slim@sha256:14eba677236d3360f2b66595c7fa0f2440ed6e33c519befe4d11c1242a8815a8
 DEBUG [c4cfd7ed]   #9 resolve registry.docker.com/library/ruby:3.2.2-slim@sha256:14eba677236d3360f2b66595c7fa0f2440ed6e33c519befe4d11c1242a8815a8 done
 DEBUG [c4cfd7ed]   #9 DONE 0.0s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #10 [internal] load build context
 DEBUG [c4cfd7ed]   #10 transferring context: 3.32MB 0.2s done
 DEBUG [c4cfd7ed]   #10 DONE 0.2s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #11 [linux/amd64 base 3/4] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y libmariadb-dev &&     rm -rf /var/lib/apt/lists/* /var/cache/apt/archives
 DEBUG [c4cfd7ed]   #11 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #12 [linux/amd64 base 4/4] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y nodejs &&     rm -rf /var/lib/apt/lists/* /var/cache/apt/archives
 DEBUG [c4cfd7ed]   #12 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #13 [linux/amd64 build_stage 1/5] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y build-essential git libvips pkg-config
 DEBUG [c4cfd7ed]   #13 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #14 [linux/amd64 build_stage 2/5] COPY Gemfile Gemfile.lock ./
 DEBUG [c4cfd7ed]   #14 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #15 [linux/amd64 base 2/4] WORKDIR /var/www/my-app.com/public
 DEBUG [c4cfd7ed]   #15 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #16 [linux/amd64 build_stage 3/5] RUN bundle install &&     rm -rf ~/.bundle/ "/usr/local/bundle"/ruby/*/cache "/usr/local/bundle"/ruby/*/bundler/gems/*/.git &&     bundle exec bootsnap precompile --gemfile
 DEBUG [c4cfd7ed]   #16 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #17 [linux/arm64 base 4/4] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y nodejs &&     rm -rf /var/lib/apt/lists/* /var/cache/apt/archives
 DEBUG [c4cfd7ed]   #17 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #18 [linux/arm64 base 2/4] WORKDIR /var/www/my-app.com/public
 DEBUG [c4cfd7ed]   #18 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #19 [linux/arm64 build_stage 1/5] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y build-essential git libvips pkg-config
 DEBUG [c4cfd7ed]   #19 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #20 [linux/arm64 base 3/4] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y libmariadb-dev &&     rm -rf /var/lib/apt/lists/* /var/cache/apt/archives
 DEBUG [c4cfd7ed]   #20 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #21 [linux/arm64 build_stage 2/5] COPY Gemfile Gemfile.lock ./
 DEBUG [c4cfd7ed]   #21 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #22 [linux/arm64 build_stage 3/5] RUN bundle install &&     rm -rf ~/.bundle/ "/usr/local/bundle"/ruby/*/cache "/usr/local/bundle"/ruby/*/bundler/gems/*/.git &&     bundle exec bootsnap precompile --gemfile
 DEBUG [c4cfd7ed]   #22 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #23 [linux/arm64 build_stage 4/5] COPY . .
 DEBUG [c4cfd7ed]   #23 DONE 0.4s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #24 [linux/amd64 build_stage 4/5] COPY . .
 DEBUG [c4cfd7ed]   #24 DONE 0.4s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #25 [linux/amd64 build_stage 5/5] RUN bundle exec bootsnap precompile app/ lib/
 DEBUG [c4cfd7ed]   #25 ...
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #26 [linux/arm64 build_stage 5/5] RUN bundle exec bootsnap precompile app/ lib/
 DEBUG [c4cfd7ed]   #26 DONE 0.4s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #25 [linux/amd64 build_stage 5/5] RUN bundle exec bootsnap precompile app/ lib/
 DEBUG [c4cfd7ed]   #25 ...
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #27 [linux/arm64 stage-2 1/4] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y curl libsqlite3-0 libvips &&     rm -rf /var/lib/apt/lists /var/cache/apt/archives
 DEBUG [c4cfd7ed]   #27 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #28 [linux/arm64 stage-2 2/4] COPY --from=build_stage /usr/local/bundle /usr/local/bundle
 DEBUG [c4cfd7ed]   #28 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #29 [linux/arm64 stage-2 3/4] COPY --from=build_stage /var/www/my-app.com/public /var/www/my-app.com/public
 DEBUG [c4cfd7ed]   #29 DONE 0.6s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #25 [linux/amd64 build_stage 5/5] RUN bundle exec bootsnap precompile app/ lib/
 DEBUG [c4cfd7ed]   #25 DONE 3.7s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #30 [linux/arm64 stage-2 4/4] RUN useradd rails --create-home --shell /bin/bash &&     chown -R rails:rails db log storage tmp
 DEBUG [c4cfd7ed]   #30 ...
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #31 [linux/amd64 stage-2 1/4] RUN apt-get update -qq &&     apt-get install --no-install-recommends -y curl libsqlite3-0 libvips &&     rm -rf /var/lib/apt/lists /var/cache/apt/archives
 DEBUG [c4cfd7ed]   #31 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #32 [linux/amd64 stage-2 2/4] COPY --from=build_stage /usr/local/bundle /usr/local/bundle
 DEBUG [c4cfd7ed]   #32 CACHED
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #33 [linux/amd64 stage-2 3/4] COPY --from=build_stage /var/www/my-app.com/public /var/www/my-app.com/public
 DEBUG [c4cfd7ed]   #33 DONE 0.7s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #30 [linux/arm64 stage-2 4/4] RUN useradd rails --create-home --shell /bin/bash &&     chown -R rails:rails db log storage tmp
 DEBUG [c4cfd7ed]   #30 DONE 3.4s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #34 [linux/amd64 stage-2 4/4] RUN useradd rails --create-home --shell /bin/bash &&     chown -R rails:rails db log storage tmp
 DEBUG [c4cfd7ed]   #34 DONE 3.2s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #35 exporting to image
 DEBUG [c4cfd7ed]   #35 exporting layers
 DEBUG [c4cfd7ed]   #35 exporting layers 2.2s done
 DEBUG [c4cfd7ed]   #35 ...
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #36 [auth] lapser/my-app:pull,push token for registry-1.docker.io
 DEBUG [c4cfd7ed]   #36 DONE 0.0s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   #35 exporting to image
 DEBUG [c4cfd7ed]   #35 exporting manifest sha256:2cd6cf5c74826dbf3b85f054aab5b6f2e987f4386147ee6620b3658b084684e0 done
 DEBUG [c4cfd7ed]   #35 exporting config sha256:27c017f0ddf8d5228173262a04d5e635977bb5d08aee6af0e351da29786d488b done
 DEBUG [c4cfd7ed]   #35 exporting attestation manifest sha256:3c1cf36cee09c9c37b46310768958fca289a86b73750cdf7b8437636f14bf9c7 0.0s done
 DEBUG [c4cfd7ed]   #35 exporting manifest sha256:3ede7069bcabdc473a89d9b4370b87dfa824ecc224a71f3747d4d335b7197a7a done
 DEBUG [c4cfd7ed]   #35 exporting config sha256:c4562d45247f3e6f36909e7cd356e86ab3c237a5fe47f98ee8f677cbb7502180 done
 DEBUG [c4cfd7ed]   #35 exporting attestation manifest sha256:fc319fc7429a3ff8b381da0ad68568cbc36cab7efec627f03c68f596fd0fcf2e done
 DEBUG [c4cfd7ed]   #35 exporting manifest list sha256:4cec728f54cb136c3f5ab0f39e1bb45d553861783086a1bd1fe199cdb0593598 done
 DEBUG [c4cfd7ed]   #35 pushing layers
 DEBUG [c4cfd7ed]   #35 pushing layers 71.8s done
 DEBUG [c4cfd7ed]   #35 pushing manifest for docker.io/lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64@sha256:4cec728f54cb136c3f5ab0f39e1bb45d553861783086a1bd1fe199cdb0593598
 DEBUG [c4cfd7ed]   #35 pushing manifest for docker.io/lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64@sha256:4cec728f54cb136c3f5ab0f39e1bb45d553861783086a1bd1fe199cdb0593598 2.6s done
 DEBUG [c4cfd7ed]   #35 pushing layers 2.3s done
 DEBUG [c4cfd7ed]   #35 pushing manifest for docker.io/lapser/my-app:latest@sha256:4cec728f54cb136c3f5ab0f39e1bb45d553861783086a1bd1fe199cdb0593598
 DEBUG [c4cfd7ed]   #35 pushing manifest for docker.io/lapser/my-app:latest@sha256:4cec728f54cb136c3f5ab0f39e1bb45d553861783086a1bd1fe199cdb0593598 1.3s done
 DEBUG [c4cfd7ed]   #35 DONE 80.2s
 DEBUG [c4cfd7ed]   
 DEBUG [c4cfd7ed]   View build details: docker-desktop://dashboard/build/kamal-my-app-multiarch/kamal-my-app-multiarch0/w94mtrvdxpqsb5p9vvl1njxq3
  INFO [c4cfd7ed] Finished in 91.878 seconds with exit status 0 (successful).
  INFO [9cb2135f] Running docker image rm --force lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 on 1.1.1.79
  INFO [9cb2135f] Finished in 0.198 seconds with exit status 0 (successful).
  INFO [2e337c16] Running docker pull lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 on 1.1.1.79
  INFO [2e337c16] Finished in 10.800 seconds with exit status 0 (successful).
  INFO [e7dc0c4f] Running docker inspect -f '{{ .Config.Labels.service }}' lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 | grep -x my-app || (echo "Image lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 is missing the `service` label" && exit 1) on 1.1.1.79
  INFO [e7dc0c4f] Finished in 0.204 seconds with exit status 0 (successful).
Ensure Traefik is running...
  INFO [c63ee557] Running docker login -u [REDACTED] -p [REDACTED] on 1.1.1.79
  INFO [c63ee557] Finished in 1.120 seconds with exit status 0 (successful).
  INFO [247ebf65] Running docker container start traefik || docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --env-file .kamal/env/traefik/traefik.env --log-opt max-size="10m" --label traefik.http.routers.catchall.entryPoints="http" --label traefik.http.routers.catchall.rule="PathPrefix(\`/\`)" --label traefik.http.routers.catchall.service="unavailable" --label traefik.http.routers.catchall.priority="1" --label traefik.http.services.unavailable.loadbalancer.server.port="0" traefik:v2.9 --providers.docker --log.level="DEBUG" on 1.1.1.79
  INFO [247ebf65] Finished in 0.946 seconds with exit status 0 (successful).
Ensure app can pass healthcheck...
  INFO [7ce63c79] Running docker run --detach --name healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 --publish 3999:3000 --label service=healthcheck-my-app -e KAMAL_CONTAINER_NAME="healthcheck-my-app" --env-file .kamal/env/roles/my-app-web.env --health-cmd "curl -f http://localhost:3000/up || exit 1" --health-interval "1s" lapser/my-app:927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64 on 1.1.1.79
  INFO [7ce63c79] Finished in 1.126 seconds with exit status 0 (successful).
  INFO [ac3a41fd] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [ac3a41fd] Finished in 0.295 seconds with exit status 0 (successful).
  INFO container not ready (starting), retrying in 1s (attempt 1/7)...
  INFO [7204352c] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [7204352c] Finished in 0.225 seconds with exit status 0 (successful).
  INFO container not ready (starting), retrying in 2s (attempt 2/7)...
  INFO [003586fa] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [003586fa] Finished in 0.337 seconds with exit status 0 (successful).
  INFO container not ready (starting), retrying in 3s (attempt 3/7)...
  INFO [bc2ff3f7] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [bc2ff3f7] Finished in 0.152 seconds with exit status 0 (successful).
  INFO container not ready (unhealthy), retrying in 4s (attempt 4/7)...
  INFO [6fcf8e35] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [6fcf8e35] Finished in 0.188 seconds with exit status 0 (successful).
  INFO container not ready (unhealthy), retrying in 5s (attempt 5/7)...
  INFO [3a579318] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [3a579318] Finished in 0.161 seconds with exit status 0 (successful).
  INFO container not ready (unhealthy), retrying in 6s (attempt 6/7)...
  INFO [6215ece6] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [6215ece6] Finished in 0.140 seconds with exit status 0 (successful).
  INFO container not ready (unhealthy), retrying in 7s (attempt 7/7)...
  INFO [49217165] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}' on 1.1.1.79
  INFO [49217165] Finished in 0.140 seconds with exit status 0 (successful).
  INFO [bb11b11e] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker logs --tail 50 2>&1 on 1.1.1.79
  INFO [bb11b11e] Finished in 0.253 seconds with exit status 0 (successful).
 ERROR bin/rails aborted!
ActiveRecord::DatabaseConnectionError: There is an issue connecting to your database with your username/password, username: root. (ActiveRecord::DatabaseConnectionError)

Please check your database configuration to ensure the username/password are valid.


Caused by:
Mysql2::Error::ConnectionError: Access denied for user 'root'@'1.1.1.79' (using password: YES) (Mysql2::Error::ConnectionError)

Tasks: TOP => db:prepare
(See full trace by running task with --trace)
 ERROR {
  "Status": "unhealthy",
  "FailingStreak": 4,
  "Log": [
    {
      "Start": "2023-12-17T04:01:58.078447733+01:00",
      "End": "2023-12-17T04:01:58.405076209+01:00",
      "ExitCode": 1,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 3000 after 11 ms: Couldn't connect to server\n"
    },
    {
      "Start": "2023-12-17T04:01:59.417997425+01:00",
      "End": "2023-12-17T04:01:59.622977468+01:00",
      "ExitCode": 1,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 3000 after 6 ms: Couldn't connect to server\n"
    },
    {
      "Start": "2023-12-17T04:02:00.631456485+01:00",
      "End": "2023-12-17T04:02:01.085450669+01:00",
      "ExitCode": 1,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 3000 after 8 ms: Couldn't connect to server\n"
    },
    {
      "Start": "2023-12-17T04:02:02.095069251+01:00",
      "End": "2023-12-17T04:02:02.329358892+01:00",
      "ExitCode": 1,
      "Output": "  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\n                                 Dload  Upload   Total   Spent    Left  Speed\n\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\r  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0\ncurl: (7) Failed to connect to localhost port 3000 after 11 ms: Couldn't connect to server\n"
    }
  ]
}
  INFO [c6da487f] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker stop on 1.1.1.79
  INFO [c6da487f] Finished in 0.221 seconds with exit status 0 (successful).
  INFO [6ee63348] Running docker container ls --all --filter name=^healthcheck-my-app-927496d2cd69ad8968ea54f00672253bab7df794_uncommitted_97ebf665bf1d3b64$ --quiet | xargs docker container rm on 1.1.1.79
  INFO [6ee63348] Finished in 0.242 seconds with exit status 0 (successful).
  Finished all in 140.0 seconds
Releasing the deploy lock...
  Finished all in 147.3 seconds
  ERROR (Kamal::Cli::Healthcheck::Poller::HealthcheckError): Exception while executing on host 1.1.1.79: container not ready (unhealthy)

What's going wrong with the code relating the MySQL username/password?

Notes:

  • I get the same error even if I use an explicit password like MYSQL_ROOT_PASSWORD=secret_mysql_password.
  • I'm disappointed that the error seems to be happening on the server 1.1.1.79 that is not the database server. The database server is at 1.1.1.172
  • Notice that MySQL 8 by default on initialization doesn't set anymore the password as in previous versions. However, even if I don't set the password at all, the difference in log is that it outputs "Mysql2::Error::ConnectionError: Access denied for user 'root'@'1.1.1.79' (using password: NO)".

How to solve the issue?

ruby-on-rails docker deployment traefik kamal
Original Q&A
1

There are 1 answers

1
Mikita Pridorozhko On BEST ANSWER

Try to put DB_HOST to your .env file and then do kamal env push

Related Questions in RUBY-ON-RAILS

Related Questions in DOCKER

Related Questions in DEPLOYMENT

Related Questions in TRAEFIK

Related Questions in KAMAL

Popular Questions

Popular Tags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs node.js arrays c asp.net json python-3.x ruby-on-rails .net sql-server swift django angular objective-c pandas excel

Trending Questions