Fix JWT::DecodeError (Nil JSON web token) for Gitlab runner / CI job?

1k views Asked by At

I am trying to install a simple job in Gitlab-CI. It should just copy most of the files (.php) in the repository to a different folder (from repository to Apache's /var/www/html/...).

The output goes like this:

Using Shell executor...
Preparing environment
Running on my.server...
Getting source from Git repository
Fetching changes with git depth set to 50...
Reinitialisierte existierendes Git-Projektarchiv in /home/gitlab-runner/builds/TyGXtawE/0/sdp-dev/ibo-php/.git/
fatal: unable to access 'https://gitlab-ci-token:[MASKED]@gitlab.sustainable-data-platform.org/sdp-dev/ibo-php.git/':
The requested URL returned error: 500

So I can find the corresponding problem in /var/log/gitlab/gitlab-rails/production.log

Started POST "/api/v4/jobs/request" for ip.ip.ip.ip at 2022-04-26 14:27:41 +0200
Started GET "/sdp-dev/ibo-php.git/info/refs?service=git-upload-pack" for ip.ip.ip.ip at 2022-04-26 14:27:44 +0200
Processing by Repositories::GitHttpController#info_refs as */*
  Parameters: {"service"=>"git-upload-pack", "repository_path"=>"sdp-dev/ibo-php.git"}
Filter chain halted as :authenticate_user rendered or redirected
Completed 401 Unauthorized in 14ms (Views: 1.3ms | ActiveRecord: 1.5ms | Elasticsearch: 0.0ms | Allocations: 4656)
Started GET "/sdp-dev/ibo-php.git/info/refs?service=git-upload-pack" for ip.ip.ip.ip at 2022-04-26 14:27:44 +0200
Processing by Repositories::GitHttpController#info_refs as */*
  Parameters: {"service"=>"git-upload-pack", "repository_path"=>"sdp-dev/ibo-php.git"}
Completed 500 Internal Server Error in 34ms (ActiveRecord: 7.6ms | Elasticsearch: 0.0ms | Allocations: 7564)
  
JWT::DecodeError (Nil JSON web token):
  
lib/gitlab/jwt_authenticatable.rb:17:in `decode_jwt_for_issuer'
lib/gitlab/workhorse.rb:210:in `decode_jwt'
# ... stacktrace

I have installed the runner from "https://gitlab-runner-downloads.s3.amazonaws.com/latest/rpm/gitlab-runner_amd64.rpm" and gave it the token from Gitlab's GUI.

Gitlab (v 14.5.2) and the runner (v 14.10.0) are running on the same machine.

All this might be overhead for some cp -r, but I would like to understand the error for upcoming more complex jobs. Any idea how to debug the registration-token, how to find the problem?

Update

403 for Gitlab Runner from Gitlab?

It seems like gitlab-runner just gets a 403 (forbidden) from Gitlab. So I might need to change something in the corresponding (Apache). But how can I debug this (ERROR: Checking for jobs... forbidden)?

How to register a runner?

There is this command: sudo gitlab-runner register --url $URL --registration-token $REGISTRATION_TOKEN. But when I put the right data (URL, REGISTRATION_TOKEN) in this command (not using bash variables!), I am asked for the same data again (URL, TOKEN) and I just do c/p. E.g. Type the URL [MY_URL]:, then I c/p MY_URL and hit enter. Is this OK?

token / unhealthy runner problems

No success after creating a new token in gitlab and registering a new runner, also after updating Gitlab to v14.10.

But I can see some problems in /var/log/messages.

systemd: Started GitLab Runner.
gitlab-runner: Runtime platform                                  #033[0;m  arch#033[0;m=amd64 os#033[0;m=linux pid#033[0;m=20848 revision#033[0;m=c6bb62f6 version#033[0;m=14.10.0
gitlab-runner: Starting multi-runner from /etc/gitlab-runner/config.toml...#033[0;m  builds#033[0;m=0
gitlab-runner: Running in system-mode.                           #033[0;m
gitlab-runner: #033[0;m
gitlab-runner: Configuration loaded                              #033[0;m  builds#033[0;m=0
gitlab-runner: listen_address not defined, metrics & debug endpoints disabled#033[0;m  builds#033[0;m=0
gitlab-runner: [session_server].listen_address not defined, session endpoints disabled#033[0;m  builds#033[0;m=0
gitlab-runner: #033[31;1mERROR: Checking for jobs... forbidden             #033[0;m  #033[31;1mrunner#033[0;m=TyGXtawE
gitlab-runner: #033[31;1mERROR: Checking for jobs... forbidden             #033[0;m  #033[31;1mrunner#033[0;m=iU4sqj-p
gitlab-runner: #033[31;1mERROR: Checking for jobs... forbidden             #033[0;m  #033[31;1mrunner#033[0;m=TyGXtawE
gitlab-runner: #033[31;1mERROR: Checking for jobs... forbidden             #033[0;m  #033[31;1mrunner#033[0;m=iU4sqj-p
gitlab-runner: #033[31;1mERROR: Checking for jobs... forbidden             #033[0;m  #033[31;1mrunner#033[0;m=TyGXtawE
gitlab-runner: #033[31;1mERROR: Runner {URL}/{token} is not healthy and will be disabled!#033[0;m
gitlab-runner: #033[31;1mERROR: Checking for jobs... forbidden             #033[0;m  #033[31;1mrunner#033[0;m=iU4sqj-p
gitlab-runner: #033[31;1mERROR: Runner {URL}/{token} is not healthy and will be disabled!#033[0;m

I am interested in listen_address not defined, metrics & debug endpoints disabled and Checking for jobs... forbidden.

You can see this problem elsewhere in SO, but I could not find a solution (googling etc.).

Here? https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28965

1

There are 1 answers

1
Richard On

Some things you might try:

  • Create a new token

  • Update Gitlab