Can’t start consul as service

Question

I installed consul on Ubuntu. consul version works fine, consul validate /etc/consul.d/ is valid. I'm trying to create a service that'll basically should do what this command does:

consul agent -server -ui -node=test -bootstrap-expect=1 -client=0.0.0.0 -data-dir /opt/consul -config-dir /etc/consul.d/

Here're configs:

/etc/consul.d/consul.hcl:

datacenter="dc1"
data_dir="/opt/consul"
encrypt="UEY...."

/etc/consul.d/server.hcl:

node_name="test"
server=true
bootstrap_expect=1
client_addr="0.0.0.0"
ui_config {
  enabled=true
}

/etc/systemd/system/consul.service:

[Unit]
Description="HashiCorp Consul - A service mesh solution"
Documentation=https://www.consul.io/
Requires=network-online.target
After=network-online.target
ConditionFileNotEmpty=/etc/consul.d/consul.hcl

[Service]
EnvironmentFile=-/etc/consul.d/consul.env
User=consul
Group=consul
ExecStart=/usr/bin/consul agent -config-dir=/etc/consul.d/
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
KillSignal=SIGTERM
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

After systemctl start consul when I check status I get this:

● consul.service - "HashiCorp Consul - A service mesh solution"
     Loaded: loaded (/etc/systemd/system/consul.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Wed 2023-10-18 14:45:40 +03; 5min ago
       Docs: https://www.consul.io/
    Process: 3172748 ExecStart=/usr/bin/consul agent -config-dir=/etc/consul.d/ (code=exited, status=1/FAILURE)
   Main PID: 3172748 (code=exited, status=1/FAILURE)

systemd[1]: consul.service: Scheduled restart job, restart counter is at 5.
systemd[1]: Stopped "HashiCorp Consul - A service mesh solution".
systemd[1]: consul.service: Start request repeated too quickly.
systemd[1]: consul.service: Failed with result 'exit-code'.
systemd[1]: Failed to start "HashiCorp Consul - A service mesh solution".

Interestingly if I manuelly run /usr/bin/consul agent -config-dir=/etc/consul.d/ it works fine.

I also found something weird in journalctl logs, it looks like counsel doesn't use the configs given as ExecStart parameters. Here's a sample of the logs:

==> Starting Consul agent
..
             Node name: '<server name>'

Here's the correct one when I start it manually with the same command in ExecStart:

==> Starting Consul agent...
..
             Node name: '<node name in config file>'

What should I do? I'd appreciate any feedback.

Answer 1

You should merge the files, "consul.hcl" and "server.hcl," to "consul.hcl":

datacenter="dc1"
data_dir="/opt/consul"
encrypt="UEY...."
node_name="test"
server=true
bootstrap_expect=1
client_addr="0.0.0.0"
ui_config {
  enabled=true
}

Answer 2

Your service configuration is running as the consul user. Possibly when you are testing it you are running it as a different user, or as root?

Here is how to create the user and set the permissions.

useradd --system --home /etc/consul.d --shell /bin/false consul
mkdir --parents /opt/consul
chown --recursive consul:consul /opt/consul
chown --recursive consul:consul /etc/consul.d
chmod 640 /etc/consul.d/consul.hcl
chmod 640 /etc/consul.d/server.hcl

This tutorial maybe will help too, I've successfully followed it in the past https://community.hetzner.com/tutorials/install-nomad-consul-cluster