spring-boot client unable to start with consul

Question

I have setup and run consul using docker on my system using following command:

sudo docker run -p 8500:8500 consul:0.9.2

Consul is running fine as I can check from the consul UI (Image attached below):

Now, I am trying to run my spring-boot service to use this consul instance for service discovery and registration. But, whenever i start it gives me following exception:

2017-09-02 18:58:17.091 ERROR 5578 --- [  restartedMain] o.s.c.c.c.ConsulPropertySourceLocator    : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:18.183 ERROR 5578 --- [  restartedMain] o.s.c.c.c.ConsulPropertySourceLocator    : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:19.375 ERROR 5578 --- [  restartedMain] o.s.c.c.c.ConsulPropertySourceLocator    : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:20.691 ERROR 5578 --- [  restartedMain] o.s.c.c.c.ConsulPropertySourceLocator    : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:22.114 ERROR 5578 --- [  restartedMain] o.s.c.c.c.ConsulPropertySourceLocator    : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:23.671 ERROR 5578 --- [  restartedMain] o.s.c.c.c.ConsulPropertySourceLocator    : Fail fast is set and there was an error reading configuration from consul.
2017-09-02 18:58:23.691 ERROR 5578 --- [  restartedMain] o.s.boot.SpringApplication               : Application startup failed

com.ecwid.consul.v1.OperationException: OperationException(statusCode=500, statusMessage='Internal Server Error', statusContent='No cluster leader')
    at com.ecwid.consul.v1.kv.KeyValueConsulClient.getKVValues(KeyValueConsulClient.java:159)
    at com.ecwid.consul.v1.ConsulClient.getKVValues(ConsulClient.java:487)
    at org.springframework.cloud.consul.config.ConsulPropertySource.init(ConsulPropertySource.java:66)
    at org.springframework.cloud.consul.config.ConsulPropertySourceLocator.create(ConsulPropertySourceLocator.java:157)
    at org.springframework.cloud.consul.config.ConsulPropertySourceLocator.locate(ConsulPropertySourceLocator.java:131)
    at org.springframework.cloud.consul.config.ConsulPropertySourceLocator$$FastClassBySpringCGLIB$$b35ebf8.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:91)
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:286)
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:163)
    at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:118)
    at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:152)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
    at org.springframework.cloud.consul.config.ConsulPropertySourceLocator$$EnhancerBySpringCGLIB$$66375879.locate(<generated>)
    at org.springframework.cloud.bootstrap.config.PropertySourceBootstrapConfiguration.initialize(PropertySourceBootstrapConfiguration.java:93)
    at org.springframework.boot.SpringApplication.applyInitializers(SpringApplication.java:567)
    at org.springframework.boot.SpringApplication.prepareContext(SpringApplication.java:338)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:301)
    at com.pyg.auth.AuthServiceApp.main(AuthServiceApp.java:71)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)

SpringBoot Main Class is annotated fine (I guess), because my spring-boot service was working fine with another consul instance earlier.

Answer 1

Check your pom.xml.

Don't use spring-cloud-starter-consul-all. Use spring-cloud-starter-consul-discovery instead.

That worked for me.

Answer 2

Use sudo docker run -p 8500:8500 consul:0.9.2 -bootstrap to resolve the problem. I have added -bootstrap one argument to bootstrap the server/servers and get elected as a leader.

Manual bootstrapping with -bootstrap is not recommended in newer versions of Consul (0.5 and newer) as it is more error-prone.

Explanation:-

As said in docs Before a Consul cluster can begin to service requests, a server node must be elected leader. And this is reason of your exception on start of spring-boot service the leader has not been elected yet!!

Why the leader has not been elected? The list of servers involved in the cluster should be bootstrapped. And the servers can be bootstrapped using the -bootstrap-expect configuration option. Recommended

Note:- Just for testing/learning purpose you can go ahead and create a single server because A single server deployment is highly discouraged as data loss is inevitable in a failure scenario.

Answer 3

I was able to solve my issue by moving consul server and port configuration from application.yml to bootstrap.yml.

I don't know much about how it was solved and why it was unable to read from application.yml. If anyone has some details about it, pl. let me know.

Answer 4

Please check pom.xml to avoid using spring-cloud-starter-consul-all If you use spring-cloud-starter-consul-all, please write the configuration to bootstrap.yml, the bootstrap.yml configuration file is the first to load.