Kubernetes plugin containers can't connect back to Jenkins

Question

I have a Jenkins and Kubernetes cluster running within the same network in AWS. Jenkins has it's own instance.

I have configured the Kubernetes plugin as follows:

The recommended JNLP docker image is used. Jenkins JNLP port is configured to be static 5000.

Now when I kick off the job, it shows me that the node is offline. When I click on the offline node I get this:

This makes me go to the k8 cluster. Running docker ps shows no containers running. However:

From there I go to find what docker container gets run and what logs it leaves after that:

I use: https://github.com/jenkinsci/docker-jnlp-slave as image

https://github.com/jenkinsci/kubernetes-plugin

Jenkins version: 2.27

k8: hyperkube:v1.4.3_coreos.0

Jenkins does spin up the container, I guess it runs and errors out because no valid arguments are provided during the container run? I need it to be a hands off process where I don't have to log in to my containers (Java clients). How do I achieve this?

UPDATE

Based on this answer: kubernetes slaves cannot register to jenkins master

If I log into the container and run the command that Jenkins displays under the host that cannot connect:

java -jar /usr/share/jenkins/slave.jar -jnlpUrl https://test.myhost.com/computer/jenkinsminions-10f0b7d49054ac/slave-agent.jnlp -secret 62637e83008f50eb94483ad609e9a2719d313fa56e640e4beca9eebeaf0b1af2

The container connects via JNLP2 and the job runs.

I tried to add the arguments as suggested, but no luck. Containers still won't connect automatically:

Answer 1

Do you have a Root directory not writable message in the container log?

[...]
Exception in thread "main" java.lang.RuntimeException: Root directory not writable
    at hudson.remoting.FileSystemJarCache.<init>(FileSystemJarCache.java:44)
    at hudson.remoting.Engine.<init>(Engine.java:139)
    at hudson.remoting.jnlp.Main.createEngine(Main.java:164)
    at hudson.remoting.jnlp.Main.main(Main.java:148)
    at hudson.remoting.jnlp.Main._main(Main.java:144)
    at hudson.remoting.jnlp.Main.main(Main.java:110)

In this case, you might have a problem similar to this.

PS: If you cannot see the logs, try removing the "Allocate pseudotty" option