Kubernetes cluster pods as Jenkins Build Agents

Question

I have installed Kubernetes plugin - 1.23.3 in our Jenkins and able to execute shell commands on kubernetes pod(Dynamic Jenkins Slave). this is working with default jnlp & jenkins/jnlp-slave:latest

Suppose if I change the below "Name" & "Docker Image" section to use our private registry & image under

Manage Jenkins --> Configure System--> Cloud --> Pod Template --> Container Template

Name: sonatype
Docker image:sonatype:4546/ubuntu-16.04

It doesn’t uses our private docker image and even doesn't run the shell commands on the pod-containers. The idea is to perform build + static analysis using our own docker image on Kubernetes cluster pods as dynamic Jenkins Build Agents.

How to use our private docker registry images and execute them as Jenkins slave in kubernetes cluster? i have below scripted pipeline code. Stage-1 "SCM Code checkout" to K8S work-node is working, next 2nd stage build it is launching pod agent from template Kubernetes Pod Template. but if fails with docker: not found. it seems trying to pull our registry image in pod. From our worker-node system's code need to be mounted in dynamic Jenkins slave pods and perform the build and next stages. Any direction to achieve would be helpful.

node ("kubupods") { 
       stage('Code Compile') { 
          sh 'hostname'
        }
                
      stage('Code Analysis') {    
         sh 'hostname'
    }
    }
    
 

Answer 1

Jenkins by default will pull image from dockerhub. in your case the image sonatype:4546/ubuntu-16.04 . In order for you to use private registry you need to provide the private registry and its credentials if you are building in the pipeline. You can either provide it on the UI or you can do it via code as well . Refer here

After that you also need to tell kubernetes YAML as well about the private registry. For that you can refer here. This is essentially two steps a) create a kubernetes secret b) Tell your deployment about the secret using imagepullsecret field.

Make sure you refer to you image as <username or registry URL/<image_name>:<tag(maybe $BUILD_NUMBER)>

=============== Edit 1 after question has been added with a new problem =====

Configure plugins to install packages using Jenkins.

1. Go to Manage Jenkins

2. Global Tools Configuration

3. Docker -> Fill name (eg: Docker-latest) Check on install automatically and then add installer (Download from here).

4. Then save

If you have installed on your machine then update the PATH variable in Jenkins with the location of Docker.

Reference for Jenkins File

Answer 2

Tarun's answer above assumes that you are building the image in the pipeline.

I do something similar in our pipeline, but we build a custom jnlp-slave image (loaded with the tools we need for CI/CD) outside of the pipeline and refer it in the kubernetes plugin yaml.

Build the custom jnlp-slave image. (Dockerfile below)

FROM jenkins/jnlp-slave:latest

# Download/install tools

ENTRYPOINT ["jenkins-slave"]

Push the custom jnlp-slave image to your private registry

docker build -t my-private-registry/jnlp-slave:custom .
docker push my-private-registry/jnlp-slave:custom

Define your Jenkinsfile so that the pod uses your image you built in the previous step.

pipeline {
    agent {
        kubernetes {
            yaml """
apiVersion: v1
kind: Pod
metadata:
  label:
    jenkins: slave
spec:
  containers:
  - name: jnlp
    image: my-private-registry/jnlp-slave:custom
}}}

stages {
  stage("Test") {
    sh("hostname")
  }
}

Our registry doesn't require authentication, but if it does, you'll need to provide the secret (as Tarun has mentioned) to the jenkins to let it authenticate into your registry.