Kubernetes copying jars into a pod and restart

Question

I have a Kubernetes problem where I need to copy 2 jars (each jar > 1Mb) into a pod after it is deployed. So ideally the solution is we cannot use configMap (> 1Mb) but we need to use "wget" in "initcontainer" and download the jars. so below is my kubernetes-template configuration which i have modified. The original one is available at https://github.com/dremio/dremio-cloud-tools/blob/master/charts/dremio/templates/dremio-executor.yaml

{{ if not .Values.DremioAdmin }}
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: dremio-executor
spec:
  serviceName: "dremio-cluster-pod"
  replicas: {{.Values.executor.count}}
  podManagementPolicy: "Parallel"
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      app: dremio-executor
  template:
    metadata:
      labels:
        app: dremio-executor
        role: dremio-cluster-pod
      annotations:
        dremio-configmap/checksum: {{ (.Files.Glob "config/*").AsConfig | sha256sum }}
    spec:
      terminationGracePeriodSeconds: 5
      {{- if .Values.nodeSelector }}
      nodeSelector:
        {{- range $key, $value := .Values.nodeSelector }}
        {{ $key }}: {{ $value }}
        {{- end }}
      {{- end }}
      containers:
      - name: dremio-executor
        image: {{.Values.image}}:{{.Values.imageTag}}
        imagePullPolicy: IfNotPresent
        securityContext:
          runAsUser: 0
        resources:
          requests:
            memory: {{.Values.executor.memory}}M
            cpu: {{.Values.executor.cpu}}
        volumeMounts:
        - name: dremio-executor-volume
          mountPath: /opt/dremio/data
       ##################### START added this section #####################
        - name: dremio-connector
          mountPath: /opt/dremio/jars
       #################### END added this section ##########################
        - name: dremio-config
          mountPath: /opt/dremio/conf
        env:
        - name: DREMIO_MAX_HEAP_MEMORY_SIZE_MB
          value: "{{ template "HeapMemory" .Values.executor.memory }}"
        - name: DREMIO_MAX_DIRECT_MEMORY_SIZE_MB
          value: "{{ template "DirectMemory" .Values.executor.memory }}"
        - name: DREMIO_JAVA_EXTRA_OPTS
          value: >-
            -Dzookeeper=zk-hs:2181
            -Dservices.coordinator.enabled=false
            {{- if .Values.extraStartParams }}
            {{ .Values.extraStartParams }}
            {{- end }}
        command: ["/opt/dremio/bin/dremio"]
        args:
        - "start-fg"
        ports:
        - containerPort: 45678
          name: server
      initContainers:
      ################ START added this section ######################
      - name: installjars
        image: {{.Values.image}}:{{.Values.imageTag}}
        imagePullPolicy: IfNotPresent 
        securityContext:
          runAsUser: 0
        volumeMounts:
        - name: dremio-connector
          mountPath: /opt/dremio/jars
        command: ["/bin/sh","-c"]
        args: ["wget --no-check-certificate -O /dir/connector.jar https://<some nexus repo URL>/connector.jar; sleep 10;"]
      ################ END added this section ###############
      - name: wait-for-zk
        image: busybox
        command:  ["sh", "-c", "until ping -c 1 -W 1 zk-hs > /dev/null; do echo waiting for zookeeper host; sleep 2; done;"]
      # since we're mounting a separate volume, reset permission to
      # dremio uid/gid
      - name: chown-data-directory
        image: {{.Values.image}}:{{.Values.imageTag}}
        imagePullPolicy: IfNotPresent
        securityContext:
          runAsUser: 0
        volumeMounts:
        - name: dremio-executor-volume
          mountPath: /opt/dremio/data
        command: ["chown"]
        args:
        - "dremio:dremio"
        - "/opt/dremio/data"
      volumes:
      - name: dremio-config
        configMap:
          name: dremio-config
      {{- if .Values.imagePullSecrets }}
      imagePullSecrets:
        - name: {{ .Values.imagePullSecrets }}
      {{- end}}
     #################### START added this section ########################
      - name: dremio-connector
        emptyDir: {}
     #################### END added this section ########################
  volumeClaimTemplates:
  - metadata:
      name: dremio-executor-volume
    spec:
      accessModes: [ "ReadWriteOnce" ]
      {{- if .Values.storageClass }}
      storageClassName: {{ .Values.storageClass }}
      {{- end }}
      resources:
        requests:
          storage: {{.Values.executor.volumeSize}}
{{ end }}

So the above is NOT working and I don't see any jars being downloaded once I "exec" into the pod. I don't understand what is wrong with the above. however do note that inside the pod if i run the same wget command it does download the jar which baffles me. So the URL is working, readwrite of directory is no problem but still jar is not downloaded ???

Answer 1

If you can remove the need for Wget altogether it would make life easier...

Option 1

Using your own docker image will save some pain if thats an option

Dockerfile

# docker build -f Dockerfile -t ghcr.io/yourOrg/projectId/dockerImageName:0.0.1 .
# docker push ghcr.io/yourOrg/projectId/dockerImageName:0.0.1

FROM nginx:1.19.10-alpine

# Use local copies of config
COPY files/some1.jar /dir/
COPY files/some2.jar /dir/

Files will be ready in the container, no need for cryptic commands in your pod definition that will make little sense. Alternatively if you need to download the files you could copy a script to do that work into the Docker image instead and run that on startup via the docker directive CMD.

Option 2

Alternatively, you could do a two stage deployment...

1. Create a persistent volume
2. mount the volume to a pod (use busybox as a base?) that will run for enough time for the files to copy across from your local machine (or for them to be downloaded if you continue to use Wget)
3. kubectl cp the files you need to the (Retained) PersistentVolume
4. Now mount the PV to your pod's container(s) so the files are readily available when the pod fires up.

Answer 2

Your approch seems right. Another solution could be to include the jar on the Docker image but I think it's not possible right ?

You could just use an emptyDir instead of a VolumeClaim.

Last one, I would have download the jar before waiting for ZooKeeper to gain some time.