Error imagepullbackoff when scaling working deployment Kubernetes

Question

I have a working deployment of a docker image on Kubernetes. However, when I want to scale it I recieve and error that it cannot find my image (even though I'm scaling something that's already working from an image?)

Here is the command I'm using to scale the deployment.

./kubectl scale deployments/mautic --replicas=2

Here is the logs when I run Kubectl describe

Name:           mautic-3389378641-jgm9b
Namespace:      default
Node:           minikube/192.168.99.101
Start Time:     Fri, 01 Sep 2017 14:34:08 +0100
Labels:         app=mautic
                pod-template-hash=3389378641
                tier=frontend
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"default","name":"mau
tic-3389378641","uid":"52a87ff6-8f06-11e7-8fbc-080027cd66fa",...
Status:         Pending
IP:             172.17.0.8
Created By:     ReplicaSet/mautic-3389378641
Controlled By:  ReplicaSet/mautic-3389378641
Containers:
  mautic:
    Container ID:
    Image:              mautic/mautic:latest
    Image ID:
    Port:               80/TCP
    State:              Waiting
      Reason:           ErrImagePull
    Ready:              False
    Restart Count:      0
    Environment:
      MAUTIC_DB_HOST:           mautic-mysql
      MAUTIC_DB_PASSWORD:       <set to the key 'password.txt' in secret 'mysql-pass'>  Optional: false
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-9drh0 (ro)
      /var/www/html from mautic-local-storage (rw)
Conditions:
  Type          Status
  Initialized   True
  Ready         False
  PodScheduled  True
Volumes:
  mautic-local-storage:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  mautic-lv-claim
    ReadOnly:   false
  default-token-9drh0:
    Type:       Secret (a volume populated by a Secret)
    SecretName: default-token-9drh0
    Optional:   false
QoS Class:      BestEffort
Node-Selectors: <none>
Tolerations:    <none>
Events:
  FirstSeen     LastSeen        Count   From                    SubObjectPath           Type            Reason                  Message
  ---------     --------        -----   ----                    -------------           --------        ------                  -------
  47m           47m             1       default-scheduler                               Normal          Scheduled               Successfully assigned mauti
c-3389378641-jgm9b to minikube
  47m           47m             1       kubelet, minikube                               Normal          SuccessfulMountVolume   MountVolume.SetUp succeeded
 for volume "pvc-5b7c14a3-8f03-11e7-8fbc-080027cd66fa"
  47m           47m             1       kubelet, minikube                               Normal          SuccessfulMountVolume   MountVolume.SetUp succeeded
 for volume "default-token-9drh0"
  47m           7m              12      kubelet, minikube       spec.containers{mautic} Normal          Pulling                 pulling image "mautic/mauti
c:latest"
  47m           6m              12      kubelet, minikube       spec.containers{mautic} Warning         Failed                  Failed to pull image "mauti
c/mautic:latest": rpc error: code = 2 desc = Network timed out while trying to connect to https://index.docker.io/v1/repositories/mautic/mautic/images. You
 may want to check your internet connection or if you are behind a proxy.
  47m           6m              170     kubelet, minikube                               Warning         FailedSync              Error syncing pod
  47m           6m              158     kubelet, minikube       spec.containers{mautic} Normal          BackOff                 Back-off pulling image "mau
tic/mautic:latest"

But the Mautic image referenced is right here and is already in use with the deployment I want to scale.

REPOSITORY                                             TAG                 IMAGE ID            CREATED             SIZE
testimage                                              v0                  0e0d4b13c0c2        10 days ago         611MB
mautic/mautic                                          latest              730d2796f904        2 weeks ago         611MB
mysql                                                  5.6                 cdfa8cc50c33        5 weeks ago         298MB
mysql                                                  latest              c73c7527c03a        5 weeks ago         412MB
gcr.io/google_containers/k8s-dns-sidecar-amd64         1.14.4              38bac66034a6        2 months ago        41.8MB
gcr.io/google_containers/k8s-dns-kube-dns-amd64        1.14.4              a8e00546bcf3        2 months ago        49.4MB
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64   1.14.4              f7f45b9cb733        2 months ago        41.4MB
gcr.io/google-containers/kube-addon-manager            v6.4-beta.2         0a951668696f        2 months ago        79.2MB
gcr.io/google_containers/kubernetes-dashboard-amd64    v1.6.1              71dfe833ce74        3 months ago        134MB
autoize/mautic                                         latest              6c99d7ce1a07        4 months ago        665MB
gcr.io/google_containers/pause-amd64                   3.0                 99e59f495ffa        16 months ago       747kB

Pods

Does anyone have any idea why this isn't working?

Answer 1

Your host has lost connectivity to Docker Hub - try running "docker pull mautic/mautic:latest" and see if that works. This may be a network issue on your host, a problem with an intermediate proxy between your host and Docker Hub or (less likely) a temporary outage at Docker Hub or something else.

Since you're using the latest tag it is likely that your Deployment is using imagePullPolicy=Always. (docs: Defaults to Always if :latest tag is specified, or IfNotPresent otherwise.) I'd suggest explicitly setting imagePullPolicy=IfNotPresent in your Deployment spec so that the local image that already exists is used when starting a new container.