How to remove a label selector from a Kubernetes deployment?

Question

I'm trying to remove a key/value pair from an existing deployment's spec.selector.matchLabels config. For example, I'm trying to remove the some.old.label: blah label from spec.selector.matchLabels and spec.template.metadata.labels. So this is an excerpt of what I'm sending to kubectl apply -f:

    spec:
      selector:
        matchLabels:
          app: my-app
      template:
        metadata:
          labels:
            app: my-app

but that gives me the following error:

selector does not match template labels

I also tried kubectl replace, which gives me this error:

v1.LabelSelector{MatchLabels:map[string]string{“app”: "my-app"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

which makes sense once I checked the deployment's config in prod:

metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
       # my config is trying to mutate the matchLabels here:
       {"apiVersion":"apps/v1", ... "selector":{"matchLabels":{"app":"my-app"} ... }
  # etc...
spec:
  selector:
    matchLabels:
      app: my-app
      some.old.label: blah  # how do I remove this label from both the metadata.labels and matchLabels?
  template:
    metadata:
      labels:
        app: my-app
        some.old.label: blah # I want to remove this label

Notice how the some.old.label: blah key/value is set under selector.matchLabels and template.metadata.labels.

Will I have to delete-then-recreate my deployment? Or perhaps call kubectl replace --force?

Notes

I came across this section in the Kubernetes Deployment docs:

Selector removals removes an existing key from the Deployment selector -- do not require any changes in the Pod template labels. Existing ReplicaSets are not orphaned, and a new ReplicaSet is not created, but note that the removed label still exists in any existing Pods and ReplicaSets.

as well as this PR and this Github issue which speak about the reasoning behind the problem, but I can't figure out how I'm can safely update my deployment to remove this selector.

Answer 1

When the error message says "field is immutable", it means you can't change it once it's been set. You need to delete and recreate the Deployment with the label selector you want (which will also temporarily delete all of the matching Pods).

kubectl delete deployment my-app
kubectl apply -f ./deployment.yaml