How prevent user from creating a pod with a specific label?

Question

I know how to use RBAC with X.509 certificates to identify a user of kubectl and restrict them (using Role and RoleBinding) from creating pods of any kind in a namespace. However, I don't know how I can prevent them from putting specific labels on a pod (or any resource) they're trying to create.

What I want to do is something like:

1. Create a NetworkPolicy that only resources in other namespaces with the label group: cross-ns are allowed to reach a resource in the special-namespace
2. Have a user who cannot create pods or other resources with the label group: cross-ns
3. Have another user who can create resources with the label group: cross-ns

Is this possible?

Answer 1

You can use the Kubernetes-native policy engine called Kyverno:

Kyverno runs as a dynamic admission controller in a Kubernetes cluster. Kyverno receives validating and mutating admission webhook HTTP callbacks from the kube-apiserver and applies matching policies to return results that enforce admission policies or reject requests.

A Kyverno policy is a collection of rules that can be applied to the entire cluster (ClusterPolicy) or to the specific namespace (Policy).

---

I will create an example to illustrate how it may work.

First we need to install Kyverno, you have the option of installing Kyverno directly from the latest release manifest, or using Helm (see: Quick Start guide):

$ kubectl create -f https://raw.githubusercontent.com/kyverno/kyverno/main/definitions/release/install.yaml

After successful installation, let's create a simple ClusterPolicy:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: labeling-policy
spec:
  validationFailureAction: enforce
  background: false
  rules:
  - name: deny-rule
    match:
      resources:
        kinds:
        - Pod
    exclude:
      clusterRoles:
      - cluster-admin
    preconditions:
      - key: "{{request.object.metadata.labels.purpose}}"
        operator: Equals
        value: "*"
    validate:
      message: "Using purpose label is not allowed for you"
      deny: {}

In the example above, only using the cluster-admin ClusterRole you can modify a Pod with a label purpose.

Suppose I have two users (john and dave), but only john is linked to the cluster-admin ClusterRole via ClusterRoleBinding:

$ kubectl describe clusterrolebinding john-binding
Name:         john-binding
Labels:       <none>
Annotations:  <none>
Role:
  Kind:  ClusterRole
  Name:  cluster-admin
Subjects:
  Kind  Name  Namespace
  ----  ----  ---------
  User  john

Finally, we can test if it works as expected:

$ kubectl run test-john --image=nginx --labels purpose=test --as john
pod/test-john created

$ kubectl run test-dave --image=nginx --labels purpose=test --as dave
Error from server: admission webhook "validate.kyverno.svc" denied the request:

resource Pod/default/test-dave was blocked due to the following policies

labeling-policy:
  deny-rule: Using purpose label is not allowed for you

$ kubectl get pods --show-labels
NAME        READY   STATUS    RESTARTS   AGE   LABELS
test-john   1/1     Running   0          32s   purpose=test

More examples with detailed explanations can be found in the Kyverno Writing Policies documentation.