i'm trying to integrate aks workload identity with elastic-agent deamon set. The yaml of elastic-agent is automatically generated by elastic cloud instance and it is this one:
---
# For more information refer to https://www.elastic.co/guide/en/fleet/current/running-on-kubernetes-managed-by-fleet.html
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: elastic-agent
namespace: elastic-cloud
labels:
app: elastic-agent
spec:
selector:
matchLabels:
app: elastic-agent
template:
metadata:
labels:
app: elastic-agent
azure.workload.identity/use: "true"
spec:
selector:
matchLabels:
app: elastic-agent
template:
metadata:
labels:
app: elastic-agent
spec:
nodeSelector:
hype.type: "default"
# Tolerations are needed to run Elastic Agent on Kubernetes control-plane nodes.
# Agents running on control-plane nodes collect metrics from the control plane components (scheduler, controller manager) of Kubernetes
tolerations:
- key: node-role.kubernetes.io/control-plane
effect: NoSchedule
- key: node-role.kubernetes.io/master
effect: NoSchedule
serviceAccountName: elastic-agent
hostNetwork: true
# 'hostPID: true' enables the Elastic Security integration to observe all process exec events on the host.
# Sharing the host process ID namespace gives visibility of all processes running on the same host.
hostPID: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: elastic-agent
image: docker.elastic.co/beats/elastic-agent:8.8.2
env:
# Set to 1 for enrollment into Fleet server. If not set, Elastic Agent is run in standalone mode
- name: FLEET_ENROLL
value: "1"
# Set to true to communicate with Fleet with either insecure HTTP or unverified HTTPS
- name: FLEET_INSECURE
value: "true"
# Fleet Server URL to enroll the Elastic Agent into
# FLEET_URL can be found in Kibana, go to Management > Fleet > Settings
- name: FLEET_URL
value: "MY FLEET URL"
# Elasticsearch API key used to enroll Elastic Agents in Fleet (https://www.elastic.co/guide/en/fleet/current/fleet-enrollment-tokens.html#fleet-enrollment-tokens)
# If FLEET_ENROLLMENT_TOKEN is empty then KIBANA_HOST, KIBANA_FLEET_USERNAME, KIBANA_FLEET_PASSWORD are needed
- name: FLEET_ENROLLMENT_TOKEN
value: MY TOKEN
- name: KIBANA_HOST
value: "http://kibana:5601"
# The basic authentication username used to connect to Kibana and retrieve a service_token to enable Fleet
- name: KIBANA_FLEET_USERNAME
value: "elastic"
# The basic authentication password used to connect to Kibana and retrieve a service_token to enable Fleet
- name: KIBANA_FLEET_PASSWORD
value: "changeme"
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
securityContext:
runAsUser: 0
# The following capabilities are needed for 'Defend for containers' integration (cloud-defend)
# If you are using this integration, please uncomment these lines before applying.
#capabilities:
# add:
# - BPF # (since Linux 5.8) allows loading of BPF programs, create most map types, load BTF, iterate programs and maps.
# - PERFMON # (since Linux 5.8) allows attaching of BPF programs used for performance metrics and observability operations.
# - SYS_RESOURCE # Allow use of special resources or raising of resource limits. Used by 'Defend for Containers' to modify 'rlimit_memlock'
resources:
limits:
cpu: 400m
memory: 500Mi
requests:
cpu: 100m
memory: 500Mi
volumeMounts:
- name: proc
mountPath: /hostfs/proc
readOnly: true
- name: cgroup
mountPath: /hostfs/sys/fs/cgroup
readOnly: true
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: varlog
mountPath: /var/log
readOnly: true
- name: etc-full
mountPath: /hostfs/etc
readOnly: true
- name: var-lib
mountPath: /hostfs/var/lib
readOnly: true
- name: etc-mid
mountPath: /etc/machine-id
readOnly: true
- name: sys-kernel-debug
mountPath: /sys/kernel/debug
volumes:
- name: proc
hostPath:
path: /proc
- name: cgroup
hostPath:
path: /sys/fs/cgroup
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: varlog
hostPath:
path: /var/log
# The following volumes are needed for Cloud Security Posture integration (cloudbeat)
# If you are not using this integration, then these volumes and the corresponding
# mounts can be removed.
- name: etc-full
hostPath:
path: /etc
- name: var-lib
hostPath:
path: /var/lib
# Mount /etc/machine-id from the host to determine host ID
# Needed for Elastic Security integration
- name: etc-mid
hostPath:
path: /etc/machine-id
type: File
# Needed for 'Defend for containers' integration (cloud-defend)
# If you are not using this integration, then these volumes and the corresponding
# mounts can be removed.
- name: sys-kernel-debug
hostPath:
path: /sys/kernel/debug
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: elastic-agent
subjects:
- kind: ServiceAccount
name: elastic-agent
namespace: elastic-cloud
roleRef:
kind: ClusterRole
name: elastic-agent
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
namespace: elastic-cloud
name: elastic-agent
subjects:
- kind: ServiceAccount
name: elastic-agent
namespace: elastic-cloud
roleRef:
kind: Role
name: elastic-agent
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: elastic-agent-kubeadm-config
namespace: elastic-cloud
subjects:
- kind: ServiceAccount
name: elastic-agent
namespace: elastic-cloud
roleRef:
kind: Role
name: elastic-agent-kubeadm-config
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: elastic-agent
labels:
k8s-app: elastic-agent
rules:
- apiGroups: [""]
resources:
- nodes
- namespaces
- events
- pods
- services
- configmaps
# Needed for cloudbeat
- serviceaccounts
- persistentvolumes
- persistentvolumeclaims
verbs: ["get", "list", "watch"]
# Enable this rule only if planing to use kubernetes_secrets provider
#- apiGroups: [""]
# resources:
# - secrets
# verbs: ["get"]
- apiGroups: ["extensions"]
resources:
- replicasets
verbs: ["get", "list", "watch"]
- apiGroups: ["apps"]
resources:
- statefulsets
- deployments
- replicasets
- daemonsets
verbs: ["get", "list", "watch"]
- apiGroups:
- ""
resources:
- nodes/stats
verbs:
- get
- apiGroups: [ "batch" ]
resources:
- jobs
- cronjobs
verbs: [ "get", "list", "watch" ]
# Needed for apiserver
- nonResourceURLs:
- "/metrics"
verbs:
- get
# Needed for cloudbeat
- apiGroups: ["rbac.authorization.k8s.io"]
resources:
- clusterrolebindings
- clusterroles
- rolebindings
- roles
verbs: ["get", "list", "watch"]
# Needed for cloudbeat
- apiGroups: ["policy"]
resources:
- podsecuritypolicies
verbs: ["get", "list", "watch"]
- apiGroups: [ "storage.k8s.io" ]
resources:
- storageclasses
verbs: [ "get", "list", "watch" ]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: elastic-agent
# Should be the namespace where elastic-agent is running
namespace: elastic-cloud
labels:
k8s-app: elastic-agent
rules:
- apiGroups:
- coordination.k8s.io
resources:
- leases
verbs: ["get", "create", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: elastic-agent-kubeadm-config
namespace: elastic-cloud
labels:
k8s-app: elastic-agent
rules:
- apiGroups: [""]
resources:
- configmaps
resourceNames:
- kubeadm-config
verbs: ["get"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
annotations:
azure.workload.identity/client-id: MY WORKLOAD IDENITYT ID
name: elastic-agent
namespace: elastic-cloud
labels:
k8s-app: elastic-agent
---
with https://akv2k8s.io/ i just modify:
- name: FLEET_ENROLLMENT_TOKEN
value: MY TOKEN
with secretname@azurekeyvault. It automatically inject the secret inside pods. With workload identity i modify the service account creation adding:
annotations:
azure.workload.identity/client-id: MY WORKLOAD IDENITYT ID
and deamon set:
spec:
selector:
matchLabels:
app: elastic-agent
template:
metadata:
labels:
app: elastic-agent
azure.workload.identity/use: "true"
after that i follow microsoft doc https://learn.microsoft.com/en-us/azure/aks/workload-identity-deploy-cluster but i do not understand how i have to setup the value to call secret. In Microsoft example they do:
apiVersion: v1
kind: Pod
metadata:
name: quick-start
namespace: elastic-cloud
labels:
azure.workload.identity/use: "true"
spec:
serviceAccountName: elastic-agent
containers:
- image: ghcr.io/azure/azure-workload-identity/msal-go
name: oidc
env:
- name: KEYVAULT_URL
value: https://mykeyvault.vault.azure.net/
- name: SECRET_NAME
value: mysecret
i tried to test the sa with last yaml and it call correctly the secret.