I am using kafka Strimzi operator. I have made a deployment with resources request and limits as shown below.
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
name: kafka
labels:
app: strimzi
spec:
kafka:
version: 3.5.0
replicas: 3
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
resources:
requests:
memory: 16Gi
cpu: "8"
limits:
memory: 16Gi
cpu: "8"
readinessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
livenessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
config:
offsets.topic.replication.factor: 3
transaction.state.log.replication.factor: 3
transaction.state.log.min.isr: 2
default.replication.factor: 3
min.insync.replicas: 2
inter.broker.protocol.version: "3.4"
storage:
type: persistent-claim
size: 80Gi
deleteClaim: false
class: longhorn
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: kafka-metrics-config.yml
zookeeper:
replicas: 3
readinessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
livenessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
resources:
requests:
memory: 4Gi
cpu: "2"
limits:
memory: 4Gi
cpu: "2"
storage:
type: persistent-claim
size: 15Gi
deleteClaim: false
class: longhorn
metricsConfig:
type: jmxPrometheusExporter
valueFrom:
configMapKeyRef:
name: kafka-metrics
key: zookeeper-metrics-config.yml
entityOperator:
topicOperator: {}
userOperator: {}
kafkaExporter:
topicRegex: ".*"
groupRegex: ".*"
Having a look at prometheus/grafana, I am sseing the resource consumption limit per kafka pod is surpassed as shown in in the figures below
Can you please what i am misinterpreting/misunderstading? In fact, my understading is that the resource limit of a pod can not be surpassed in any way, and it will lead to throttling in case of CPU, and to an OOM for the memory case!!!