I've set up a Flink cluster operating in session mode, with Kubernetes underneath. I'm using FlinkK8Operator with helm charts to launch my jobs. I've configured a simple HPA with the deployment, with the below YAML configuration for it. However, the issue is only the job manager pod is scaling, and not task manager pods. Is there a way for me to scale the task manager pods? Additionally, I'm wondering if using HPA for autoscaling the flink session deployment is the right approach?
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: sample-flink-session-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: sample-flink-session-deployment
minReplicas: 1
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 30