Why I am not able to scale down my HPA based on the stabilization window parameter in the AWS EKS cluster?

15 views Asked by At

I am trying to scale my pods to N number of Pods by creating the replicas, but once the replicas are live, I want to scale the replica down in 30 minutes. My AWS EKS cluster scales down all the replicas immediately after 10 minutes.

I am using Python 3.8 and python kubernetes client 28.1.0.

Here is my class.


class K8sInfra:
    """ Class to hanlde the kubernetes infra structure"""
    def __init__(self, namespace,deployment_name,my_hpa) -> None:
        self.namespace = namespace
        self.deployment_name = deployment_name
        self.hpa_ = my_hpa

    def scale_pod(self, number_pods=10):
        """Scale a pod with the given name and namespace, with the given number of replicas.
        Also increase the scale down stabilizing window time

        Args:
            namespace: The namespace to create the pod in.
            deployment_name: The deplyment name for the interested pod.
            number_pods: The number of replicas of the pod to create.
        """
        
        # load the ocnfigurations
        config.load_kube_config()
        # use the right API
        api_instance = client.AppsV1Api()
        # set the  stabilization window time also during scaling down
        api_instance_1 = client.AutoscalingV2Api()
        # Retrieve the HPA you want to modify
        hpa_name = self.hpa_
        try:
            hpa = api_instance_1.read_namespaced_horizontal_pod_autoscaler(name=hpa_name, namespace=self.namespace)
        except client.exceptions.ApiException as e:
            if e.status == 404:
                exit(1)
            else:
                raise

        if not hpa.spec.behavior:
            hpa.spec.behavior = V2HorizontalPodAutoscalerBehavior()

        new_stabilization_window_seconds = 1800  

        if not hpa.spec.behavior:
            hpa.spec.behavior.scale_down = V2HPAScalingRules(stabilization_window_seconds=new_stabilization_window_seconds)
        updated_hpa = api_instance_1.replace_namespaced_horizontal_pod_autoscaler(name=hpa_name, namespace=self.namespace, body=hpa)
        print("HPA updated. status='%s'" % str(updated_hpa.status))
        time.sleep(2)
        # part where we scale the pod
        # Get the current pod spec.
        deployment = api_instance.read_namespaced_deployment(name=self.deployment_name, namespace=self.namespace)
        deployment.spec.replicas = number_pods

        api_response = api_instance.replace_namespaced_deployment(name=self.deployment_name, namespace=self.namespace, body=deployment)
        print("Deployment updated. status='%s'" % str(api_response.status))

I am not getting any error. What am I doing wrong here?

0

There are 0 answers