How to increase dockerhub rate limits within kubeless?

Question

I have a kubeless version of v1.0.8 and I am building a machine learning mechanism that requires functions autoscaling on demand (approximately requests the generation of 100 pods per hour).

Being an anonymous Docker Hub user limits my downloads to 100 container image pull requests per six hours.

Is there any way to configure kubeless so as to include my Docker credentials secret during deployment?

Thank very much for you time.

Answer 1

A good start is to set the imagePullPolicy for your PodSpec to IfNotPresent, so that you'll only have to pull once per version per node.

Depending on the criticality of the workload you should also consider mirroring the image to a container registry you control. You don't want to be hitting rate limits when you need to roll out a hotfix at 3 AM.

Answer 2

This is what worked for EKS (AWS K8s)

1. Buy a dockerhub pro account.
2. Create a docker registry secret:

#!/bin/bash

for ns in $(kubectl get namespaces |grep -v NAME|awk '{print $1}')
do
   kubectl create secret docker-registry docker.registry \
       --docker-username=<MyAccountName> \
       --docker-password='MyDockerHubPassword' -n $ns
done

3. Patch all the dynamic service accounts in all the namesapces with the secret you created in step 2

for ns in $(kubectl get namespaces|grep -v NAME|awk '{print $1}')
do
        for sa in $(kubectl -n $ns get sa|grep -v SECRETS|awk '{print $1}')
        do
           kubectl patch serviceaccount $sa -p '{"imagePullSecrets": [{"name": "docker.registry"}]}' -n $ns
           if [ $? -eq 0 ]; then
                echo $ns $sa patched
           else
                echo Error patching $ns $sa
           fi
        done
done

Let me know how it goes.

Note: You will need to run the patch script (3) every time you deploy a new workload that depends on dockerhub.