Unable to get an Azure Container App Job to scale up on Redis KEDA trigger

Question

I have set up a simple Container App Job that is meant to trigger when there are items present in a Redis cache queue. The Redis cache is in Azure as well. I was having issues getting the proper KEDA configuration to get the job to connect to the Redis instance, but the configuration below eliminated my connection errors. However, I cannot seem to get the job to trigger even though I have placed items into the queue.

Here is the Bicep for my job:

param location string = resourceGroup().location

param containerAppsEnvironmentName string
param containerAppJobName string
param imageName string
param environmentVariables array
param containerRegistryName string
param containerRegistryConfiguration object
param cpuCores string
param memory string
param redisCacheName string
@secure()
param containerRegistryPasswordSecret object
@secure()
param databasePasswordSecret object
@secure()
param redisKeySecret object
@secure()
param railsMasterKeySecret object

resource containerAppsEnvironment 'Microsoft.App/managedEnvironments@2022-11-01-preview' existing = {
  name: containerAppsEnvironmentName
  scope: resourceGroup()
}
var containerAppsEnvironmentId = containerAppsEnvironment.id

resource redis 'Microsoft.Cache/redis@2023-08-01' existing = {
  name: redisCacheName
  scope: resourceGroup()
}
var redisAddress = '${redis.properties.hostName}:${redis.properties.sslPort}'

resource containerAppImporterJob 'Microsoft.App/jobs@2023-05-02-preview' = {
  location: location
  name: containerAppJobName
  properties: {
    environmentId: containerAppsEnvironmentId
    configuration: {
      replicaTimeout: 120 
      secrets: [containerRegistryPasswordSecret, databasePasswordSecret, redisKeySecret, railsMasterKeySecret]
      triggerType: 'Event'
      eventTriggerConfig: {
        scale: {
          minExecutions: 0
          maxExecutions: 10
          rules: [
            {
              name: 'redis-importer-queue-rule'
              type: 'redis'
              auth: [
                {
                  secretRef: 'redis-key'
                  triggerParameter: 'password'
                }
              ]
              metadata: any(
                {
                  address: redisAddress
                  password: 'redis-key'
                  listName: 'imports'
                  listLength: '1'
                  activationListLength: '1'
                  enableTLS: 'true'
                  unsafeSsl: 'false'
                }
              ) 
            }
          ]
        }
      }
      registries: [
        containerRegistryConfiguration
      ]
    }
    template: {
      containers: [
        {
          image: '${containerRegistryName}.azurecr.io/${imageName}:latest'
          command: [
            '/bin/bash'
            '-c'
            'bin/importer'
          ]
          env: environmentVariables
          name: imageName
          resources: {
            cpu: cpuCores
            memory: memory
          }
        }
      ]
    }
  }
}

I can confirm that there is exactly 1 item in the imports queue (using the Azure Cache extension in VS Code):

The logs for the job simply say "Scaler redis is built" but nothing else.

Any thoughts? Has anyone successfully trigger a job from a generic KEDA configuration like this?

Answer 1

If you want to a job to be triggered when there is at least one message in the queue, you need to set the activationListLength to 0.

See explanation here:

Activation: Defines when the scaler is active or not and scales from/to 0 based on it.

In your case if the number of messages in the queue is <=1, the ScaledObject is not Active and it’ll scale to 0 if it’s allowed (when minExecutions: 0).

This template worked for me (container registry is not configured with managed identity/RBAC for simplicity):

param location string = resourceGroup().location
param containerRegistryName string = 'thomastestacr'
param redisCacheName string = 'thomastestredis'
param containerAppEnvName string = 'thomastestcenv'
param redisJobName string = 'redis-job'

resource containerRegistry 'Microsoft.ContainerRegistry/registries@2023-07-01' existing = {
  name: containerRegistryName
}

resource redisCache 'Microsoft.Cache/redis@2023-08-01' existing = {
  name: redisCacheName
}

resource containerAppEnv 'Microsoft.App/managedEnvironments@2023-05-01' existing = {
  name: containerAppEnvName
}

resource redisJob 'Microsoft.App/jobs@2023-08-01-preview' = {
  name: redisJobName
  location: location
  identity: {
    type: 'None'
  }
  properties: {
    environmentId: containerAppEnv.id
    workloadProfileName: 'Consumption'
    configuration: {
      secrets: [
        {
          name: 'acr-password'
          value: containerRegistry.listCredentials().passwords[0].value
        }
        {
          name: 'redis-key'
          value: redisCache.listKeys().primaryKey
        }
      ]
      triggerType: 'Event'
      replicaRetryLimit: 5
      replicaTimeout: 3600
      eventTriggerConfig: {
        replicaCompletionCount: 1
        parallelism: 1
        scale: {
          minExecutions: 0
          maxExecutions: 5
          pollingInterval: 20
          rules: [
            {
              name: 'redis-scaledobject'
              type: 'redis'              
              metadata: {
                address: '${redisCache.properties.hostName}:${redisCache.properties.sslPort}'
                listName: 'myqueue'
                listLength: '1'
                activationListLength: '0'
                enableTLS: 'true'
                unsafeSsl: 'false'
              }
              auth: [
                {
                  secretRef: 'redis-key'
                  triggerParameter: 'password'
                }
              ]
            }
          ]
        }
      }
      registries: [
        {
          server: '${containerRegistry.name}${environment().suffixes.acrLoginServer}'
          username: containerRegistry.listCredentials().username
          passwordSecretRef: 'acr-password'
        }
      ]
    }
    template: {
      containers: [
        {
          name: 'redis-job'
          image: '${containerRegistry.name}${environment().suffixes.acrLoginServer}/sample/hello-world:latest'
          command: []
          resources: {
            cpu: json('0.5')
            memory: '1Gi'
          }
        }
      ]
    }
  }
}

Connecting to the redis console, I was able to push a message:

LPUSH myqueue "Task1"

The scaler started scaling a nerw instance:

From the container app env log stream, i was able to see the scaler in action:

And see the executed jobs as well: