Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Container App Job Bicep Template Validation does not fit to Keda Azure Pipelines Trigger Specification #914

Closed
1 of 3 tasks
fl-lutz opened this issue Sep 26, 2023 · 7 comments
Assignees
Labels
enhancement New feature or request In progress Solution/feature is being worked on Jobs Relating to Azure Container Apps Jobs

Comments

@fl-lutz
Copy link

fl-lutz commented Sep 26, 2023

Please provide us with the following information:

This issue is a:

  • bug report
  • documentation issue or request
  • regression

Issue description

When deploying a container app job through a bicep template, it is not feasible to configure the scale rule metadata value "requireAllDemands" as a bool, as documented in the Keda Azure Pipelines Trigger. The Keda Scaler disregards the value if set as a string, rendering it ineffective.

Steps to reproduce

  1. Try deploying a Bicep Template similar to this one:
resource agentJob 'Microsoft.App/jobs@2023-05-01' = {
  name: jobName
  location: location
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${managedIdentityResourceId}': {}
    }
  }
  properties: {
    configuration: {
      eventTriggerConfig: {
        parallelism: 1
        replicaCompletionCount: 1
        scale: {
          minExecutions: 0
          maxExecutions: 1
          pollingInterval: 5
          rules: [
            {
              name: 'azure-pipelines'
              type: 'azure-pipelines'
              metadata: {
                poolName: agentPool
                demands: 'someDemand'
                requireAllDemands: false
              }
              auth: [
                {
                  secretRef: 'organization-url'
                  triggerParameter: 'organizationURL'
                }
                {
                  secretRef: 'personal-access-token'
                  triggerParameter: 'personalAccessToken'
                }
              ]
            }
          ]
        }
      }
      replicaTimeout: 240
      replicaRetryLimit: 1
      registries: [
        {
          server: '${acrName}.azurecr.io'
          identity: managedIdentityResourceId
        }
      ]
      secrets: [
        {
          name: 'personal-access-token'
          value: personalAccessToken
        }
        {
          name: 'organization-url'
          value: organizationUrl
        }
      ]
      triggerType: 'Event'
    }
    environmentId: acaenv.id
    template: {
      containers: [
        {
          image: '${acrName}.azurecr.io/${imageName}'
          name: jobName
          resources: {
            cpu: json('1')
            memory: '2Gi'
          }
          env: [
            {
              name: 'AZP_TOKEN'
              secretRef: 'personal-access-token'
            }
            {
              name: 'AZP_URL'
              secretRef: 'organization-url'
            }
            {
              name: 'AZP_POOL'
              value: agentPool
            }
            {
              name: 'someDemand'
              value: 'true'
            }
          ]
        }
      ]
    }
  }
}

Expected behavior
It deploys the Container App Job with the metadata field "requireAllDemands".
Actual behavior
Invalid request body for container apps job. Does not conform to Container Apps Job schema, please visit for more information https://docs.microsoft.com/azure/container-apps/azure-resource-manager-api-spec?tabs=arm-template#container-app

The deployment validation failed
Screenshots
Documentation of Keda Azure Pipelines Trigger:
image

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs: triage 🔍 Pending a first pass to read, tag, and assign label Sep 26, 2023
@owattley-rotageek
Copy link

owattley-rotageek commented Sep 27, 2023

requireAllDemands only exists in v2.10+ of the KEDA scaler. You can see the docs for v2.9 do not contain that metadata property:

https://keda.sh/docs/2.9/scalers/azure-pipelines/

image

Whereas v2.10 does:

https://keda.sh/docs/2.10/scalers/azure-pipelines/
image

The problem here is likely to do with Microsoft not disclosing which version of KEDA is in use behind the scenes. Versions of KEDA come out frequently and scalers seems to have breaking changes with each new version. It's basically impossible to know what to do without pure trial and error and a lot of guesswork.

I have the same problem with GitHub Runner scaler. I cannot work out which version Container Apps is using. My gut feel is it's currently v2.10 because the GitHub scaler didn't exist prior to v2.10, however this conflicts with what I have pasted above so I am left even more confused. Maybe Microsoft are using a custom build of KEDA with cherry-picked scalers just to make it even more fun.

@vinisoto vinisoto self-assigned this Sep 28, 2023
@vinisoto
Copy link
Collaborator

vinisoto commented Sep 28, 2023

Hi, in Container Apps and Container Apps Jobs, scaler metadata is passed as string in all cases. Looking at the Azure Pipelines Scaler code, it seems that it is parsing to bool https://github.com/kedacore/keda/blob/8d16e417f5a81b71d9028b90888b522fe9e0a136/pkg/scalers/azure_pipelines_scaler.go#L233

image

@vinisoto
Copy link
Collaborator

vinisoto commented Oct 4, 2023

@owattley-rotageek: you can check the kedaConfiguration property in the managed environment to see which KEDA version is being used:

image

@fl-lutz: Looks like there was fix for this particular scaler: kedacore/keda#4404 which went in KEDA 2.11 release (https://github.com/kedacore/keda/releases/tag/v2.11.0)

Container Apps currently uses KEDA 2.10.

@vinisoto vinisoto added enhancement New feature or request Jobs Relating to Azure Container Apps Jobs and removed Needs: triage 🔍 Pending a first pass to read, tag, and assign labels Oct 4, 2023
@vinisoto
Copy link
Collaborator

vinisoto commented Oct 4, 2023

We will rollout 2.11 in the next couple of weeks.

@vinisoto vinisoto added the In progress Solution/feature is being worked on label Oct 4, 2023
@DSpirit
Copy link

DSpirit commented Dec 3, 2023

@vinisoto may I kindly ask about an update regarding 2.11? The "requireAllDemands" functionality is something we would definitely need and until now, I still only see the version 2.10 in our Azure Container Environment.

@vinisoto
Copy link
Collaborator

vinisoto commented Dec 5, 2023

@DSpirit rollout started last week. We are expecting to complete it by the week of 12/11. By the way, we are upgrading to KEDA 2.12

@vinisoto
Copy link
Collaborator

vinisoto commented Jan 8, 2024

Support for KEDA 2.12 has now been rolled out globally. Closing this issue thanks @fl-lutz for your patience here.

@vinisoto vinisoto closed this as completed Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request In progress Solution/feature is being worked on Jobs Relating to Azure Container Apps Jobs
Projects
None yet
Development

No branches or pull requests

4 participants