Setting resource annotations via OLM

39 views Asked by At

Problem

I am trying to use terraform to deploy a cluster with a predefined set of applications/resources. I want to deploy operators to manage my resources, but I am running into an issue when attempting to set resource annotations. The simple issue being that I am unsure how to pass the annotations to the resources.

I am still new to OLM so I am very likely just not aware of the existing solution, but any searches I run don't seem to explain how any annotations can be passed from OLM to any operators managed resources.

My question

Is it possible to pass annotations to OLM natively? or am I supposed to annotate the resource after its deployed? if so what happens when the resources is redeployed by OLM? If an OLM provisioned operator will just "operate" any follow-on resources, then can I just set the annotations in that resource?

  • A1: this seems to be "NO OLM does not natively support annotations"?
  • A2: OLM does not currently pass on any values to managed resources other than some Tolerations and resource limit/needs. (I have an issue submitted on this)
  • A3: Unsure as to what OLM does when a resource is recreated, but on upgrade most changes will be lost. Looks like patching is required here. :sad-face:

Should I try and move away from OLM to a Helm focused deployment instead? I know some applications have both OLM subscriptions AND Helm charts to deploy operators, but I'm not clear on the pros vs cons of the two. Any clarity there would be appreciated.

  • Status: WIP

What is required/ how big of a lift is it to extend the Subscription resource to enable passing values?

  • Status: WIP

Are there any other approaches that you know of to ensure my resources have the right annotations?

Thanks in advance for any advice!

Updates

Update 1
I found a resource in the external-secrets-operator called OperatorConfig that seems to be a solution? But I can't find any external documentation on it, so this may just be an external-secrets-operator thing?

Update 2
I found some talk by Cert-Manager on OLM configuration limitations. The way they describe it is less than heartening. Source

Update 3
Following up, I see that there are not a lot of options for enabling configurations via OLM. The issue seems to be that each time OLM updates an operator it overwrites the CSV of the operator, the Subscriptions seems to be the part that persists between different operator versions, but it has limited configuration options. Source

I have taken a shallow dive into what would be required to expand the subscription object, and created an issue (kind/feature) in the OLM repo on GitHub. Issue

As of right now, I think the answer is that "OLM is not currently capable of passing persistent values to operator managed resources"? Though, I have to question how OLM intends to remain relevant w/o it? Never the less, I am leaving this question open while I continue trying out my other potential options. I am also adding to my question with a 3rd option, "making a PR for the Subscription resource"

My current attempts

Example Templated Subscription CRD

apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  annotations: <maybe-here?> # I found one example with an annotations block here, but it was the CSV not subscriptions, and it was null valued (empty).
  name: ${svc-name}-operator
  namespace: operators
spec:
  channel: ${channel}
  name: ${op-name}
  source: ${catalog}
  sourceNamespace: olm
  installPlanApproval: Automatic
  configs:
    annotations: <maybe-here?> #except the docs say that only a select set of values are supported

Example local.merged.operators

"splunk" : {
  isDefaultApp = true
  name         = "splunk"
  source       = "operatorhubio-catalog"
  channel      = "stable"
}

Example Terraform for Subscription CRD kubectl apply

# Cycles through a pre-filtered list of map objects
# pulls the related templated subscription file
# then applies that
resource "kubernetes_manifest" "subscriptions" {
  for_each = local.merged.operators
  manifest = yamldecode(file(pathexpand("${path.module}/olm/subscriptions/${each.key}.yaml")))

  depends_on = [local_file.subscriptions]
}
0

There are 0 answers