Applicationset ArgoCD Helm with Git Matrix generator list and directory

Question

I am currently working on managing different clusters with the use of applicationsets. In the applicationset, we are using a matrix generator for GIT and the cluster desicision resource. In the applicationset, I want to use directories and files for the GIT generator. For the directories because we use subfolders for the values.yaml file per tenant using the cluster. Example:

env
├── ENV
│       ├── sub-folder1/values.yaml
│       ├── sub-folder2/values.yaml
│       └── sub-folder3/values.yaml
├── DEV
│       ├── sub-folder1/values.yaml
│       ├── sub-folder2/values.yaml
│       └── sub-folder3/values.yaml
└── PROD
        ├── sub-folder1/values.yaml
        ├── sub-folder2/values.yaml
        └── sub-folder3/values.yaml

For the file, we use a gitconfig.json file in which we can define which branch we want to use per cluster so that we can test and make changes per cluster. Example gitconfig.json file:

{
  "app": {
    "source": "https://github.com/_git/exampple",
    "targetRevision": "0.1.0",
    "path": "applications/namesapces/"
  },
  "destination": {
    "namespace": "default"
  },
  "clusters": {
    "eng": {
      "revision": "feature/0000"
    },
    "dev": {
      "revision": "main"
    }
  }
}

But unfortunately, this does not work. It seems that the file path is genealogised in the applicationset and only the directories path is used. This means that we cannot then change branches via pipelines and are thus always tied to the main branch.

Application set example:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: example-config
  namespace: example
name: example-config
namespace: openshift-gitops
spec:
  generators:
    - matrix:
        generators:
          - git:
              directories:
                - path: applications/namespaces/env/*/*
              files:
                - path: applications/persistent-volumes/gitconfig.json
              repoURL: >-
                "https://github.com/_git/example
              revision: main
          - clusterDecisionResource:
              configMapRef: aplacement
              labelSelector:
                matchLabels:
                  cluster.open-cluster-management.io/placement: placement
              requeueAfterSeconds: 180
  goTemplate: true
  template:
    metadata:
      name: '{{index .path.segments 3}}-pv-{{index .path.segments 4}}'
    spec:
      destination:
        name: '{{index .path.segments 3}}'
        namespace: '{{index .path.segments 3}}'
      project: default
      sources:
        - ref: values
          repoURL: '{{ .app.source }}'
          targetRevision: '{{ dig "clusters" .name "revision" "HEAD" . }}'
        - chart: '{{ default "" .app.chart }}'
          helm:
            valueFiles:
              - >-
                env/{{index .path.segments 3}}/{{index .path.segments
                4}}/values.yaml
          path: '{{ default "" .app.path }}'
          repoURL: '{{ .app.source }}'
          targetRevision: '{{ dig "clusters" .name "revision" "HEAD" . }}'

As you can see in the applicationset at sources, we are using templating in which we want to reference the gitconfig.json file that we define in the matrix git generator. But unfortunately that doesn't work.

We have also tried other setups. For example, using only the file path in the matrix git generator or other options but then we again run into problems that it seems ArgoCD loops through all the cluster files and then creates them twice.

Am I missing something? Should I use a different kind of generator or is this just not possible?