How to schedule replicas and persistent volume in different availability zones

Question

I have an AWS EKS cluster and I deployed the MongoDB replica set using operator version 0.9.0. The replica set deployment file I used is given below,

apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
  name: mongodb-test
  namespace: mongodb-test
spec:
  members: 3
  type: ReplicaSet
  version: "7.0.0"
  security:
    authentication:
      modes: ["SCRAM"]
  users:
    - name: mongouser
      db: admin
      passwordSecretRef: # a reference to the secret that will be used to generate the user's password
        name: password
      roles:
        - name: clusterAdmin
          db: admin
        - name: userAdminAnyDatabase
          db: admin
      scramCredentialsSecretName: mongodb-scram
  statefulSet:
    spec:
      template:
        spec:
          nodeSelector:
            server: mongo
          # resources can be specified by applying an override
          # per container name.
          containers:
            - name: mongod
              resources:
                limits:
                  cpu: "0.3"
                  memory: 700M
                requests:
                  cpu: "0.2"
                  memory: 500M
            - name: mongodb-agent
              resources:
                limits:
                  cpu: "0.2"
                  memory: 500M
                requests:
                  cpu: "0.1"
                  memory: 250M
      volumeClaimTemplates:
        - metadata:
            name: data-volume
          spec:
            resources:
              storageClassName: sc1
              requests:
                storage: 10Gi
        - metadata:
            name: logs-volume
          spec:
            storageClassName: sc1
            resources:
              requests:
                storage: 2Gi

Below is the commands I used to deploy mongodb,

kubectl apply -f mongodb-kubernetes-operator/crd/mongodbcommunity.mongodb.com_mongodbcommunity.yaml

kubectl get crd/mongodbcommunity.mongodbcommunity.mongodb.com

kubectl apply -k mongodb-kubernetes-operator/rbac/ --namespace mongodb-test

kubectl create -f mongodb-kubernetes-operator/manager/manager.yaml --namespace mongodb-test

kubectl apply -f replicaset/rbac -n mongodb-test

kubectl apply -f replicaset/replica-set.yaml -n mongodb-test

Is there any way I can,

1. Schedule MongoDB replicas in different availability zones.
2. Schedule the replica in the same availability zone as the replica's persistent volume.

        eg: replica1, replica1's persistent volume in eu-west-1a
            replica2, replica2's persistent volume in eu-west-1b
            replica, replica3's persistent volume in eu-west-1c

Operator Version: v0.9.0

MongoDB Image used: quay.io/mongodb/mongodb-community-server:7.0.0-ubi8

Kubernetes Cluster Version: v1.26.11-eks-8cb36c9

Answer 1

Make sure your StorageClass has the volumeBindingMode set to WaitForFirstConsumer. This ensures the volumes won't be provisioned until the pod is scheduled on the appropriate node.

Disclaimer: I have not used topologySpreadConstraints before but this should work.

apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
  name: mongodb-test
  namespace: mongodb-test
spec:
  members: 3
  type: ReplicaSet
  version: "7.0.0"
  security:
    authentication:
      modes: ["SCRAM"]
  users:
    - name: mongouser
      db: admin
      passwordSecretRef: # a reference to the secret that will be used to generate the user's password
        name: password
      roles:
        - name: clusterAdmin
          db: admin
        - name: userAdminAnyDatabase
          db: admin
      scramCredentialsSecretName: mongodb-scram
  statefulSet:
    spec:
      template:
        # Add a label to identify the pods for spread constraint
        metadata:
          labels:
            app.kubernetes.io/part-of: mongodb
        spec:
          # Add a topologySpreadConstraint to spread the pods across AZs
          topologySpreadConstraints:
          - maxSkew: 1
            topologyKey: topology.ebs.csi.aws.com/zone
            labelSelector:
              matchLabels:
                app.kubernetes.io/part-of: mongodb
          nodeSelector:
            server: mongo
          # resources can be specified by applying an override
          # per container name.
          containers:
            - name: mongod
              resources:
                limits:
                  cpu: "0.3"
                  memory: 700M
                requests:
                  cpu: "0.2"
                  memory: 500M
            - name: mongodb-agent
              resources:
                limits:
                  cpu: "0.2"
                  memory: 500M
                requests:
                  cpu: "0.1"
                  memory: 250M
      volumeClaimTemplates:
        - metadata:
            name: data-volume
          spec:
            resources:
              storageClassName: sc1
              requests:
                storage: 10Gi
        - metadata:
            name: logs-volume
          spec:
            storageClassName: sc1
            resources:
              requests:
                storage: 2Gi