# Could not find host matching read preference { mode: "nearest" } for set rs0

**URL:** https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324
**Category:** Percona Operator for MongoDB
**Created:** [September 20, 2023, 2:15pm UTC](https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324 "2023-09-20T14:15:44Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![Kay\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/kay_khan/32/17023_2.png) [@Kay\_Khan](https://forums.percona.com/u/Kay_Khan)
#### Post date: [September 20, 2023, 2:15pm UTC](https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324/1 "2023-09-20T14:15:44Z")

</div>

## Description:

Unable to connect to mongodb after changing load balancer configuration scheme from `internet-facing` to `internal`. Is there anyway to recover mongodb from this error? I tried switching the configuration back and redeploying, but still same problem.

```auto
kubectl run -i --rm --tty percona-client --image=percona/percona-server-mongodb:5.0 --restart=Never \
  -- mongo "mongodb+srv://${ADMIN_USER}:${ADMIN_PASSWORD}@psmdb-db-rs0.mongodb.svc.cluster.local/admin?replicaSet=rs0&ssl=false"

```

> {“t”:{“$date”:“2023-09-20T14:10:39.819Z”},“s”:“I”, “c”:“NETWORK”, “id”:4333208, “ctx”:“ReplicaSetMonitor-TaskExecutor”,“msg”:“RSM host selection timeout”,“attr”:{“replicaSet”:“rs0”,“error”:“FailedToSatisfyReadPreference: Could not find host matching read preference { mode: "nearest" } for set rs0”}}  
> Error: Could not find host matching read preference { mode: “nearest” } for set rs0, rs0/psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017 :  
> connect@src/mongo/shell/mongo.js:372:17  
> @(connect):2:6

## Steps to Reproduce:

```auto
    expose:
      enabled: true
      exposeType: LoadBalancer
      serviceAnnotations:
        service.beta.kubernetes.io/aws-load-balancer-scheme: internal

```

## Version:

```auto
psmdb-db mongodb 3 2023-09-20 15:01:26.76669879 +0100 BST deployed psmdb-db-1.14.3 1.14.0
psmdb-operator mongodb 1 2023-09-20 14:12:14.052409889 +0100 BST deployed psmdb-operator-1.14.2 1.14.0

```

## Logs:

```auto
2023-09-20T14:17:35.989Z ERROR Reconciler error {"controller": "psmdb-controller", "object": {"name":"psmdb-db","namespace":"mongodb"}, "namespace": "mongodb", "name": "psmdb-db", "reconcileID": "1dcb0931-3c79-4865-aede-6035aca1cb8d", "error": "reconcile StatefulSet for rs0: failed to run smartUpdate: failed to check active jobs: getting PBM object: create PBM connection to psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017: create mongo connection: mongo ping: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017, Type: RSGhost, Average RTT: 749120 },] }", "errorVerbose": "reconcile StatefulSet for rs0: failed to run smartUpdate: failed to check active jobs: getting PBM object: create PBM connection to psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017: create mongo connection: mongo ping: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017, Type: RSGhost, Average RTT: 749120 },] }\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).Reconcile\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/psmdb_controller.go:412\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235
2023-09-20T14:17:36.031Z INFO StatefulSet is changed, starting smart update {"controller": "psmdb-controller", "object": {"name":"psmdb-db","namespace":"mongodb"}, "namespace": "mongodb", "name": "psmdb-db", "reconcileID": "3b5be707-93b3-4aea-922d-6a8dc71e67ab", "name": "psmdb-db-rs0"}
2023-09-20T14:18:07.050Z ERROR Reconciler error {"controller": "psmdb-controller", "object": {"name":"psmdb-db","namespace":"mongodb"}, "namespace": "mongodb", "name": "psmdb-db", "reconcileID": "3b5be707-93b3-4aea-922d-6a8dc71e67ab", "error": "reconcile StatefulSet for rs0: failed to run smartUpdate: failed to check active jobs: getting PBM object: create PBM connection to psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017: create mongo connection: mongo ping: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017, Type: RSGhost, Average RTT: 768844 },] }", "errorVerbose": "reconcile StatefulSet for rs0: failed to run smartUpdate: failed to check active jobs: getting PBM object: create PBM connection to psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017: create mongo connection: mongo ping: server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: psmdb-db-rs0-0.psmdb-db-rs0.mongodb.svc.cluster.local:27017, Type: RSGhost, Average RTT: 768844 },] }\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).Reconcile\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/psmdb_controller.go:412\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
 /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235
2023-09-20T14:18:07.132Z INFO StatefulSet is changed, starting smart update {"controller": "psmdb-controller", "object": {"name":"psmdb-db","namespace":"mongodb"}, "namespace": "mongodb", "name": "psmdb-db", "reconcileID": "2fee95ad-48f3-4665-8e8b-e3d2e3396d52", "name": "psmdb-db-rs0"}

```

## Expected Result:

**kubectl get all -n mongodb**

```auto
NAME READY STATUS RESTARTS AGE
pod/psmdb-db-rs0-0 1/1 Running 6 (2m49s ago) 21m
pod/psmdb-operator-869b9b99d-fm66v 1/1 Running 0 21m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/psmdb-db-rs0 ClusterIP None <none> 27017/TCP 61m
service/psmdb-db-rs0-0 LoadBalancer 172.20.23.165 k8s-mongodb-psmdbdbr-03fa115926-cb82804e1fb78c41.elb.eu-west-2.amazonaws.com 27017:30564/TCP 61m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/psmdb-operator 1/1 1 1 62m

NAME DESIRED CURRENT READY AGE
replicaset.apps/psmdb-operator-869b9b99d 1 1 1 62m

NAME READY AGE
statefulset.apps/psmdb-db-rs0 1/1 61m

```

Expect to be able to connect into mongodb instance

## Actual Result:

 ![image](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/1/1a84d2cee30f9e7c95feb857c8a409b9cdcc428c.png)

## Additional Information:

**kubectl get all -n mongodb**

```auto
NAME READY STATUS RESTARTS AGE
pod/psmdb-db-rs0-0 1/1 Running 6 (2m49s ago) 21m
pod/psmdb-operator-869b9b99d-fm66v 1/1 Running 0 21m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/psmdb-db-rs0 ClusterIP None <none> 27017/TCP 61m
service/psmdb-db-rs0-0 LoadBalancer 172.20.23.165 k8s-mongodb-psmdbdbr-03fa115926-<redacted>.elb.eu-west-2.amazonaws.com 27017:30564/TCP 61m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/psmdb-operator 1/1 1 1 62m

NAME DESIRED CURRENT READY AGE
replicaset.apps/psmdb-operator-869b9b99d 1 1 1 62m

NAME READY AGE
statefulset.apps/psmdb-db-rs0 1/1 61m

```

Uninstalling and reinstalling mongodb server still throws the same error, but i noticed this in the operator logs.

```auto
2023-09-20T14:58:22.415Z INFO initiating replset {"controller": "psmdb-controller", "object": {"name":"psmdb-db","namespace":"mongodb"}, "namespace": "mongodb", "name": "psmdb-db", "reconcileID": "dda62130-0f5f-4a90-99de-f5ead14b119d", "replset": "rs0", "pod": "psmdb-db-rs0-0"}
2023-09-20T14:58:31.553Z ERROR failed to reconcile cluster {"controller": "psmdb-controller", "object": {"name":"psmdb-db","namespace":"mongodb"}, "namespace": "mongodb", "name": "psmdb-db", "reconcileID": "dda62130-0f5f-4a90-99de-f5ead14b119d", "replset": "rs0", "error": "handleReplsetInit: exec add admin user: command terminated with exit code 1 / Current Mongosh Log ID:\t650b089716c0f0dcef831f23\nConnecting to:\t\tmongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.2\nUsing MongoDB:\t\t6.0.4-3\nUsing Mongosh:\t\t1.6.2\n\nFor mongosh info see: https://docs.mongodb.com/mongodb-shell/\n\n / MongoServerError: command createUser requires authentication\n", "errorVerbose": "exec add admin user: command terminated with exit code 1 / Current Mongosh Log ID:\t650b089716c0f0dcef831f23\nConnecting to:\t\tmongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.2\nUsing MongoDB:\t\t6.0.4-3\nUsing Mongosh:\t\t1.6.2\n\nFor mongosh info see: https://docs.mongodb.com/mongodb-shell/\n\n / MongoServerError: command createUser requires authentication\n\nhandleReplsetInit\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).reconcileCluster\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/mgo.go:99\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).Reconcile\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/psmdb_controller.go:487\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:122\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:323\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1594"}
github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).Reconcile
        /go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/psmdb_controller.go:489
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:122
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:323
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:274
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
        /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.14.4/pkg/internal/controller/controller.go:235

```

---

<div class="post-metadata">

### Author: ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)
#### Post date: [October 2, 2023, 8:12am UTC](https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324/2 "2023-10-02T08:12:22Z")

</div>

@Kay_Khan I would suspect it has smth to do with certificates and the way replica set is configured.

Could you please share your values.yaml? I’m mostly interested in how `clusterServiceDNSMode` is configured. Or you can just share your CR manifest: `kubectl get psmdb <CLUSTER> -o yaml`

---

<div class="post-metadata">

### Author: ![Kay\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/kay_khan/32/17023_2.png) [@Kay\_Khan](https://forums.percona.com/u/Kay_Khan)
#### Post date: [October 2, 2023, 8:34am UTC](https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324/3 "2023-10-02T08:34:44Z")

</div>

@Sergey_Pronin Please see below the entire values.yaml file aswell as the CR manifest. Just FYI ive left the mongodb intance running for 1 week now as i had to move onto something else. However if there are anymore tests you need to be perform and information to provide please let me know as this problem will be blocking me very soon.

**values.yaml**

```auto
# Default values for psmdb-cluster.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

# Platform type: kubernetes, openshift
# platform: kubernetes

# Cluster DNS Suffix
# clusterServiceDNSSuffix: svc.cluster.local
clusterServiceDNSMode: "External"

finalizers:
## Set this if you want that operator deletes the primary pod last
  - delete-psmdb-pods-in-order
## Set this if you want to delete database persistent volumes on cluster deletion
# - delete-psmdb-pvc

nameOverride: ""
fullnameOverride: ""

crVersion: 1.14.0
pause: false
unmanaged: false
allowUnsafeConfigurations: true
# ignoreAnnotations:
# - service.beta.kubernetes.io/aws-load-balancer-backend-protocol
# ignoreLabels:
# - rack
multiCluster:
  enabled: false
  # DNSSuffix: svc.clusterset.local
updateStrategy: SmartUpdate
upgradeOptions:
  versionServiceEndpoint: https://check.percona.com
  apply: disabled
  schedule: "0 2 * * *"
  setFCV: false

image:
  repository: percona/percona-server-mongodb
  tag: 6.0.8-6

imagePullPolicy: Always
# imagePullSecrets: []
# initImage:
# repository: percona/percona-server-mongodb-operator
# tag: 1.14.0
# initContainerSecurityContext: {}
# tls:
# # 90 days in hours
# certValidityDuration: 2160h
secrets: {}
  # If you set users secret here the operator will use existing one or generate random values
  # If not set the operator generates the default secret with name <cluster_name>-secrets
  # users: my-cluster-name-secrets
  # encryptionKey: my-cluster-name-mongodb-encryption-key

pmm:
  enabled: false
  image:
    repository: percona/pmm-client
    tag: 2.35.0
  serverHost: monitoring-service

replsets:
  - name: rs0
    size: 1
    # externalNodes:
    # - host: 34.124.76.90
    # - host: 34.124.76.91
    # port: 27017
    # votes: 0
    # priority: 0
    # - host: 34.124.76.92
    # configuration: |
    # operationProfiling:
    # mode: slowOp
    # systemLog:
    # verbosity: 1
    antiAffinityTopologyKey: "kubernetes.io/hostname"
    # tolerations: []
    # priorityClass: ""
    # annotations: {}
    # labels: {}
    nodeSelector:
      acme/node-type: "ops"
    # livenessProbe:
    # failureThreshold: 4
    # initialDelaySeconds: 60
    # periodSeconds: 30
    # timeoutSeconds: 10
    # startupDelaySeconds: 7200
    # readinessProbe:
    # failureThreshold: 8
    # initialDelaySeconds: 10
    # periodSeconds: 3
    # successThreshold: 1
    # timeoutSeconds: 2
    # runtimeClassName: image-rc
    # storage:
    # engine: wiredTiger
    # wiredTiger:
    # engineConfig:
    # cacheSizeRatio: 0.5
    # directoryForIndexes: false
    # journalCompressor: snappy
    # collectionConfig:
    # blockCompressor: snappy
    # indexConfig:
    # prefixCompression: true
    # inMemory:
    # engineConfig:
    # inMemorySizeRatio: 0.5
    sidecars:
    - image: percona/mongodb_exporter:0.36
      env:
      - name: EXPORTER_USER
        valueFrom:
          secretKeyRef:
            name: psmdb-db-secrets
            key: MONGODB_CLUSTER_MONITOR_USER
      - name: EXPORTER_PASS
        valueFrom:
          secretKeyRef:
            name: psmdb-db-secrets
            key: MONGODB_CLUSTER_MONITOR_PASSWORD
      - name: POD_IP
        valueFrom:
          fieldRef:
            fieldPath: status.podIP
      - name: MONGODB_URI
        value: "mongodb://$(EXPORTER_USER):$(EXPORTER_PASS)@$(POD_IP):27017"
      args: ["--discovering-mode", "--compatible-mode", "--collect-all", "--mongodb.uri=$(MONGODB_URI)"]
      name: metrics
    # volumeMounts:
    # - mountPath: /volume1
    # name: sidecar-volume-claim
    # - mountPath: /secret
    # name: sidecar-secret
    # - mountPath: /configmap
    # name: sidecar-config
    # sidecarVolumes:
    # - name: sidecar-secret
    # secret:
    # secretName: mysecret
    # - name: sidecar-config
    # configMap:
    # name: myconfigmap
    # sidecarPVCs:
    # - apiVersion: v1
    # kind: PersistentVolumeClaim
    # metadata:
    # name: sidecar-volume-claim
    # spec:
    # resources:
    # requests:
    # storage: 1Gi
    # volumeMode: Filesystem
    # accessModes:
    # - ReadWriteOnce
    podDisruptionBudget:
      maxUnavailable: 1
    expose:
      enabled: true
      exposeType: LoadBalancer
      # loadBalancerSourceRanges:
      # - 10.0.0.0/8
      serviceAnnotations:
        service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
          # service.beta.kubernetes.io/aws-load-balancer-security-groups: sg-0beb09a596e969cfe
          # service.beta.kubernetes.io/aws-load-balancer-manage-backend-security-group-rules: "true"
        # service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
      # serviceLabels: 
      # some-label: some-key
    nonvoting:
      enabled: false
      # podSecurityContext: {}
      # containerSecurityContext: {}
      size: 3
      # configuration: |
      # operationProfiling:
      # mode: slowOp
      # systemLog:
      # verbosity: 1
      antiAffinityTopologyKey: "kubernetes.io/hostname"
      # tolerations: []
      # priorityClass: ""
      # annotations: {}
      # labels: {}
      # nodeSelector: {}
      podDisruptionBudget:
        maxUnavailable: 1
      resources:
        limits:
          cpu: "300m"
          memory: "0.5G"
        requests:
          cpu: "300m"
          memory: "0.5G"
      volumeSpec:
        # emptyDir: {}
        # hostPath:
        # path: /data
        pvc:
          # annotations:
          # volume.beta.kubernetes.io/storage-class: example-hostpath
          # labels:
          # rack: rack-22
          # storageClassName: standard
          # accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 3Gi
    arbiter:
      enabled: false
      size: 1
      antiAffinityTopologyKey: "kubernetes.io/hostname"
      # tolerations: []
      # priorityClass: ""
      # annotations: {}
      # labels: {}
      # nodeSelector: {}
    # schedulerName: ""
    # resources:
    # limits:
    # cpu: "300m"
    # memory: "0.5G"
    # requests:
    # cpu: "300m"
    # memory: "0.5G"
    volumeSpec:
      # emptyDir: {}
      # hostPath:
      # path: /data
      pvc:
        # annotations:
        # volume.beta.kubernetes.io/storage-class: example-hostpath
        # labels:
        # rack: rack-22
        storageClassName: mongodb
        # accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 250Gi

sharding:
  enabled: false 

backup:
  enabled: true
  image:
    repository: percona/percona-backup-mongodb
    tag: 2.0.5
  serviceAccountName: percona-server-mongodb-operator
  # annotations:
  # iam.amazonaws.com/role: arn:aws:iam::700849607999:role/acme-test-default-eks-mongodb
  # resources:
  # limits:
  # cpu: "300m"
  # memory: "0.5G"
  # requests:
  # cpu: "300m"
  # memory: "0.5G"
  storages:
    s3-us-east:
      type: s3
      s3:
        bucket: acme-prod-mongodb-backup
        credentialsSecret: prod-aws-mongodb
        region: us-east-2
        prefix: ""
        uploadPartSize: 10485760
        maxUploadParts: 10000
        storageClass: STANDARD
        insecureSkipTLSVerify: false
    # minio:
    # type: s3
    # s3:
    # bucket: MINIO-BACKUP-BUCKET-NAME-HERE
    # region: us-east-1
    # credentialsSecret: my-cluster-name-backup-minio
    # endpointUrl: http://minio.psmdb.svc.cluster.local:9000/minio/
    # prefix: ""
    # azure-blob:
    # type: azure
    # azure:
    # container: CONTAINER-NAME
    # prefix: PREFIX-NAME
    # credentialsSecret: SECRET-NAME
  pitr:
    enabled: false
    # oplogSpanMin: 10
    # compressionType: gzip
    # compressionLevel: 6
  tasks:
   - name: "daily-s3-backup"
     enabled: true
     schedule: "0 1 * * *"
     keep: 3
     type: logical
     storageName: s3-us-east

  # - name: daily-s3-us-west
  # enabled: true
  # schedule: "0 0 * * *"
  # keep: 3
  # storageName: s3-us-west
  # compressionType: gzip
  # - name: weekly-s3-us-west
  # enabled: false
  # schedule: "0 0 * * 0"
  # keep: 5
  # storageName: s3-us-west
  # compressionType: gzip
  # - name: weekly-s3-us-west-physical
  # enabled: false
  # schedule: "0 5 * * 0"
  # keep: 5
  # type: physical
  # storageName: s3-us-west
  # compressionType: gzip
  # compressionLevel: 6

# If you set users here the secret will be constructed by helm with these values
# users:
# MONGODB_BACKUP_USER: backup
# MONGODB_BACKUP_PASSWORD: backup123456
# MONGODB_DATABASE_ADMIN_USER: databaseAdmin
# MONGODB_DATABASE_ADMIN_PASSWORD: databaseAdmin123456
# MONGODB_CLUSTER_ADMIN_USER: clusterAdmin
# MONGODB_CLUSTER_ADMIN_PASSWORD: clusterAdmin123456
# MONGODB_CLUSTER_MONITOR_USER: clusterMonitor
# MONGODB_CLUSTER_MONITOR_PASSWORD: clusterMonitor123456
# MONGODB_USER_ADMIN_USER: userAdmin
# MONGODB_USER_ADMIN_PASSWORD: userAdmin123456
# PMM_SERVER_API_KEY: apikey
# # PMM_SERVER_USER: admin
# # PMM_SERVER_PASSWORD: admin

```

**kubectl get psmdb psmdb-db -o yaml -n mongodb**

```auto
apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDB
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"psmdb.percona.com/v1","kind":"PerconaServerMongoDB"}
    meta.helm.sh/release-name: psmdb-db
    meta.helm.sh/release-namespace: mongodb
  creationTimestamp: "2023-09-25T10:22:17Z"
  finalizers:
  - delete-psmdb-pods-in-order
  generation: 1
  labels:
    app.kubernetes.io/instance: psmdb-db
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: psmdb-db
    app.kubernetes.io/version: 1.14.0
    helm.sh/chart: psmdb-db-1.14.4
  name: psmdb-db
  namespace: mongodb
  resourceVersion: "5398779"
  uid: eaa25b03-4842-4001-b64b-c528510bd5ff
spec:
  allowUnsafeConfigurations: true
  backup:
    enabled: false
    image: percona/percona-backup-mongodb:2.0.5
    pitr:
      enabled: false
    serviceAccountName: percona-server-mongodb-operator
    storages:
      s3-us-east:
        s3:
          bucket: acme-prod-mongodb-backup
          credentialsSecret: prod-aws-mongodb
          insecureSkipTLSVerify: false
          maxUploadParts: 10000
          prefix: ""
          region: us-east-2
          storageClass: STANDARD
          uploadPartSize: 10485760
        type: s3
    tasks:
    - enabled: false
      keep: 3
      name: daily-s3-backup
      schedule: 0 1 * * *
      storageName: s3-us-east
      type: logical
  clusterServiceDNSMode: External
  crVersion: 1.14.0
  image: percona/percona-server-mongodb:6.0.4-3
  imagePullPolicy: Always
  multiCluster:
    enabled: false
  pause: false
  pmm:
    enabled: false
    image: percona/pmm-client:2.35.0
    serverHost: monitoring-service
  replsets:
  - arbiter:
      enabled: false
      size: 1
    expose:
      enabled: true
      exposeType: LoadBalancer
      serviceAnnotations:
        service.beta.kubernetes.io/aws-load-balancer-scheme: internal
    name: rs0
    nodeSelector:
      acme/node-type: ops
    nonvoting:
      enabled: false
      podDisruptionBudget:
        maxUnavailable: 1
      resources:
        limits:
          cpu: 300m
          memory: 0.5G
        requests:
          cpu: 300m
          memory: 0.5G
      size: 3
      volumeSpec:
        persistentVolumeClaim:
          resources:
            requests:
              storage: 3Gi
    podDisruptionBudget:
      maxUnavailable: 1
    sidecars:
    - args:
      - --discovering-mode
      - --compatible-mode
      - --collect-all
      - --mongodb.uri=$(MONGODB_URI)
      env:
      - name: EXPORTER_USER
        valueFrom:
          secretKeyRef:
            key: MONGODB_CLUSTER_MONITOR_USER
            name: psmdb-db-secrets
      - name: EXPORTER_PASS
        valueFrom:
          secretKeyRef:
            key: MONGODB_CLUSTER_MONITOR_PASSWORD
            name: psmdb-db-secrets
      - name: POD_IP
        valueFrom:
          fieldRef:
            fieldPath: status.podIP
      - name: MONGODB_URI
        value: mongodb://$(EXPORTER_USER):$(EXPORTER_PASS)@$(POD_IP):27017
      image: percona/mongodb_exporter:0.36
      name: metrics
    size: 1
    volumeSpec:
      persistentVolumeClaim:
        resources:
          requests:
            storage: 250Gi
        storageClassName: mongodb
  secrets:
    users: psmdb-db-secrets
  sharding:
    configsvrReplSet:
      affinity:
        antiAffinityTopologyKey: kubernetes.io/hostname
      expose:
        enabled: false
        exposeType: ClusterIP
      podDisruptionBudget:
        maxUnavailable: 1
      resources:
        limits:
          cpu: 300m
          memory: 0.5G
        requests:
          cpu: 300m
          memory: 0.5G
      size: 3
      volumeSpec:
        persistentVolumeClaim:
          resources:
            requests:
              storage: 3Gi
    enabled: false
    mongos:
      affinity:
        antiAffinityTopologyKey: kubernetes.io/hostname
      expose:
        exposeType: ClusterIP
      podDisruptionBudget:
        maxUnavailable: 1
      resources:
        limits:
          cpu: 300m
          memory: 0.5G
        requests:
          cpu: 300m
          memory: 0.5G
      size: 2
  unmanaged: false
  updateStrategy: SmartUpdate
  upgradeOptions:
    apply: disabled
    schedule: 0 2 * * *
    setFCV: false
    versionServiceEndpoint: https://check.percona.com
status:
  conditions:
  - lastTransitionTime: "2023-10-02T08:27:46Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:27:53Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:28:57Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:28:59Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:29:40Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:29:42Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:29:53Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:30:02Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:30:16Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:30:24Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:31:32Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:31:35Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:31:46Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:31:59Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:32:27Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:32:34Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:32:46Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:32:48Z"
    status: "True"
    type: ready
  - lastTransitionTime: "2023-10-02T08:33:15Z"
    status: "True"
    type: initializing
  - lastTransitionTime: "2023-10-02T08:33:17Z"
    status: "True"
    type: ready
  host: k8s-mongodb-psmdbdbr-9ca3fb6f1f-<redacted>.elb.eu-west-2.amazonaws.com:27017
  mongoImage: percona/percona-server-mongodb:6.0.4-3
  mongoVersion: 6.0.4-3
  observedGeneration: 1
  ready: 1
  replsets:
    rs0:
      initialized: true
      ready: 1
      size: 1
      status: ready
  size: 1
  state: ready

```

---

<div class="post-metadata">

### Author: ![Kay\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/kay_khan/32/17023_2.png) [@Kay\_Khan](https://forums.percona.com/u/Kay_Khan)
#### Post date: [October 4, 2023, 3:01pm UTC](https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324/4 "2023-10-04T15:01:04Z")

</div>

Hi,

I remember you mentioned something about certificates, we saw this when attempting to provision a new instance on our production cluster. Any help is appreciated, we are not sure what to do

```auto
Liveness probe failed: command "/opt/percona/mongodb-healthcheck k8s liveness --ssl --sslInsecure --sslCAFile /etc/mongodb-ssl/ca.crt --sslPEMKeyFile /tmp/tls.pem --startupDelaySeconds 7200" timed out

```

 ![image](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/5/57bea7a5abbd7305d08446f706f6bc85dd60fb93.png)

---

<div class="post-metadata">

### Author: ![Kay\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/kay_khan/32/17023_2.png) [@Kay\_Khan](https://forums.percona.com/u/Kay_Khan)
#### Post date: [October 4, 2023, 3:10pm UTC](https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324/5 "2023-10-04T15:10:29Z")

</div>

Also note we do have cert manager installed

**helm ls -n cert-manager**

```auto
cert-manager cert-manager 1 2023-05-03 15:07:42.725175923 +0100 BST deployed cert-manager-v1.11.1 v1.11.1

```

And we’ve have a mongodb instance running for a few months now which is working fine. But this new one we deployed is failing `psmdb-db-internal `

**helm ls -n mongodb**

psmdb-db - Original  
psmdb-db-internal - New one which is failing

```auto
psmdb-db mongodb 2 2023-08-03 09:31:58.57105479 +0100 BST deployed psmdb-db-1.14.3 1.14.0
psmdb-db-internal mongodb 2 2023-10-04 15:58:00.373192349 +0100 BST deployed psmdb-db-1.14.4 1.14.0
psmdb-operator mongodb 1 2023-05-06 10:35:46.776038271 +0100 BST deployed psmdb-operator-1.14.2 1.14.0

```

---

<div class="post-metadata">

### Author: ![Kay\_Khan](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/kay_khan/32/17023_2.png) [@Kay\_Khan](https://forums.percona.com/u/Kay_Khan)
#### Post date: [October 5, 2023, 8:00am UTC](https://forums.percona.com/t/could-not-find-host-matching-read-preference-mode-nearest-for-set-rs0/25324/6 "2023-10-05T08:00:59Z")

</div>

I found the the issue.

My original instance was externally facing and i had to use `clusterServiceDNSMode: "External"` to get that to work. I had forgotten about that change, so when i wanted to create a new instance which was internal by default. I had to remember to set `clusterServiceDNSMode` back to default `Internal`

thanks for pointing that out.