# Error in backups after upgrading mongodb version

**URL:** https://forums.percona.com/t/error-in-backups-after-upgrading-mongodb-version/30663
**Category:** MongoDB
**Tags:** percona, mongodb, closed-no-reply
**Created:** [May 28, 2024, 9:47pm UTC](https://forums.percona.com/t/error-in-backups-after-upgrading-mongodb-version/30663 "2024-05-28T21:47:36Z")
**Posts on this page:** 1
**Page:** 1

<div class="post-metadata">

### Author: ![Vatsal\_Sharma](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vatsal_sharma/32/14840_2.png) [@Vatsal\_Sharma](https://forums.percona.com/u/Vatsal_Sharma)
#### Post date: [May 28, 2024, 9:47pm UTC](https://forums.percona.com/t/error-in-backups-after-upgrading-mongodb-version/30663/1 "2024-05-28T21:47:36Z")

</div>

I have deployed percona mongodb server on my eks cluster using its operator, recently I upgraed the cr version to 1.16.0 and the mongodb version to 7.0.8-5, I have configured scheduled backups in my manifest but still when the back is triggered it gives error which is →

```auto
no available agent(s) on replsets: rs0, cfg

```

Also when checked the operator logs I found this →

```auto
2024-05-28T21:13:03.657Z	INFO	Creating or updating backup job	{"controller": "psmdb-controller", "object": {"name":"perconamongodb-psmdb","namespace":"perconamongodb"}, "namespace": "perconamongodb", "name": "perconamongodb-psmdb", "reconcileID": "d535b9fa-c8f3-46b3-8e4c-3d2bd88bf020", "name": "mongobackup_night", "namespace": "perconamongodb", "schedule": "0 22 * * *"}
2024-05-28T21:13:06.084Z	ERROR	Reconciler error	{"controller": "psmdb-controller", "object": {"name":"perconamongodb-psmdb","namespace":"perconamongodb"}, "namespace": "perconamongodb", "name": "perconamongodb-psmdb", "reconcileID": "d535b9fa-c8f3-46b3-8e4c-3d2bd88bf020", "error": "create pbm object: create PBM connection to perconamongodb-psmdb-rs0-0.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local:27017,perconamongodb-psmdb-rs0-1.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local:27017,perconamongodb-psmdb-rs0-2.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local:27017: create mongo connection: ping: connection() error occurred during connection handshake: dial tcp: lookup perconamongodb-psmdb-rs0-0.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local on 172.20.0.10:53: no such host", "errorVerbose": "connection() error occurred during connection handshake: dial tcp: lookup perconamongodb-psmdb-rs0-0.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local on 172.20.0.10:53: no such host\nping\ncreate mongo connection\ncreate PBM connection to perconamongodb-psmdb-rs0-0.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local:27017,perconamongodb-psmdb-rs0-1.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local:27017,perconamongodb-psmdb-rs0-2.perconamongodb-psmdb-rs0.perconamongodb.svc.cluster.local:27017\ngithub.com/percona/percona-server-mongodb-operator/pkg/psmdb/backup.NewPBM\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/psmdb/backup/pbm.go:173\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).updatePITR\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/backup.go:376\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).Reconcile\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/psmdb_controller.go:434\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:222\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1695\ncreate pbm object\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).updatePITR\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/backup.go:378\ngithub.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb.(*ReconcilePerconaServerMongoDB).Reconcile\n\t/go/src/github.com/percona/percona-server-mongodb-operator/pkg/controller/perconaservermongodb/psmdb_controller.go:434\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:311\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:261\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.1/pkg/internal/controller/controller.go:222\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1695"}
2024-05-28T21:22:01.557Z	INFO	Creating or updating backup job	{"controller": "psmdb-controller", "object": {"name":"perconamongodb-psmdb","namespace":"perconamongodb"}, "namespace": "perconamongodb", "name": "perconamongodb-psmdb", "reconcileID": "895834c1-0945-4849-95af-9a8ed552968c", "name": "mongobackup_night", "namespace": "perconamongodb", "schedule": "0 0 * * *"}
2024-05-28T21:22:57.667Z	INFO	Starting backup	{"controller": "psmdbbackup-controller", "object": {"name":"ondemand-local-backup","namespace":"perconamongodb"}, "namespace": "perconamongodb", "name": "ondemand-local-backup", "reconcileID": "558d1cbf-361a-4d7a-9ac8-b5dbfa7c1376", "backup": "ondemand-local-backup", "storage": "backupstorage"}
2024-05-28T21:22:57.667Z	INFO	Setting PBM config	{"controller": "psmdbbackup-controller", "object": {"name":"ondemand-local-backup","namespace":"perconamongodb"}, "namespace": "perconamongodb", "name": "ondemand-local-backup", "reconcileID": "558d1cbf-361a-4d7a-9ac8-b5dbfa7c1376", "backup": "perconamongodb-psmdb"}
2024-05-28T21:23:08.672Z	INFO	Sending backup command	{"controller": "psmdbbackup-controller", "object": {"name":"ondemand-local-backup","namespace":"perconamongodb"}, "namespace": "perconamongodb", "name": "ondemand-local-backup", "reconcileID": "558d1cbf-361a-4d7a-9ac8-b5dbfa7c1376", "backupCmd": "backup [name: 2024-05-28T21:23:08Z, compression: gzip (level: default)] <ts: 0>"}

```

and here is my configuration for backup section →

```auto
backup:
  enabled: true
  image:
    repository: percona/percona-backup-mongodb
    tag: 2.4.1
  serviceAccountName: percona-server-mongodb-operator
  annotations:
     iam.amazonaws.com/role: arn:aws:iam::645193536862:role/prod-percona-s3-role
  # podSecurityContext: {}
  # containerSecurityContext: {}
  # resources:
  # limits:
  # cpu: "300m"
  # memory: "0.5G"
  # requests:
  # cpu: "300m"
  # memory: "0.5G"
  storages:
     backupstorage:
       type: s3
       s3:
         bucket: "ds-mongodb-backup"
         credentialsSecret: percona-backup-secret
    # serverSideEncryption:
    # kmsKeyID: 1234abcd-12ab-34cd-56ef-1234567890ab
    # sseAlgorithm: aws:kms
    # sseCustomerAlgorithm: AES256
    # sseCustomerKey: Y3VzdG9tZXIta2V5
    # retryer:
    # numMaxRetries: 3
    # minRetryDelay: 30ms
    # maxRetryDelay: 5m
         region: us-east-1
         prefix: "perconabackup"
# uploadPartSize: 10485760
# maxUploadParts: 10000
         storageClass: STANDARD
         insecureSkipTLSVerify: true
    # minio:
    # type: s3
    # s3:
    # bucket: MINIO-BACKUP-BUCKET-NAME-HERE
    # region: us-east-1
    # credentialsSecret: my-cluster-name-backup-minio
    # endpointUrl: http://minio.psmdb.svc.cluster.local:9000/minio/
    # prefix: ""
    # azure-blob:
    # type: azure
    # azure:
    # container: CONTAINER-NAME
    # prefix: PREFIX-NAME
    # endpointUrl: https://accountName.blob.core.windows.net
    # credentialsSecret: SECRET-NAME
  pitr:
    enabled: false
    oplogOnly: false
    # oplogSpanMin: 10
    # compressionType: gzip
    # compressionLevel: 6
  # configuration:
  # backupOptions:
  # priority:
  # "localhost:28019": 2.5
  # "localhost:27018": 2.5
  # timeouts:
  # startingStatus: 33
  # oplogSpanMin: 10
  # restoreOptions:
  # batchSize: 500
  # numInsertionWorkers: 10
  # numDownloadWorkers: 4
  # maxDownloadBufferMb: 0
  # downloadChunkMb: 32
  # mongodLocation: /usr/bin/mongo
  # mongodLocationMap:
  # "node01:2017": /usr/bin/mongo
  # "node03:27017": /usr/bin/mongo
  tasks:
   - name: mongobackup_night
     enabled: true
     schedule: "0 0 * * *"
     keep: 5
     storageName: backupstorage
     compressionType: gzip
     type: logical
   - name: mongobackup_day
     enabled: true
     schedule: "0 12 * * *"
     keep: 5
     storageName: backupstorage
     compressionType: gzip
     type: logical

```

Earlier version of it was working perfectly fine with same configs, the only change while updating I did was this here →

```auto
unsafeFlags:
  tls: false
  replsetSize: false
  mongosSize: false
  terminationGracePeriod: false
  backupIfUnhealthy: true

```

I set the parameter backupIfUnhealthy as true.

Does this has any effect over it or am I doing something wrong.
