# ERROR with storage: storage check failed with: get S3 object header: RequestError: send request failed **URL:** \ **Category:** Percona Operator for MongoDB\ **Tags:** percona, mongodb\ **Created:** [May 12, 2021, 12:46pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433 "2021-05-12T12:46:37Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![SpoorthiPalakshaiah](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@SpoorthiPalakshaiah](https://forums.percona.com/u/SpoorthiPalakshaiah)\ **Post date:** [May 12, 2021, 12:46pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/1 "2021-05-12T12:46:37Z")
I’m using percona mongodb kubernetes operator [https://github.com/percona/percona-server-mongodb-operator/blob/main/deploy/cr.yaml](https://github.com/percona/percona-server-mongodb-operator/blob/main/deploy/cr.yaml) I’m trying to store backup in minio; But when I do so the backup is not getting stored When I check for status using pbm status --mongodb-uri=‘mongodb://\:\@localhost:27017/?replicaSet=rs0’ I get this error \> - rs0/mongo-db-rs0-1.mongo-db-rs0.mongodb.svc.cluster.local:27017: pbm-agent v1.4.1 FAILED status: > ``` > > ERROR with storage: storage check failed with: get S3 object header: RequestError: send request failed > caused by: Head "https://..svc.cluster.local:9000//.pbm.init": http: server gave HTTP response to HTTPS client > > ``` I tried to disable TLS as mentioned in [docs]([Transport encryption (TLS/SSL) - Percona Operator for MongoDB](https://www.percona.com/doc/kubernetes-operator-for-psmongodb/TLS.html#:~:text=Run%20Percona%20Server%20for%20MongoDB%20without%20TLS,-Omitting%20TLS%20is&text=To%20disable%20TLS%20protocol%20(e.g.,are%20no%20certificate%20secrets%20available.)) But the shard pods will not come up; Please suggest the way such that backup with s3 minio works; ---
**Author:** ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)\ **Post date:** [May 12, 2021, 1:24pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/2 "2021-05-12T13:24:47Z")
Hello @SpoorthiPalakshaiah , are you taking the backup through psmdb-backup Custom Resource? If so, please share the backup.storages section and backup.yaml config. If not - please share the steps to reproduce. Judging by the error I would say that your minio server is running as http one, but you are trying to refer to it as https. In Operator you can try to resolve this by setting endpointUrl correctly. ---
**Author:** ![SpoorthiPalakshaiah](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@SpoorthiPalakshaiah](https://forums.percona.com/u/SpoorthiPalakshaiah)\ **Post date:** [May 12, 2021, 1:51pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/3 "2021-05-12T13:51:19Z")
Hello @Sergey_Pronin, Here is the backup file and storage details backup.yaml > apiVersion: [psmdb.percona.com/v1](http://psmdb.percona.com/v1) > kind: PerconaServerMongoDBBackup > metadata: > name: mongodb-test-backup > spec: > psmdbCluster: mongo-db > storageName: mongo-backup Storage is included in cr.yaml; > storages: > mongo-backup: > type: s3 > s3: > bucket: mongodb-bkp > region: us-east-1 > credentialsSecret: mongodb-cluster-backup-secret > endpointUrl: minio-svc.minio-namespace.svc.cluster.local:9000 When I try to give endpoint as [http://minio-svc.minio-namespace.svc.cluster.local:9000](http://minio-svc.minio-namespace.svc.cluster.local:9000); It gives same error ---
**Author:** ![Andrew\_Pogrebnoi](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/andrew_pogrebnoi/32/77_2.png) [@Andrew\_Pogrebnoi](https://forums.percona.com/u/Andrew_Pogrebnoi)\ **Post date:** [May 12, 2021, 2:40pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/4 "2021-05-12T14:40:00Z")
@SpoorthiPalakshaiah try to set `endpointUrl` to “[http://minio-svc.minio-namespace.svc.cluster.local:9000](http://minio-svc.minio-namespace.svc.cluster.local:9000)”. Without schema set it tries to connect via https by default. ---
**Author:** ![Andrew\_Pogrebnoi](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/andrew_pogrebnoi/32/77_2.png) [@Andrew\_Pogrebnoi](https://forums.percona.com/u/Andrew_Pogrebnoi)\ **Post date:** [May 12, 2021, 2:40pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/5 "2021-05-12T14:40:56Z")
Oh, you did this already… ---
**Author:** ![Andrew\_Pogrebnoi](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/andrew_pogrebnoi/32/77_2.png) [@Andrew\_Pogrebnoi](https://forums.percona.com/u/Andrew_Pogrebnoi)\ **Post date:** [May 12, 2021, 2:44pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/6 "2021-05-12T14:44:54Z")
@SpoorthiPalakshaiah After applying [http://minio-svc.minio-namespace.svc.cluster.local:9000](http://minio-svc.minio-namespace.svc.cluster.local:9000) what does `pbm config` shows? Has the new config applied? ---
**Author:** ![SpoorthiPalakshaiah](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@SpoorthiPalakshaiah](https://forums.percona.com/u/SpoorthiPalakshaiah)\ **Post date:** [May 12, 2021, 2:48pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/7 "2021-05-12T14:48:41Z")
@Andrew_Pogrebnoi In pbm config the endpoint shows with [http://minio-svc.minio-namespace.svc.cluster.local:9000](http://minio-svc.minio-namespace.svc.cluster.local:9000/) But in pbm status it shows > 2021-05-11T08:15:16Z 0.00B [ERROR: mongodump: write data: upload to S3: RequestError: send request failed > caused by: Put “[https://minio-svc.minio-ns.svc.cluster.local:9000/mongodb-bkp/2021-05-11T08%3A15%3A16Z\_cfg.dump.gz](https://minio-svc.minio-ns.svc.cluster.local:9000/mongodb-bkp/2021-05-11T08%3A15%3A16Z_cfg.dump.gz)”: http: server gave HTTP response to HTTPS client.] [2021-05-11T08:16:01] > 2021-05-11T00:00:12Z 0.00B [ERROR: some of pbm-agents were lost during the backup] [2021-05-11T00:01:07] ---
**Author:** ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)\ **Post date:** [May 12, 2021, 7:04pm UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/8 "2021-05-12T19:04:16Z")
Even you change the endpointUrl in cr.yaml, it will be updated in PBM (backup-agent) container only after you try to take another backup or perform the restoration. Could you please set endpointUrl in cr.yaml with http and try to take the backup through psmdb-backup CR? ---
**Author:** ![SpoorthiPalakshaiah](https://avatars.discourse-cdn.com/v4/letter/s/8dc957/32.png) [@SpoorthiPalakshaiah](https://forums.percona.com/u/SpoorthiPalakshaiah)\ **Post date:** [May 13, 2021, 5:13am UTC](https://forums.percona.com/t/error-with-storage-storage-check-failed-with-get-s3-object-header-requesterror-send-request-failed/10433/9 "2021-05-13T05:13:15Z")
@Sergey_Pronin When I take another backup only the .pbm.init file will be stored inside minio bucket; And the pbm status is shown as > # Backups: > > S3 us-east-1 http://..svc.cluster.local:9000/mongodb-bkp > Snapshots: > 2021-05-13T05:06:52Z 0.00B [ERROR: check cluster for backup started: convergeClusterWithTimeout: lost shard cfg, last beat ts: 1620882434] [2021-05-13T05:07:46] > 2021-05-13T05:01:01Z 0.00B [ERROR: some of pbm-agents were lost during the backup] [2021-05-13T05:01:57]