# Operator cannot delete backups on s3 with custom TLS on 1.14.0 version

**URL:** <https://forums.percona.com/t/operator-cannot-delete-backups-on-s3-with-custom-tls-on-1-14-0-version/29532>\
**Category:** Percona Operator for MySQL\
**Created:** [April 3, 2024, 2:03pm UTC](https://forums.percona.com/t/operator-cannot-delete-backups-on-s3-with-custom-tls-on-1-14-0-version/29532 "2024-04-03T14:03:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![yotles](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/yotles/32/1293_2.png) [@yotles](https://forums.percona.com/u/yotles)\
**Post date:** [April 3, 2024, 2:03pm UTC](https://forums.percona.com/t/operator-cannot-delete-backups-on-s3-with-custom-tls-on-1-14-0-version/29532/1 "2024-04-03T14:03:55Z")

</div>

Hi!

After upgrade to pxc-operator 1.13.3 → 1.14.0 and pxc-db 1.13.2 → 1.14.1 faced with issue `INFO	failed to delete backup	"error": "new s3 storage: failed to check if bucket exists: Head \"https://xxx.xxx": tls: failed to verify certificate: x509: certificate signed by unknown authority"}`

Also I’ve found that such issue resolved with 1.13.0 and there present ticket [[K8SPXC-1205] - Percona JIRA](https://perconadev.atlassian.net/browse/K8SPXC-1205)

So in 1.14.0 this bug returned?

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![lalit.choudhary](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/lalit.choudhary/32/51_2.png) [@lalit.choudhary](https://forums.percona.com/u/lalit.choudhary)\
**Post date:** [April 4, 2024, 5:27pm UTC](https://forums.percona.com/t/operator-cannot-delete-backups-on-s3-with-custom-tls-on-1-14-0-version/29532/2 "2024-04-04T17:27:54Z")

</div>

Hello @yotles

This issue indeed fixed in 1.14.0 by introducing new option _`verifyTLS`_  
**Solution:**  
\*Add `verifyTLS`[backup.storages.storage-name.verifyTLS](https://docs.percona.com/percona-operator-for-mysql/pxc/operator.html?h=#backup-storages-verifytls)

> Enable or disable verification of the storage server TLS certificate. Disabling it may be useful e.g. to skip TLS verification for private S3-compatible storage with a self-issued certificate

[K8SPXC-1205](https://jira.percona.com/browse/K8SPXC-1205?_gl=1*1crd0x0*_gcl_aw*R0NMLjE3MDY1MTQ0MjEuQ2p3S0NBaUFrOWl0QmhBU0Vpd0ExbXlfNjJfOW1VS0dVX3pUSDZFc1dCSU1MSmJGOFNBaC03b1dPdUxmZWdFQ1Z1b2FmcHREUV9jOGRSb0NiYVVRQXZEX0J3RQ..*_gcl_au*MTkzMDYwMTM4OC4xNzA1MzIwNzMz): Fix a bug which made the Operator to ignore the `verifyTLS` option for backups deletion caused by the `delete-s3-backup` finalizer

Test file example: [percona-xtradb-cluster-operator/e2e-tests/demand-backup/conf/demand-backup.yml at 9469092046febf70d346a12f3f223cac076142d1 · percona/percona-xtradb-cluster-operator · GitHub](https://github.com/percona/percona-xtradb-cluster-operator/blob/9469092046febf70d346a12f3f223cac076142d1/e2e-tests/demand-backup/conf/demand-backup.yml)

Are you using `verifyTLS` option ?

---

<div class="post-metadata">

**Author:** ![yotles](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/yotles/32/1293_2.png) [@yotles](https://forums.percona.com/u/yotles)\
**Post date:** [April 6, 2024, 1:55pm UTC](https://forums.percona.com/t/operator-cannot-delete-backups-on-s3-with-custom-tls-on-1-14-0-version/29532/3 "2024-04-06T13:55:10Z")

</div>

Hi @lalit.choudhary  
Yep, it has been added from the beginning of using operator, so on 1.13.0 everything ok with `verifyTLS: false`, but after upgrade it changed.

---

<div class="post-metadata">

**Author:** ![lalit.choudhary](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/lalit.choudhary/32/51_2.png) [@lalit.choudhary](https://forums.percona.com/u/lalit.choudhary)\
**Post date:** [April 8, 2024, 4:52am UTC](https://forums.percona.com/t/operator-cannot-delete-backups-on-s3-with-custom-tls-on-1-14-0-version/29532/4 "2024-04-08T04:52:40Z")

</div>

Hello @yotles

Thank you for the update.  
Please report this issue as bug for pxc-operator 1.14.0 version adding [[K8SPXC-1205]](https://perconadev.atlassian.net/browse/K8SPXC-1205) as a reference in report.

[https://perconadev.atlassian.net](https://perconadev.atlassian.net)
