Operator cannot delete backups on s3 with custom TLS on 1.14.0 version

MySQL & MariaDB Percona Operator for MySQL
1

Hi!

After upgrade to pxc-operator 1.13.3 → 1.14.0 and pxc-db 1.13.2 → 1.14.1 faced with issue INFO failed to delete backup "error": "new s3 storage: failed to check if bucket exists: Head \"https://xxx.xxx": tls: failed to verify certificate: x509: certificate signed by unknown authority"}

Also I’ve found that such issue resolved with 1.13.0 and there present ticket [K8SPXC-1205] - Percona JIRA

So in 1.14.0 this bug returned?

Thanks in advance!

2

Hello @yotles

This issue indeed fixed in 1.14.0 by introducing new option verifyTLS
Solution:
*Add verifyTLS backup.storages.storage-name.verifyTLS

Enable or disable verification of the storage server TLS certificate. Disabling it may be useful e.g. to skip TLS verification for private S3-compatible storage with a self-issued certificate

K8SPXC-1205: Fix a bug which made the Operator to ignore the verifyTLS option for backups deletion caused by the delete-s3-backup finalizer

Test file example: percona-xtradb-cluster-operator/e2e-tests/demand-backup/conf/demand-backup.yml at 9469092046febf70d346a12f3f223cac076142d1 · percona/percona-xtradb-cluster-operator · GitHub

Are you using verifyTLS option ?

3

Hi @lalit.choudhary
Yep, it has been added from the beginning of using operator, so on 1.13.0 everything ok with verifyTLS: false, but after upgrade it changed.

4

Hello @yotles

Thank you for the update.
Please report this issue as bug for pxc-operator 1.14.0 version adding [K8SPXC-1205] as a reference in report.

https://perconadev.atlassian.net