I’m testing the XtraDB Operator v.1.17 on a AKS cluster (version 1.33.2). The cluster is running well, but I’m unable to make the backups and PITR to work properly.
I deployed MinIO in my cluster to use as S3 compatible storage for backups. I’m able to reach MinIO endpoint from another pod, and I tested my access key and secret key.
Unfortunately, backup jobs and pitr are both crashing. I’ve attached the pods’ logs
Here is the backup section from my cluster’s yaml file:
Hello, @eprevost ! Thank you for your request.
Could you please recheck your azure secret? It should look like this:
apiVersion: v1
kind: Secret
metadata:
name: azure-secret
type: Opaque
data:
AZURE_STORAGE_ACCOUNT_NAME:
AZURE_STORAGE_ACCOUNT_KEY:
Without any newlines or spaces
Could you please also try to verify that the base64-encoded values decode to clean strings with no extra characters like line breaks, spaces, or hidden symbols.
Yes, that is exactly the template I used for the secret. The strings are base64 encoded. I decoded them to check if they have any extra character, it looks good. I followed instructions in the documentation to encode the values: echo “the string” | base64 –wrap=0
the only space in the yaml is between the “:” and the base64 value
Oh. I used different tools to check the encoded values. echo “the string” | base64 –wrap=0 adds a new line character at the end of the string. echo -n “the string” | base64 –wrap=0 should be used instead.
I doubled-check the documentation.. It mentions echo -n. So.. my bad, I read too quickly.
PITR works, but full backups still fails. I modified the configuration to add –verbose parameter to xbcloud. It looks like it still have issues with authentication with Azure Blob storage:
If you try to run fixed keys for minio, does it work? I can suggest to add another key for azure storage (if it is possible) and test with it. Sometimes we met the problem with keys, if they have special characters.
I will recheck why we return successful backup if it is actually failed.