Percona Operator XtraDB Backup Cron to Minio Fails to add credentials

[Zach]

Hello all,

I’m trying to get the operator’s backup function to work. But it appears to be something in the container. I’ve inspected the backup.sh inside the backup container and it appears that the adding of credentials is not working. I have tested it from other machines and containers with the same credentials and my account settings do work.

Any advice or assistance would be appreciated.

Zach

[spronin]

Hello @Zach ,

the error is returned by mc (Minio Client). It says Access Denied, which usually means that credentials are wrong.

From restore-backup.sh I can see that we execute the following command:

mc -C /tmp/mc config host add dest “${ENDPOINT:-<a href=“https://s3.amazonaws.com}””>https://s3.amazonaws.com}" “$ACCESS_KEY_ID” “$SECRET_ACCESS_KEY”

And in hereI see that ACCESS_KEY_ID and key are defined based on what you have in the secrets:

accessKey := corev1.EnvVar{
		Name: "ACCESS_KEY_ID",
		ValueFrom: &corev1.EnvVarSource{
			SecretKeyRef: app.SecretKeySelector(s3.CredentialsSecret, "AWS_ACCESS_KEY_ID"),
		},
	}

Do I understand correctly that if you execute mc command manually with correct keys - it works fine, but it does not work in the operator?

If so, please double check if the keys are set correctly in your secrets file (like backup-s3.yaml).

[Zach]

I’ve discovered the issue which if you are using minio you need to specify the API on the minio client command “–api S3v4” for the add credentials to work. Could this be a fix on the backup script?

I tested this manually from inside the container and it works only when applying the API parameter

Zach

[Zach]

adding this to the startup command for the backup container allows the backup to happen

sed ‘s/"$SECRET_ACCESS_KEY/"$SECRET_ACCESS_KEY"\ --api\ S3v4/’ /usr/bin/backup.sh | bash

[spronin]

Hello @Zach,

thank you for submitting details. Was it you or someone from your team who created this ticket:https://jira.percona.com/browse/K8SPXC-570 ?

As you see in the comments there we have tested the latest minio client and it works. We will release PXC Operator 1.7.0 in January with the latest client. Please stay tuned.

[Zach]

That is not me, but that is legit the issue Thank you!