Backup error: Process completed with error: /usr/bin/run_backup.sh. What is it?

MySQL & MariaDB Percona XtraDB Cluster 8.x
1

I got the following error log in backup pod after backup to an s3 storage run:

Process completed with error: /usr/bin/run_backup.sh: 1 (Operation not permitted)[INFO] garbd was finished

Iam running latest version of XTraDB installed using Helm char.t.

It looks like the backup is created, the backup files are in place in s3 storage.
What is wrong?
Where is this /usr/bin/run_backup.sh placed ?

1 Like
2

Hi @Michael_Chudinov,

You can see the scripts here: percona-docker/percona-xtradb-cluster-8.0-backup at main · percona/percona-docker · GitHub

It’s hard to tell what went wrong with the output above. Could you please share some details about your environment?

1 Like
3

@Michael_Chudinov just to be sure, please double check your s3 credentials

1 Like
4

S3 credentials are correct, because the backup is created.
But it looks like something is wrong with permissions. backup routine can not find the files that were created. :face_with_raised_eyebrow:
Here is the log

 INFO: [SST script] mc: <ERROR> Unable to stat `dest/backup/xtradb-pxc-db-2021-11-04-16:50:00-full.md5`. Target `dest/backup/xtradb-pxc-db-2021-11-04-16:50:00-full.md5` not found

And since the backup job failing (even the backup files are there), the Kubernetes job retries to run backup every approximately minute.

1 Like
5

@Ege_Gunes hi, I have got the same issue.

I use 1.11.0 image with Percona XtraDB Cluster Operator:

the next configuration of PXC’s CRD

    s3:
      bucket: perc-perc-bucket
      credentialsSecret: percona-test-backup-cred-s3
      region: eu-central-1

doesn’t work. It fails with above error. I checked access logs for s3 bucket and found:

b77ec9bd777a28752e142bbccb15b6710a162641885904a8bc2*** perc-perc-bucket [02/Sep/2022:08:58:05 +0000] 18.157.98.158 - 1HKAS*** REST.GET.BUCKET - “GET /?delimiter=%2F&encoding-type=url&fetch-owner=true&list-type=2&prefix=percona-test%2Fc1-2022-09-02-08%3A57%3A35-full.sst_info HTTP/1.1” 301 PermanentRedirect 482 - 2 - “-” “MinIO (linux; amd64) minio-go/v7.0.24 mc/RELEASE.2022-05-04T06-07-55Z” - AIEkVthcm7ROrNSU1GPBIONRYszwqs3D0c4QH472HDakdEEGJ6*** = - ECDHE-RSA-AES128-GCM-SHA256 - perc-perc-bucket.s3.dualstack.us-east-1.amazonaws.com TLSv1.2 -

wrong region is used by script - us-east-1.amazonaws.com

It has been fixed after I wrote config like this:

    s3:
      bucket: perc-perc-bucket
      credentialsSecret: percona-test-backup-cred-s3
      endpointUrl: https://s3.eu-central-1.amazonaws.com
      region: eu-central-1

I hope this information will be helpful for developers or users. Thanks for your operator!

1 Like
6

it’s internal behavior of minio client:

1 Like