Backup Not starting

MySQL & MariaDB Percona Operator for MySQL
1

Hello, i already get my cluster runing but backup pods are returning errors when starting only when i use S3, if i use permanent storage it seems to work.

NAME READY STATUS RESTARTS AGE
68e50-daily-backup-1628202300-ktxcj 0/1 Completed 0 3m50s
cluster1-proxysql-0 3/3 Running 0 23m
cluster1-proxysql-1 3/3 Running 0 22m
cluster1-proxysql-2 3/3 Running 0 22m
cluster1-pxc-0 3/3 Running 0 23m
cluster1-pxc-1 3/3 Running 0 21m
cluster1-pxc-2 3/3 Running 0 18m
percona-xtradb-cluster-operator-77bfd8cdc5-ktttb 1/1 Running 0 31m
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-5jxsj 0/1 StartError 0 5m33s
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-7tpl7 0/1 StartError 0 13s
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-cdcjm 0/1 StartError 0 8m13s
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-pfzwq 0/1 StartError 0 9m33s
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-qtjcz 0/1 StartError 0 9m53s
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-rhmvt 0/1 StartError 0 10m
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-snmqn 0/1 StartError 0 10m
xb-cron-cluster1-s3-us-east-20210805221513-8fa30-zdvbb 0/1 StartError 0 10m
xb-cron-cluster1-s3-us-east-20210805222016-8fa30-2xt6s 0/1 StartError 0 8m11s
xb-cron-cluster1-s3-us-east-20210805222016-8fa30-9drdq 0/1 StartError 0 3m31s
xb-cron-cluster1-s3-us-east-20210805222016-8fa30-fc4dj 0/1 StartError 0 8m43s
xb-cron-cluster1-s3-us-east-20210805222016-8fa30-fz4kh 0/1 StartError 0 8m31s
xb-cron-cluster1-s3-us-east-20210805222016-8fa30-hjfv9 0/1 StartError 0 8m41s
xb-cron-cluster1-s3-us-east-20210805222016-8fa30-n95t4 0/1 StartError 0 7m31s
xb-cron-cluster1-s3-us-east-20210805222016-8fa30-zg4vd 0/1 StartError 0 6m11s
xb-cron-cluster1-s3-us-east-20210805222510-8fa30-2qbtl 0/1 StartError 0 77s
xb-cron-cluster1-s3-us-east-20210805222510-8fa30-clxpf 0/1 StartError 0 3m47s
xb-cron-cluster1-s3-us-east-20210805222510-8fa30-gtbb5 0/1 StartError 0 3m37s
xb-cron-cluster1-s3-us-east-20210805222510-8fa30-hc9jv 0/1 StartError 0 3m49s
xb-cron-cluster1-s3-us-east-20210805222510-8fa30-rzskh 0/1 StartError 0 3m17s
xb-cron-cluster1-s3-us-east-20210805222510-8fa30-v68s2 0/1 StartError 0 2m37s

the only message that i have seen is this, i set up my aws secrets as in the example in documentation, but is not working.

Message
failed to create containerd task: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: setenv: invalid argument: unknown

My config file for the backup is this.

backup:
    image: percona/percona-xtradb-cluster-operator:1.8.0-pxc5.7-backup
#    serviceAccountName: percona-xtradb-cluster-operator
#    imagePullSecrets:
#      - name: private-registry-credentials
    pitr:
      enabled: false
      storageName: STORAGE-NAME-HERE
      timeBetweenUploads: 60
    storages:
      s3-us-east:
        type: s3
        s3:
          bucket: percona-cluster-kubernetes
          region: us-east-1
          credentialsSecret: secret-backups          
      fs-pvc:
        type: filesystem
#        nodeSelector:
#          storage: tape
#          backupWorker: 'True'
#        resources:
#          requests:
#            memory: 1G
#            cpu: 600m
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
              nodeSelectorTerms:
              - matchExpressions:
                - key: servicio
                  operator: In
                  values:
                  - Otros
#        tolerations:
#          - key: "backupWorker"
#            operator: "Equal"
#            value: "True"
#            effect: "NoSchedule"
#        annotations:
#          testName: scheduled-backup
        labels:
          servicio: Otros
#          backupWorker: 'True'
#        schedulerName: 'default-scheduler'
#        priorityClassName: 'high-priority'
#        containerSecurityContext:
#          privileged: true
#        podSecurityContext:
#          fsGroup: 1001
#          supplementalGroups: [1001, 1002, 1003]
        volume:
          persistentVolumeClaim:
#            storageClassName: standard
            accessModes: [ "ReadWriteOnce" ]
            resources:
              requests:
                storage: 1000Gi
    schedule:
      - name: "sat-night-backup"
        schedule: "0 0 * * 6"
        keep: 3
        storageName: s3-us-east
      - name: "daily-backup"
        schedule: "*/5 * * * *"
        keep: 3
        storageName: s3-us-east

I hope you can help me to make this work , Thanks !

1 Like
2

Hi @Rub_Av
do you use base64 encoding for passwords?

1 Like
3

Hello , i set them as aws print them when are created , i need to convert them to base 64 ? If yes let me try this and i will update the progress … i think I didn’t understood well this step .

Thanks for your reply !

1 Like
4

Hello, you were right it was my mistake !

But i have an other question , i have a operating non kubernetes Percona Kluster, the docs tells you how to restore backup from existing kubernetes cluster, but how can i make this possible from a non kubernetes cluster?

1 Like
5

Hi @Rub_Av

it is not tested instruction, but it should work

  1. download backup, it can be done only with xbcloud tool
export CREDENTIALS="ENDPOINT=$ENDPOINT ACCESS_KEY_ID=$ACCESS_KEY_ID SECRET_ACCESS_KEY=$SECRET_ACCESS_KEY DEFAULT_REGION=$DEFAULT_REGION"
env -i $CREDENTIALS xbcloud get "${backup_path}" --parallel=10 1>"$dest_dir/xtrabackup.stream"
  1. prepare mysql dir
service mysqld stop
rm -rf /var/lib/mysql/*
  1. unarchive
cat $dest_dir/xtrabackup.stream | xbstream --decompress -x -C /var/lib/mysql
xtrabackup --prepare --target-dir=/var/lib/mysql
  1. set credential and start mysql
chown -R mysql:mysql /var/lib/mysql
service mysqld start
1 Like
6

Thanks, let me try this and i will update !

1 Like