Percona-xtrabackup-2.4.20 Unable to prepare (--apply-log) for an encrypted database

been using Percona MySQL, PXC, and Xtrabackup for MANY years.

I have recently had issues with Xtrabackup regarding vault configured / encrypted data.
It seems I can easily:
*configure the database environments for vault configured encryption.
*encrypt tables.
*easily backup the same encrypted database to build another slave,

but to date I have been unable to --apply-log for the new slaves. I have spent hours trying to correct the missing arguments with no success.

Why xtrabackup has never been improved to at least mention which argument is missing is beyond comprehension.

If I remove the encrypted tables, and the encrypted syntax from the backups all works well.

I even downloaded the xtrabackup SOURCE to locate the code that indicates missing arguments so I could add debug logic to same.
I had no success in locating the missing argument code.

sudo cat /var/lib/mysql-keyring/keyring_vault.conf
vault_url = https://vault.???.co
secret_mount_point = ???/???/slave
token = s.???
vault_ca = /mysql/data/smule_chain.crt

sudo cat /etc/my.cnf | grep keyring
early-plugin-load = " keyring _vault="
loose-keyring _vault_config = “/mysql/data/ keyring_vault.conf”

The backup steps succeeds based on the vault configuration

sudo ssh_cmd="$ssh_cmd" SOURCE=$SOURCE SOURCETYPE=$SOURCETYPE MYSQLDATA=$MYSQLDATA sh -c ‘$ssh_cmd root@${SOURCE} “innobackupex --user=??? --password=??? --slave-info --transition-key=/qwMUldCpSLdHrrStYEAbPquPDlt6ixV4PVW1+1+zBQ= --safe-slave-backup --stream=xbstream --parallel=4 /tmp 2>/tmp/innobackupex.log” | xbstream -x -C $MYSQLDATA’

The prepare step fails

Xtrabackup is indicating a missing argument without indicating which argument is missing. This is probably the most frustrating aspect of this utility, very poor error messaging. I have spent hours pouring over the many blogs.

sudo innobackupex --user=??? --password=??? --apply-log --use-memory=2G --target-dir=$MYSQLDATA --transition-key=/qwMUldCpSLdHrrStYEAbPquPDlt6ixV4PVW1+1+zBQ= --generate-new-master-key --keyring-vault-config=/var/lib/mysql-keyring/keyring_vault.conf

Job Output

xtrabackup: recognized server arguments: --innodb_checksum_algorithm=crc32 --innodb_log_checksum_algorithm=strict_crc32 --innodb_data_file_path=ibdata1:10M:autoextend --innodb_log_files_in_group=3 --innodb_log_file_size=1073741824 --innodb_fast_checksum=0 --innodb_page_size=16384 --innodb_log_block_size=512 --innodb_undo_directory=./ --innodb_undo_tablespaces=0 --server-id=2021140 --redo-log-version=1 --transition-key=*

220526 16:25:47 innobackupex: Missing argument

Any help is greatly appreciated. MySQL version is 5.7.29-32.

1 Like