I am seeing this error during prepare. Is it related to InnoDB? My tables are all RocksDB tables though.
Created 128 and tracked 128 new rollback segment(s) in the temporary tablespace. 128 are now active.
8.0.23 started; log sequence number 3585483693664 Allocated tablespace ID 1 for sys/sys_config, old maximum was 0
xtrabackup: Last MySQL binlog file position 198671947, file name mysql-bin.002091
xtrabackup: error: The transaction log file is corrupted. xtrabackup: error: The log was not applied to the intended LSN!
xtrabackup: Log applied to lsn 3585483693664 xtrabackup: The intended lsn is 3585483693674
I also see XtraBackup is version 8.0.23-16 while mysql seems version 8.0.22-13. Are they compatible?
xtrabackup version (this is the one used for prepare; the backup itself was taken with a few months older version):
/usr/bin/xtrabackup version 8.0.23-16 based on MySQL server 8.0.23 Linux (x86_64) (revision id: 934bc8f)
/usr/bin/mysql Ver 8.0.22-13 for Linux on x86_64 (Percona Server (GPL), Release ‘13’, Revision ‘6f7822f’)
Any clues on what may be causing the above error?
In MySQL 8.0, all system tables became InnoDB. So even though “my tables are all MyRocks tables”, there are still a significant number of tables which are InnoDB.
Yes, any newer version of xtrabackup is always backwards compatible with older versions of MySQL.
I suspect that your original backup process failed. Do you have these logs/output?
Thanks for the response. During the backups no errors were returned. Also, the hosts that are doing backups have been working fine for a while. What is new is that the host where I am doing restore was setup yesterday and it has new xtrabackup build.
This new build gave the same error on 3 different backups, and those backups were taken from at least 2 different old hosts. Rather than suspecting that all those 3 backups are bad, I am suspecting that something may be wrong with the new xtrabackup build.
To narrow down the possibilities, when installing percona-xtrabackup-80 on a new host, how do I install the previous version of xtrabackup rather than latest version?
Ok, I was able to install old version of xtrabackup by doing:
sudo apt-get install percona-xtrabackup-80=8.0.22-15-1.bionic
I found this one was able to restore the backup fine.
So there seems to be something wrong with the latest xtrabackup build 8.0.23-16-1.bionic.
So it looks like new xtrabackup version is unable to restore backups taken from those older servers where we set sql_log_bin to 0. Those are servers where we do only bulk load operations, and so turned off writing to bin logs. We bulk load same files to all replicas rather than using MySQL replication.
But I believe those backups are good because previous xtrabackup version is able to restore them fine. We have been running such servers for several months now and regularly testing backups and restores.
New xtrabackup version is unable to restore those backups. So fr now we are pinning the old xtrabackup, percona-server-server , pecona-server-client and percona-server-rocksdb.