Percona xtrabackup intermediately failing

MySQL & MariaDB Percona XtraBackup
1

We are encountering intermittent failures while rebuilding our Percona MySQL slaves using XtraBackup. However, after retrying a few times, the backup is successful. Below are the error logs, please help us with the possible reasons for the failures:

Error logs:

applying transactional logs failed: exit status 1

2023-08-05 12:01:27 ERROR [xxxx] Error occurred while running xbstream process: exit status 1
2023-08-05 12:01:27 INFO [xxxx] Completed xbstream process. Output:

2

Hi Pravata,
Can you provide some more information?
MySQL version,
Xtrabackup version that you are using.
Is the version same on source and destination?
How frequently you test backups and from when did it start failing?

3

Hi Yunus,

Please find the below details:
MySQL version: 5.7.39-42
Xtrabackup version:2.4.26-1.bullseye

Is the version same on source and destination? —> YES

How frequently you test backups and from when did it start failing?
Ans: When the slave replication breaks for any reason, xtrabackup automatically rebuilds it. We have encountered backup failures due to the error mentioned above, but after retrying 2-3+ times, it succeeds.

4

Looking at your answer, it seems you are using some script which rebuilds your replica using xtrabackup.
Strange to see that after retrying 2-3 times, it succeeds.
It is possible that the backup does not complete due to some reason, may be because of any DDL or any other reason while streaming.
Do you redirect complete backup logs and prepare logs to some file in your script? Can you share them for the failed instance?

5

I sure hope that you are logging these error messages and reasons for replication breaking. This could represent a more serious issue in your application. Blindly taking action is not a good practice.