Xtrabackup prepare consumes high amounts of memory

MySQL & MariaDB Percona XtraBackup
1

Hello,

After upgrading to mysql 8.0.25 from 5.7.31, xtrabackup --prepare consumes high amounts of memory for databases with many tables.

Options like --use-memory does not make any difference.
Number of tables is around 1-2 millions.

For example with --use-memory 256M argument for a server with 400 000 tables (each in a separate file) xtrabackup consumes around 8Gi of memory.
When number of tables is close to 1 000 000, it consumes more than 20Gi of memory.

Is it expected? With mysql 5.7 --use-memory argument worked fine, and memory consumptions was close it.

2

Hi @Mikhail_Nikitin . Thanks for your request.
We are reworking the way xtrabackup opens tables for it to be smart regarding memory footprint.
The behavior you are seeing is aligned with some internal research. We plan to land a new data dictionary cache functionality in the next release of xtrabackup. You can subscribe to [PXB-2955] Implement dictionary table cache - Percona JIRA to get updates when this work is released.

3

PXB-2955 is completed and released as part PXB 8.0.33-28