Using Clone for addInstance on MySQL Innodb Cluster uses 100% Memory

MySQL & MariaDB Percona Server for MySQL 8.0
1

When trying to add a new node to a MySQL Innodb cluster using Cluster.addInstance() and selecting the Clone option, within 2 or 3 minutes the donor nodes memory usages hits 100% on a server with 36Gb of memory. At 100% memory usage the server becomes unresponsive and the recipient server moves to another server in the cluster, tries to clone, and the memory usage of the 2nd donor hits 100%. The entire cluster is degraded in under 10 min and unavailable.

This is on a Production server so the consequences here are pretty devastating.

I’m running Percona Release 27 (MySQL 8.0.35-27) on Debian 12 Bookworm.

I have not encountered this problem setting up a test cluster with a small amount of data, which I have done several times to ensure I’m doing everything correctly. However on my Production server with about 125Gb of data, this problem occurs consistently.

Has anyone else experienced this problem or know what the cause might be?

Thank you.

2

What is the current, Memory, and what is buffer pool of your instances? Please share that,

3

Total server memory = 36Gb. Dedicated to MySQL.
Buffer pool size = 24Gb

As soon as the clone process starts, it depletes the donor’s memory so fast that I don’t even have time to analyze what is going on. Even killing the clone query doesn’t stop the memory from being consumed.