# Async replication breaking after another a node joins the cluster

**URL:** <https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708>\
**Category:** Percona XtraDB Cluster 5.x\
**Created:** [August 10, 2021, 8:04am UTC](https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708 "2021-08-10T08:04:35Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![acampoh](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/acampoh/32/4179_2.png) [@acampoh](https://forums.percona.com/u/acampoh)\
**Post date:** [August 10, 2021, 8:04am UTC](https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708/1 "2021-08-10T08:04:35Z")

</div>

Hello,

I’m struggling trying to set up async replication between 2 xtradb 5.7.31 clusters. Each cluster has 3 nodes.

If i only start one node of the secondary cluster and start re plication, it works fine, but as soon as i start another node and it joins after performing an SST, the async replication breaks due to a duplicated key.

At first i thought it was because when it performs an SST, it also brings the slave information and it starts the slave in the new node, but i put on my.cnf the skip-slave-start option, but keeps failing.

I have tried also to start the cluster in RO just to be sure nobody else writes, but i still have the same problem.

Last thing i have tried is setup the IGNORE\_SERVER\_IDS on change master query, but still keeps failing.

the change master query is this one

CHANGE MASTER TO  
MASTER\_HOST = “MY\_SERVER\_IP”,  
MASTER\_PORT = 3306,  
MASTER\_USER = “replic”,  
MASTER\_PASSWORD = “what\_a\_passwd”,  
MASTER\_AUTO\_POSITION = 1, # to auto-position using GTID on slave  
IGNORE\_SERVER\_IDS = (1,2,3);

for the records, both clusters have GTID enabled

i guess i’m missing something, but i cannot see what. Any ideas / suggestions are more than welcomed,

thanks in advance,  
Adrián

---

<div class="post-metadata">

**Author:** ![Michael\_Coburn](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/michael_coburn/32/18_2.png) [@Michael\_Coburn](https://forums.percona.com/u/Michael_Coburn)\
**Post date:** [August 10, 2021, 3:52pm UTC](https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708/2 "2021-08-10T15:52:13Z")

</div>

Hi @acampoh , thanks for posting to the Forums 🙂

I have used @yves.trudeau wonderful Replication Manager between two PXC 5.7 clusters

> **[Mysql-tools/PXC at master · y-trudeau/Mysql-tools](https://github.com/y-trudeau/Mysql-tools/tree/master/PXC)**
>
> master/PXC

In fact there is a section on ensuring SSTs are done as well as steps to ensure there is only one GTID sequence in the cluster.

> **[Mysql-tools/PXC at master · y-trudeau/Mysql-tools - Configuration steps](https://github.com/y-trudeau/Mysql-tools/tree/master/PXC#configuration-steps)**
>
> master/PXC

Best of luck,

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [August 10, 2021, 4:15pm UTC](https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708/3 "2021-08-10T16:15:41Z")

</div>

Just to understand, when the 2nd node joins the secondary cluster, ‘SHOW SLAVE STATUS’ actually reports information?

---

<div class="post-metadata">

**Author:** ![acampoh](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/acampoh/32/4179_2.png) [@acampoh](https://forums.percona.com/u/acampoh)\
**Post date:** [August 10, 2021, 4:28pm UTC](https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708/4 "2021-08-10T16:28:02Z")

</div>

Yes, it reports all the info about the replication on that node

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [August 10, 2021, 8:34pm UTC](https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708/5 "2021-08-10T20:34:28Z")

</div>

Ok. Most likely because the SST process is using xtrabackup which clones the entire $DATADIR which would include the master.info file. It is surprising that skip-slave-start is not having any affect. That would seem to be a bug in base MySQL. Can you try setting server\_id=0 in the joining node along with s-s-s? 0 means “cannot be slave nor master”.

---

<div class="post-metadata">

**Author:** ![acampoh](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/acampoh/32/4179_2.png) [@acampoh](https://forums.percona.com/u/acampoh)\
**Post date:** [August 12, 2021, 3:42am UTC](https://forums.percona.com/t/async-replication-breaking-after-another-a-node-joins-the-cluster/11708/6 "2021-08-12T03:42:36Z")

</div>

Hello,

Just found the issue. It wasn’t fully related with replication, it was more with the startup process of the 2nd and 3rd nodes of the cluster.

My mistake was to think that having the first node as RO, the other nodes when they do sst and join were going to inherit the RO variable too, but they didn’t. As we have sentry running there, it was writting and creating a duplicated key error everytime one of these nodes started.

I fixed it once setting the read\_only option in my.cnf.

thanks a lot!
