Handling concurrent writes on multiple nodes without coordination

Hi Percona experts,

We are seeking guidance on a following issue in Percona XtraDB Cluster.

In a multi-master environment like PXC, it might be possible to encounter an issue with concurrent writes on multiple nodes without coordination. From my understanding, this may happen when the same underlying data is simultaneously modified on different nodes with conflicting changes.

We are trying to understand and analyze this specific type of failure. Could you describe how this situation might happen and how to troubleshoot it? On what logs and status variables should we focus? Additionally, how to solve this once it happens?

Thank you in advance for your input!

Hello @SQLCaesar,

This situation happens when best practices are not followed. It not considered best practice to write to multiple nodes in PXC, in general. PXC is a high-availability solution, not a write-scaling solution. This means only 1 node should be used as the primary writer and the other nodes as readers. In the event of primary failure, one of the readers can immediately take over as writer.

There is a special situations where you can write to multiple nodes. This would be application1 writes only to tables in database1, app2 writes only to database2, etc. In this situation, no writes from app1 can interfere with writes from app2, with regards to table/row locks.

There are two types of errors: Local Certification Failure, and Brute-Force Aborts. Both have counters in SHOW GLOBAL STATUS. Both situations return back specific error codes to your application. Your could should be validating the result of any SQL. If you get one of these, you simply try the transaction again. But again, the best practice is to only write to 1 node.

1 Like