I am having an issue with my Galera setup and I am not sure how to fix it, or even if it is possible to fix it. 3 node cluster
So we use drupal and are in the process of upgraing our main site, the webteam is attempting to sync with new development servers from production, however there are a ton of problematic SQL being generated from drush, or even the XML imports so we see errors like
WSREP: Ignoring error for TO isolated action: source: 5a5cadb2-0d9c-11e7-ab52-a200c70c84fe version: 3 local: 0 state: APPLYING flags: 65 conn_id: 24791 trx_id: -1 seqnos (l: 160865, g: 21333163, s: 21333162, d: 21333162, ts: 2849868880088527
[ERROR] Slave SQL: Error 'Out of range value for column ‘created’
Internal MariaDB error code: 1264
[Warning] WSREP: RBR event 1 Query apply warning: 1, 213331
Caused by something like a out of range int(11) in about 50 of the SQL inserts.
This eventually leads to the node disjoining
2017-03-21 13:46:23 140707578771200 [Note] WSREP: forgetting 557b5e4c (tcp://10.76.10.103:4567)
2017-03-21 13:46:23 140707578771200 [Note] WSREP: forgetting 5a5cadb2 (tcp://10.76.10.102:4567)
2017-03-21 13:46:23 140707578771200 [Note] WSREP: Node 41acb7bc state prim
Now this is bad enough as it will cause other applications connecting to the server to start receiving state errors from the node, however the problem gets worse as on occasion this will happen to multiple servers at the same time in the cluster starts to fail communication with each other and I will have to bootstrap from the last node to go down.
ive read this article:
But I cannot seem to find out how to keep the cluster from crashing like this, other than to ensure the SQL is always clean, which is unrealistic. I could increase the nodes to 5, so 2 could fail and the system is maintained, but I am not sure that is best for our setup. TOI is set on all the machines.
thanks in advance for any help.
These groups are read by MariaDB server.
Use it for options that only the server (but not clients) should see
See the examples of server my.cnf files in /usr/share/mysql/
this is read by the standalone daemon and embedded servers
bind-address = 0.0.0.0
this is only for the mysqld standalone daemon
general_log = on
* Galera-related settings
wsrep_cluster_name=’ cluster1 ’
this is only for embedded server
This group is only read by MariaDB servers, not by MySQL.
If you use the same .cnf file for MySQL and MariaDB,
you can put MariaDB-only options here
This group is only read by MariaDB-10.0 servers.
If you use the same .cnf file for MariaDB of different versions,
use this group for options that older servers don’t understand