Cluster seems properly synced, but clustercheck script fails

MySQL & MariaDB Percona XtraDB Cluster 5.x
1

Hello,

I have a completely new install of Percona Cluster 5.6 on CentOS 6.6 along the lines of the setup in the following blog post:

https://www.percona.com/blog/2012/06…-with-haproxy/

There are three hosts: each has a MySQL node, plus an HAProxy instance using the standard “clustercheck” script to health-check the local nodes via xinetd. All three hosts share a floating ip address via keepalived.

Every part of the stack except the clustercheck script seems to be working properly.

I.e, I get output like the following on all three hosts:

$ mysql -e "SHOW GLOBAL STATUS LIKE 'wsrep_cluster_%'"
+--------------------------+--------------------------------------+
| Variable_name | Value |
+--------------------------+--------------------------------------+
| wsrep_cluster_conf_id | 9 |
| wsrep_cluster_size | 3 |
| wsrep_cluster_state_uuid | dca6d39f-42dc-11e5-946f-b7d45ebb1bba |
| wsrep_cluster_status | Primary |
+--------------------------+--------------------------------------+

I also created users on one of the nodes, and I can see that it propagated as it should to mysql.user on all cluster members:

mysql> SELECT Host, User FROM mysql.user;
+----------------------+----------+
| Host | User |
+----------------------+----------+
| % | agunn |
| % | ahough |
| % | bharden |
| % | clott |
| % | gloy |
| % | joomla |
| % | mcdba |
| 127.0.0.1 | root |
| ::1 | root |
| localhost | mysqlchk |
| localhost | root |
| localhost | sstuser |
+----------------------+----------+
12 rows in set (0.00 sec)

In HAProxy, all three nodes are failing the health check via xinetd.

I know the xinetd configuration is working because I can reproduce the failed health checks via curl:

$ curl [URL]http://joomdb01:9200/[/URL]
Percona XtraDB Cluster Node is not synced or non-PRIM.

And when I run the clustercheck script manually at the command line outside of xinted, it returns the same output:

$ /usr/bin/clustercheck [actual_password_redacted] 1 /var/log/mysqlchk 0 /etc/my.cnf
HTTP/1.1 503 Service Unavailable
Content-Type: text/plain
Connection: close
Content-Length: 57

Percona XtraDB Cluster Node is not synced or non-PRIM.

Basically, it looks like everything except the clustercheck script is working as intended. Any ideas as to where I should be looking?

Thanks,

-Cal

2

So it turns out this was a typo in the clustercheck user name. I corrected the provisioned userid in MySQL and all was well.

The clustercheck script does not have very informative error message for this particular failure scenario.