Percona XtraDB Cluster 8.0 - cluster size as 1 at node 2 and 3

MySQL & MariaDB Percona XtraDB Cluster 8.x
,
1

Hi there…

when the checked the cluster size showing as below at node 2 and node 3, what could be the problem…?

Attaching the config file of 3 nodes.

Config file - PXC 8.0 .pdf

Node -2

mysql@pxc2 > show status like ‘%cluster_size%’;

±-------------------±------+

| Variable_name | Value |

±-------------------±------+

| wsrep_cluster_size | 1 |

±-------------------±------+

1 row in set (0.00 sec)

Node -3

mysql@pxc3 > show status like ‘%cluster_size%’;

±-------------------±------+

| Variable_name | Value |

±-------------------±------+

| wsrep_cluster_size | 1 |

±-------------------±------+

1 row in set (0.00 sec)

Below is the status when issued the clustercheck command.

root@pxc-1:~# clustercheck

HTTP/1.1 503 Service Unavailable

Content-Type: text/plain

Connection: close

Content-Length:

Percona XtraDB Cluster Node is not synced or non-PRIM.

Config file - PXC 8.0 .pdf (98.8 KB)

1 Like
2

In your configs you should have

wsrep_cluster_address = gcomm://192.168.0.1,192.168.0.2,192.168.0.3

on ALL your nodes, not only on Node 1

1 Like
3

Hi @vadimtk

added the wsrep_cluster_address = gcomm://192.168.0.1,192.168.0.2,192.168.0.3 in all the nodes 1,2,3.

After bootstrapped the first node ,Started the mysql service at node 2 ,facing the below error.

Attaching the error log file for your reference.

node 2 error.log

Note : I have changed the variable from ON to pxc-encrypt-cluster-traffic=OFF in all the config files.

root@pxc-2:~# service mysql start

Job for mysql.service failed because the control process exited with error code.

See “systemctl status mysql.service” and “journalctl -xe” for details.

root@pxc-2:~# systemctl status mysql.service

● mysql.service - Percona XtraDB Cluster

Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)

Active: failed (Result: exit-code) since Fri 2020-10-23 17:54:16 UTC; 1min 21s ago

Process: 6245 ExecStopPost=/usr/bin/mysql-systemd stop-post (code=exited, status=0/SUCCESS)

Process: 3748 ExecStop=/usr/bin/mysql-systemd stop (code=exited, status=0/SUCCESS)

Process: 2375 ExecStartPost=/usr/bin/mysql-systemd start-post $MAINPID (code=exited, status=0/SUCCESS)

Process: 2373 ExecStartPost=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)

Process: 6240 ExecStart=/usr/sbin/mysqld $_WSREP_START_POSITION (code=exited, status=1/FAILURE)

Process: 6192 ExecStartPre=/bin/sh -c VAR=bash /usr/bin/mysql-systemd galera-recovery; [ $? -eq 0 ] && systemctl set-environment _WSREP_START_POSITION=$VAR || exit 1 (code=exited, s

Process: 6186 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)

Process: 6155 ExecStartPre=/usr/bin/mysql-systemd check-grastate (code=exited, status=0/SUCCESS)

Process: 6106 ExecStartPre=/usr/bin/mysql-systemd start-pre (code=exited, status=0/SUCCESS)

Main PID: 6240 (code=exited, status=1/FAILURE)

Status: “Server startup in progress”

Oct 23 17:53:43 pxc-2 systemd[1]: Starting Percona XtraDB Cluster…

Oct 23 17:54:16 pxc-2 systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE

Oct 23 17:54:16 pxc-2 mysql-systemd[6245]: WARNING: mysql pid file /var/run/mysqld/mysqld.pid empty or not readable

Oct 23 17:54:16 pxc-2 mysql-systemd[6245]: WARNING: mysql may be already dead

Oct 23 17:54:16 pxc-2 systemd[1]: mysql.service: Failed with result ‘exit-code’.

Oct 23 17:54:16 pxc-2 systemd[1]: Failed to start Percona XtraDB Cluster.

lines 1-21/21 (END)

[8]+ Stopped systemctl status mysql.service

1 Like
4

Hi there …

Can anyone look in to this issue …?

1 Like
5

Hi @AneeshBabuAs I can see from log

2020-10-23T17:06:18.566323Z 0 [ERROR] [MY-000000] [Galera] failed to open gcomm backend connection: 110: failed to reach primary view (pc.wait_prim_timeout): 110 (Connection timed out)
     at gcomm/src/pc.cpp:connect():159
2020-10-23T17:06:18.566350Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs_core.cpp:gcs_core_open():220: Failed to open backend connection: -110 (Connection timed out)
2020-10-23T17:06:18.566426Z 0 [ERROR] [MY-000000] [Galera] gcs/src/gcs.cpp:gcs_open():1700: Failed to open channel 'pxc-cluster' at 'gcomm://192.168.0.1,192.168.0.2,192.168.0.3': -110 (Connection timed out)
2020-10-23T17:06:18.566444Z 0 [ERROR] [MY-000000] [Galera] gcs connect failed: Connection timed out

It might be connected with SELinux polices.

According to documentation it is recommended the best solution is to change the mode from enforcing to permissive by running the following command:

setenforce 0

This only changes the mode at runtime. To run SELinux in permissive mode after a reboot, set SELINUX=permissive in the /etc/selinux/config configuration file.

1 Like
6

Hii @AneeshBabu is ur issue solved?Iam also facing similar issue can anyone help me.

1 Like
7

Hii @Evgeniy_Patlan i followed ur steps but still iam getting same error.

1 Like
8

@Naveen could you please share your log?

1 Like
9

Hii@Evgeniy_patlan thanks for ur rply ,that was a firewall issue.Now iam getting another issue.
iam trying to add second node to my cluster(xtradb cluster 8.0) but it is giving sst error.

perc
perc1066×652 8.32 KB

1 Like
10

@Evgeniy_Patlan It would be great if u could help me in solving this error.

1 Like
11

@Naveen please create ticket regarding this issue in PXC project in jira.percona.com

1 Like
12

@matthewb can u see this issue once

1 Like
13

I see an issue with socat and your tcp/ip ports. Please make sure SELinux is in permissive mode, turn off any firewalls, and make sure your gcomm:// ip addresses are correct.

1 Like
14

@matthewb their is no SELinux in my machines,turned off all firewalls and ip addresses are correct.

1 Like
15

Ok, perform some networking troubleshooting basics. Try to telnet from A to B on port 4567. Do you get a response? If not, you still have networking issues.

Try this, on B, run socat - TCP-LISTEN:4444,reuseaddr and on A run this
echo "hello" | socat - TCP:nodeB:4444

Did that work? If not, you have networking issues.

1 Like
16

@matthewb exactly 2 addresses required (there are 0) on node A and node B.This is the error iam getting.can u tell me any solutions to solve it

1 Like
17

what do you mean “there are 0”? Your servers must have IP addresses on them. How are you accessing them otherwise?

1 Like
18

on Node A :E getaddrinfo(ip):name or service not known
on Node B:E bind(5,{AF= ip},16):address already in use

1 Like
19

How can node A not have an IP? How are you accessing this server? Console? Node A is clearly messed up at a deeper level.

1 Like
20

Hii @matthewb i will attach screenshots of commands
1.ON node B
NB1

2.On Node A
NA

3.On Node B i got output
NB2

So i think their is no network issue…

1 Like