Cannot use MySQL my.cnf centos7

MySQL & MariaDB Percona XtraDB Cluster 5.x
,
1

I have 2 nodes to join percona xtaDB cluster 5.7 . I installed them successfully , node 1 was able to start working . when I switched to node 2 configuration, I fixed the file my.cnf

cat >>/etc/my.cnf<<EOF
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_name=democluster
wsrep_cluster_address=gcomm://node1,node2
wsrep_node_name=centosvm02
wsrep_node_address=node2
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=user:123abc@A
pxc_strict_mode=ENFORCING
binlog_format=ROW
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
EOF

I have 2 nodes to join percona cluster . I installed them successfully , node 1 was able to start working . when I switched to node 2 configuration, I fixed the file my.cnf

cat >>/etc/my.cnf<<EOF
[mysqld]
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_name=democluster
wsrep_cluster_address=gcomm://node1,node2
wsrep_node_name=centosvm02
wsrep_node_address=node2
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=user:123abc@A
pxc_strict_mode=ENFORCING
binlog_format=ROW
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
EOF

After fixing the file, I tried to run systemctl start mysql , and it gave me an error job for mysql.service failed because a fatal signal was delivered to the control process. see “systemctl status mysql.service” and “journalctl -xe” for details. tôi đã mở theo gợi ý journalctl -xe .Every time I edit the my.cnf file, I get an error, when I go back to default, the error goes away

Screenshot_135
Screenshot_135784Ă—446 210 KB

1 Like
2

You need to use journalctl -u mysqld to only show log entries for MySQL.

Your gcomm:// lists node1,node2 as servers. Those should be hostnames. Do they resolve correctly in your environment? If they do not, then that is 1 problem you must fix. Try using IP addresses and not hostnames.

1 Like
3

i used ip address instead of nameserver
I opened the file /var/log/mysq.log got this error . currently i enable port (3306/tcp,4444/tcp,4567/tcp,4568/tcp) on my firewall both nodes

2022-05-25T08:42:54.273962Z 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address 'xyz.abc.0.127' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '9560' --mysqld-version '5.7.37-40-57'   '' 
	Read: '(null)'
2022-05-25T08:42:54.274008Z 0 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address 'xyz.abc.0.127' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '9560' --mysqld-version '5.7.37-40-57'   '' : 2 (No such file or directory)
2022-05-25T08:42:54.273962Z 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address 'xyz.abc.0.127' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '9560' --mysqld-version '5.7.37-40-57'   '' 
	Read: '(null)'
2022-05-25T08:42:54.274008Z 0 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address 'xyz.abc.0.127' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '9560' --mysqld-version '5.7.37-40-57'   '' : 2 (No such file or directory)
1 Like
4

Can you please show the my.cnf for both servers again.

1 Like
5

@hieu_nguyen,
Make your gcomm like this, make sure to include the port:

gcomm://10.10.10.10:4567,10.10.10.11:4567
1 Like
6

this is what’s in my my.cnf

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
wsrep_provider=/usr/lib64/galera3/libgalera_smm.so
wsrep_cluster_name=democluster
wsrep_cluster_address=gcomm://abc.112.0.1,abc.112.0.2
wsrep_node_name=centosvm02
wsrep_node_address=abc.112.0.1
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=user:123abc@A
pxc_strict_mode=ENFORCING
binlog_format=ROW
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

[xtrabackup]
target_dir = /opt/backupdb/mysql
1 Like
7

my node 2 still can’t join

8

Did you bootstrap node1?

1 Like
9

yes it’s turned on

 mysql@bootstrap.service - Percona XtraDB Cluster with config /etc/sysconfig/mysql.bootstrap
   Loaded: loaded (/usr/lib/systemd/system/mysql@.service; disabled; vendor preset: disabled)
   Active: active (running) since Fri 2022-05-27 08:28:02 +07; 17min ago
  Process: 19785 ExecStartPost=/usr/bin/mysql-systemd start-post $MAINPID (code=exited, status=0/SUCCESS)
  Process: 19739 ExecStartPre=/usr/bin/mysql-systemd start-pre (code=exited, status=0/SUCCESS)
 Main PID: 19783 (mysqld_safe)
   CGroup: /system.slice/system-mysql.slice/mysql@bootstrap.service
           ├─19783 /bin/sh /usr/bin/mysqld_safe --basedir=/usr  --wsrep-new-cluster ...
           └─20407 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin...

May 27 08:27:52 localhost.localdomain systemd[1]: Starting Percona XtraDB Cluster w....
May 27 08:27:52 localhost.localdomain mysql-systemd[19785]: State transfer in progre...
May 27 08:27:53 localhost.localdomain mysqld_safe[19783]: 2022-05-27T01:27:53.291396...
May 27 08:27:53 localhost.localdomain mysqld_safe[19783]: 2022-05-27T01:27:53.295110...
May 27 08:27:53 localhost.localdomain mysqld_safe[19783]: 2022-05-27T01:27:53.330597...
May 27 08:27:53 localhost.localdomain mysqld_safe[19783]: 2022-05-27T01:27:53.343160...
May 27 08:27:53 localhost.localdomain mysqld_safe[19783]: 2022-05-27T01:27:53.345240...
May 27 08:28:02 localhost.localdomain mysql-systemd[19785]: SUCCESS!
May 27 08:28:02 localhost.localdomain systemd[1]: Started Percona XtraDB Cluster wi....
Hint: Some lines were ellipsized, use -l to show in full.
1 Like
10

after fixing it came this error

2022-05-27T01:28:28.793480Z 0 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '192.168.254.127' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '25039' --mysqld-version '5.7.37-40-57'   ''
Read: '(null)'
2022-05-27T01:28:28.793518Z 0 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '192.168.254.127' --datadir '/var/lib/mysql/' --defaults-file '/etc/my.cnf' --defaults-group-suffix '' --parent '25039' --mysqld-version '5.7.37-40-57'   '' : 13 (Permission denied)
2022-05-27T01:28:28.793569Z 2 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
2022-05-27T01:28:28.793586Z 2 [ERROR] Aborting
1 Like
11

What does mysql’s error log on node1 say when you start node2? (not journalctl, need to see mysql’s error log)

1 Like
12

Here is a log of node 1

2022-05-27T02:58:22.197073Z 0 [Note] WSREP: (3adb7bd1, 'tcp://[0.0.0.0:4567](http://0.0.0.0:4567)') connection established to e022125e tcp://[192.168.254.127:4567](http://192.168.254.127:4567)
2022-05-27T02:58:22.197551Z 0 [Note] WSREP: (3adb7bd1, 'tcp://[0.0.0.0:4567](http://0.0.0.0:4567)') turning message relay requesting on, nonlive peers:
2022-05-27T02:58:22.698234Z 0 [Note] WSREP: declaring e022125e at tcp://[192.168.254.127:4567](http://192.168.254.127:4567) stable
2022-05-27T02:58:22.698891Z 0 [Note] WSREP: Node 3adb7bd1 state primary
2022-05-27T02:58:22.699468Z 0 [Note] WSREP: Current view of cluster as seen by this node
view (view_id(PRIM,3adb7bd1,4)
memb {
3adb7bd1,0
e022125e,0
}
joined {
}
left {
}
partitioned {
}
)
2022-05-27T02:58:22.699521Z 0 [Note] WSREP: Save the discovered primary-component to disk
2022-05-27T02:58:22.701222Z 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
2022-05-27T02:58:22.701451Z 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: dedd1d39-dd68-11ec-8e6b-7b524de222d7
2022-05-27T02:58:22.703332Z 0 [Note] WSREP: STATE EXCHANGE: sent state msg: dedd1d39-dd68-11ec-8e6b-7b524de222d7
2022-05-27T02:58:22.703627Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: dedd1d39-dd68-11ec-8e6b-7b524de222d7 from 0 (centosvm01)
2022-05-27T02:58:23.197567Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: dedd1d39-dd68-11ec-8e6b-7b524de222d7 from 1 (centosvm02)
2022-05-27T02:58:23.197620Z 0 [Note] WSREP: Quorum results:
version = 6,
component = PRIMARY,
conf_id = 3,
members = 1/2 (primary/total),
act_id = 8,
last_appl. = 0,
protocols = 0/9/3 (gcs/repl/appl),
group UUID = 165040b8-dbcf-11ec-895e-6b97581f251f
2022-05-27T02:58:23.197646Z 0 [Note] WSREP: Flow-control interval: [141, 141]
2022-05-27T02:58:23.198210Z 1 [Note] WSREP: REPL Protocols: 9 (4, 2)
2022-05-27T02:58:23.198253Z 1 [Note] WSREP: REPL Protocols: 9 (4, 2)
2022-05-27T02:58:23.198284Z 1 [Note] WSREP: New cluster view: global state: 165040b8-dbcf-11ec-895e-6b97581f251f:8, view# 4: Primary, number of nodes: 2, my index: 0, protocol version 3
2022-05-27T02:58:23.198297Z 1 [Note] WSREP: Setting wsrep_ready to true
2022-05-27T02:58:23.198306Z 1 [Note] WSREP: Auto Increment Offset/Increment re-align with cluster membership change (Offset: 1 -> 1) (Increment: 1 -> 2)
2022-05-27T02:58:23.198321Z 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2022-05-27T02:58:23.198342Z 1 [Note] WSREP: Assign initial position for certification: 8, protocol version: 4
2022-05-27T02:58:23.198420Z 0 [Note] WSREP: Service thread queue flushed.
2022-05-27T02:58:25.535524Z 0 [Note] WSREP: (3adb7bd1, 'tcp://[0.0.0.0:4567](http://0.0.0.0:4567)') turning message relay requesting off
2022-05-27T02:58:26.201773Z 0 [Note] WSREP: forgetting e022125e (tcp://[192.168.254.127:4567](http://192.168.254.127:4567))
2022-05-27T02:58:26.201856Z 0 [Note] WSREP: Node 3adb7bd1 state primary
2022-05-27T02:58:26.201889Z 0 [Note] WSREP: Current view of cluster as seen by this node
view (view_id(PRIM,3adb7bd1,5)
memb {
3adb7bd1,0
}
joined {
}
left {
}
partitioned {
e022125e,0
}
)
2022-05-27T02:58:26.201908Z 0 [Note] WSREP: Save the discovered primary-component to disk
2022-05-27T02:58:26.203550Z 0 [Note] WSREP: forgetting e022125e (tcp://[192.168.254.127:4567](http://192.168.254.127:4567))
2022-05-27T02:58:26.203592Z 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
2022-05-27T02:58:26.203835Z 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: e0f386f3-dd68-11ec-80bd-6b73714f5954
2022-05-27T02:58:26.203865Z 0 [Note] WSREP: STATE EXCHANGE: sent state msg: e0f386f3-dd68-11ec-80bd-6b73714f5954
2022-05-27T02:58:26.203880Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: e0f386f3-dd68-11ec-80bd-6b73714f5954 from 0 (centosvm01)
2022-05-27T02:58:26.203893Z 0 [Note] WSREP: Quorum results:
version = 6,
component = PRIMARY,
conf_id = 4,
members = 1/1 (primary/total),
act_id = 8,
last_appl. = 0,
protocols = 0/9/3 (gcs/repl/appl),
group UUID = 165040b8-dbcf-11ec-895e-6b97581f251f
2022-05-27T02:58:26.203904Z 0 [Note] WSREP: Flow-control interval: [100, 100]
2022-05-27T02:58:26.204001Z 1 [Note] WSREP: REPL Protocols: 9 (4, 2)
2022-05-27T02:58:26.204020Z 1 [Note] WSREP: REPL Protocols: 9 (4, 2)
2022-05-27T02:58:26.204035Z 1 [Note] WSREP: New cluster view: global state: 165040b8-dbcf-11ec-895e-6b97581f251f:8, view# 5: Primary, number of nodes: 1, my index: 0, protocol version 3
2022-05-27T02:58:26.204044Z 1 [Note] WSREP: Setting wsrep_ready to true
2022-05-27T02:58:26.204053Z 1 [Note] WSREP: Auto Increment Offset/Increment re-align with cluster membership change (Offset: 1 -> 1) (Increment: 2 -> 1)
2022-05-27T02:58:26.204062Z 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2022-05-27T02:58:26.204075Z 1 [Note] WSREP: Assign initial position for certification: 8, protocol version: 4
2022-05-27T02:58:26.204163Z 0 [Note] WSREP: Service thread queue flushed.
2022-05-27T02:58:31.536792Z 0 [Note] WSREP: cleaning up e022125e (tcp://[192.168.254.127:4567](http://192.168.254.127:4567))
2022-05-27T04:10:19.547842Z 0 [Note] WSREP: Received shutdown signal. Will sleep for 10 secs before initiating shutdown. pxc_maint_mode switched to SHUTDOWN
2022-05-27T04:10:29.548767Z 0 [Note] WSREP: Stop replication
2022-05-27T04:10:29.548825Z 0 [Note] WSREP: Closing send monitor...
2022-05-27T04:10:29.548844Z 0 [Note] WSREP: Closed send monitor.
2022-05-27T04:10:29.548916Z 0 [Note] WSREP: gcomm: terminating thread
2022-05-27T04:10:29.548947Z 0 [Note] WSREP: gcomm: joining thread
2022-05-27T04:10:29.549178Z 0 [Note] WSREP: gcomm: closing backend
2022-05-27T04:10:29.549251Z 0 [Note] WSREP: Current view of cluster as seen by this node
view ((empty))
2022-05-27T04:10:29.549461Z 0 [Note] WSREP: gcomm: closed
2022-05-27T04:10:29.549497Z 0 [Note] WSREP: Received self-leave message.
2022-05-27T04:10:29.549528Z 0 [Note] WSREP: Flow-control interval: [0, 0]
2022-05-27T04:10:29.549544Z 0 [Note] WSREP: Received SELF-LEAVE. Closing connection.
2022-05-27T04:10:29.549553Z 0 [Note] WSREP: Shifting SYNCED -> CLOSED (TO: 8)
2022-05-27T04:10:29.549584Z 0 [Note] WSREP: RECV thread exiting 0: Success
2022-05-27T04:10:29.549618Z 0 [Note] WSREP: recv_thread() joined.
2022-05-27T04:10:29.549635Z 0 [Note] WSREP: Closing replication queue.
2022-05-27T04:10:29.549655Z 0 [Note] WSREP: Closing slave action queue.
2022-05-27T04:10:29.549671Z 0 [Note] WSREP: Waiting for active wsrep applier to exit
2022-05-27T04:10:29.549624Z 1 [Note] WSREP: New cluster view: global state: 165040b8-dbcf-11ec-895e-6b97581f251f:8, view# -1: non-Primary, number of nodes: 0, my index: -1, protocol version 3
2022-05-27T04:10:29.549729Z 1 [Note] WSREP: Setting wsrep_ready to false
2022-05-27T04:10:29.549740Z 1 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2022-05-27T04:10:29.569386Z 1 [Note] WSREP: applier thread exiting (code:0)
2022-05-27T04:10:29.569858Z 7 [Note] WSREP: applier thread exiting (code:6)
2022-05-27T04:10:29.569881Z 10 [Note] WSREP: applier thread exiting (code:6)
2022-05-27T04:10:29.569873Z 5 [Note] WSREP: applier thread exiting (code:6)
2022-05-27T04:10:29.569877Z 4 [Note] WSREP: applier thread exiting (code:6)
2022-05-27T04:10:29.569883Z 6 [Note] WSREP: applier thread exiting (code:6)
2022-05-27T04:10:29.569887Z 3 [Note] WSREP: applier thread exiting (code:6)
2022-05-27T04:10:29.569863Z 8 [Note] WSREP: applier thread exiting (code:6)
2022-05-27T04:10:29.570162Z 2 [Note] WSREP: rollbacker thread exiting
2022-05-27T04:10:29.570353Z 0 [Note] Giving 0 client threads a chance to die gracefully
2022-05-27T04:10:29.570368Z 0 [Note] Shutting down slave threads
2022-05-27T04:10:29.570387Z 0 [Note] Forcefully disconnecting 0 remaining clients
2022-05-27T04:10:29.570414Z 0 [Note] Event Scheduler: Purging the queue. 0 events
2022-05-27T04:10:29.580844Z 0 [Note] WSREP: Service thread queue flushed.
2022-05-27T04:10:29.581021Z 0 [Note] WSREP: MemPool(SlaveTrxHandle): hit ratio: 0, misses: 0, in use: 0, in pool: 0
2022-05-27T04:10:29.636067Z 0 [Note] Binlog end
2022-05-27T04:10:29.641970Z 0 [Note] Shutting down plugin 'validate_password'
2022-05-27T04:10:29.642007Z 0 [Note] Shutting down plugin 'ngram'
2022-05-27T04:10:29.642019Z 0 [Note] Shutting down plugin 'BLACKHOLE'
2022-05-27T04:10:29.642029Z 0 [Note] Shutting down plugin 'ARCHIVE'
2022-05-27T04:10:29.642038Z 0 [Note] Shutting down plugin 'partition'
2022-05-27T04:10:29.642046Z 0 [Note] Shutting down plugin 'MRG_MYISAM'
2022-05-27T04:10:29.642058Z 0 [Note] Shutting down plugin 'MyISAM'
2022-05-27T04:10:29.642078Z 0 [Note] Shutting down plugin 'CSV'
2022-05-27T04:10:29.642090Z 0 [Note] Shutting down plugin 'MEMORY'
2022-05-27T04:10:29.642099Z 0 [Note] Shutting down plugin 'INNODB_TABLESPACES_SCRUBBING'
2022-05-27T04:10:29.642108Z 0 [Note] Shutting down plugin 'INNODB_TABLESPACES_ENCRYPTION'
2022-05-27T04:10:29.642116Z 0 [Note] Shutting down plugin 'INNODB_SYS_VIRTUAL'
2022-05-27T04:10:29.642125Z 0 [Note] Shutting down plugin 'INNODB_CHANGED_PAGES'
2022-05-27T04:10:29.642133Z 0 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2022-05-27T04:10:29.642141Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2022-05-27T04:10:29.642149Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2022-05-27T04:10:29.642158Z 0 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2022-05-27T04:10:29.642166Z 0 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2022-05-27T04:10:29.642174Z 0 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2022-05-27T04:10:29.642191Z 0 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2022-05-27T04:10:29.642199Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2022-05-27T04:10:29.642208Z 0 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2022-05-27T04:10:29.642216Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2022-05-27T04:10:29.642224Z 0 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2022-05-27T04:10:29.642232Z 0 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2022-05-27T04:10:29.642240Z 0 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2022-05-27T04:10:29.642248Z 0 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2022-05-27T04:10:29.642256Z 0 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2022-05-27T04:10:29.642277Z 0 [Note] Shutting down plugin 'INNODB_METRICS'
2022-05-27T04:10:29.642289Z 0 [Note] Shutting down plugin 'INNODB_TEMP_TABLE_INFO'
2022-05-27T04:10:29.642297Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2022-05-27T04:10:29.642305Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2022-05-27T04:10:29.642313Z 0 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2022-05-27T04:10:29.642321Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2022-05-27T04:10:29.642330Z 0 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2022-05-27T04:10:29.642338Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2022-05-27T04:10:29.642346Z 0 [Note] Shutting down plugin 'INNODB_CMPMEM'
2022-05-27T04:10:29.642354Z 0 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2022-05-27T04:10:29.642362Z 0 [Note] Shutting down plugin 'INNODB_CMP'
2022-05-27T04:10:29.642370Z 0 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2022-05-27T04:10:29.642378Z 0 [Note] Shutting down plugin 'INNODB_LOCKS'
2022-05-27T04:10:29.642387Z 0 [Note] Shutting down plugin 'INNODB_TRX'
2022-05-27T04:10:29.642395Z 0 [Note] Shutting down plugin 'XTRADB_ZIP_DICT_COLS'
2022-05-27T04:10:29.642403Z 0 [Note] Shutting down plugin 'XTRADB_ZIP_DICT'
2022-05-27T04:10:29.642411Z 0 [Note] Shutting down plugin 'XTRADB_RSEG'
2022-05-27T04:10:29.642419Z 0 [Note] Shutting down plugin 'XTRADB_INTERNAL_HASH_TABLES'
2022-05-27T04:10:29.642427Z 0 [Note] Shutting down plugin 'XTRADB_READ_VIEW'
2022-05-27T04:10:29.642435Z 0 [Note] Shutting down plugin 'InnoDB'
2022-05-27T04:10:29.642524Z 0 [Note] InnoDB: FTS optimize thread exiting.
2022-05-27T04:10:29.642690Z 0 [Note] InnoDB: Starting shutdown...
2022-05-27T04:10:29.742891Z 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
2022-05-27T04:10:29.743381Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 220527 11:10:29
2022-05-27T04:10:29.943333Z 0 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool
2022-05-27T04:10:30.752789Z 0 [Note] InnoDB: Shutdown completed; log sequence number 2758228
2022-05-27T04:10:30.753169Z 0 [Note] InnoDB: Removed temporary tablespace data file: "ibtmp1"
2022-05-27T04:10:30.753202Z 0 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2022-05-27T04:10:30.753241Z 0 [Note] Shutting down plugin 'sha256_password'
2022-05-27T04:10:30.753250Z 0 [Note] Shutting down plugin 'mysql_native_password'
2022-05-27T04:10:30.753258Z 0 [Note] Shutting down plugin 'wsrep'
2022-05-27T04:10:30.753593Z 0 [Note] Shutting down plugin 'binlog'
2022-05-27T04:10:30.754131Z 0 [Note] /usr/sbin/mysqld: Shutdown complete

2022-05-27T04:10:39.109767Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2022-05-27T04:10:39.111840Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.37-40-57-log) starting as process 996 ...
2022-05-27T04:10:39.156310Z 0 [Warning] No argument was provided to --log-bin, and --log-bin-index was not used; so replication may break when this MySQL server acts as a master and has his hostname changed!! Please use '--log-bin=localhost-bin' to avoid this problem.
2022-05-27T04:10:39.156703Z 0 [Note] WSREP: Setting wsrep_ready to false
2022-05-27T04:10:39.156725Z 0 [Note] WSREP: No pre-stored wsrep-start position found. Skipping position initialization.
2022-05-27T04:10:39.156745Z 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera3/libgalera_smm.so'
2022-05-27T04:10:39.181661Z 0 [Note] WSREP: wsrep_load(): Galera 3.57(rb2289d0) by Codership Oy <[info@codership.com](mailto:info@codership.com)> loaded successfully.
2022-05-27T04:10:39.181784Z 0 [Note] WSREP: CRC-32C: using 64-bit x86 acceleration.
2022-05-27T04:10:39.182571Z 0 [Note] WSREP: Found saved state: 165040b8-dbcf-11ec-895e-6b97581f251f:8, safe_to_bootstrap: 1
2022-05-27T04:10:39.201777Z 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 192.168.254.126; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 10; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 4; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.freeze_purge_at_seqno = -1; gcache.keep_pages_count = 0; gcache.keep_pages_size = 0; gcache.mem_size = 0; [gcache.name](http://gcache.name) = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 100; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.recovery = true; pc.version = 0; pc.wait_prim = true; pc.wait_prim_timeout = PT30S; pc.weight = 1; protonet.backend = asio; protonet.version = 0; repl.causal_read_timeout = PT30S; repl.commit_order = 3; repl.key_format = FLAT8; repl.max_ws_size = 2147483647; repl.proto_max = 9; socket.checksum = 2; socket.recv_buf_size = auto; socket.send_buf_size = auto;
2022-05-27T04:10:39.215811Z 0 [Note] WSREP: GCache history reset: 165040b8-dbcf-11ec-895e-6b97581f251f:0 -> 165040b8-dbcf-11ec-895e-6b97581f251f:8
2022-05-27T04:10:39.216315Z 0 [Note] WSREP: Assign initial position for certification: 8, protocol version: -1
2022-05-27T04:10:39.216342Z 0 [Note] WSREP: Preparing to initiate SST/IST
2022-05-27T04:10:39.216352Z 0 [Note] WSREP: Starting replication
2022-05-27T04:10:39.216372Z 0 [Note] WSREP: Setting initial position to 165040b8-dbcf-11ec-895e-6b97581f251f:8
2022-05-27T04:10:39.216603Z 0 [Note] WSREP: Using CRC-32C for message checksums.
2022-05-27T04:10:39.216727Z 0 [Note] WSREP: gcomm thread scheduling priority set to other:0
2022-05-27T04:10:39.216874Z 0 [Warning] WSREP: Fail to access the file (/var/lib/mysql//gvwstate.dat) error (No such file or directory). It is possible if node is booting for first time or re-booting after a graceful shutdown
2022-05-27T04:10:39.216892Z 0 [Note] WSREP: Restoring primary-component from disk failed. Either node is booting for first time or re-booting after a graceful shutdown
2022-05-27T04:10:39.217181Z 0 [Note] WSREP: GMCast version 0
2022-05-27T04:10:39.217510Z 0 [Note] WSREP: (f7a0e393, 'tcp://[0.0.0.0:4567](http://0.0.0.0:4567)') listening at tcp://[0.0.0.0:4567](http://0.0.0.0:4567)
2022-05-27T04:10:39.217532Z 0 [Note] WSREP: (f7a0e393, 'tcp://[0.0.0.0:4567](http://0.0.0.0:4567)') multicast: , ttl: 1
2022-05-27T04:10:39.217995Z 0 [Note] WSREP: EVS version 0
2022-05-27T04:10:39.218126Z 0 [Note] WSREP: gcomm: bootstrapping new group 'democluster'
2022-05-27T04:10:39.218173Z 0 [Note] WSREP: start_prim is enabled, turn off pc_recovery
2022-05-27T04:10:39.218925Z 0 [Note] WSREP: Node f7a0e393 state primary
2022-05-27T04:10:39.218976Z 0 [Note] WSREP: Current view of cluster as seen by this node
view (view_id(PRIM,f7a0e393,1)
memb {
f7a0e393,0
}
joined {
}
left {
}
partitioned {
}
)
2022-05-27T04:10:39.218997Z 0 [Note] WSREP: Save the discovered primary-component to disk
2022-05-27T04:10:39.224570Z 0 [Note] WSREP: discarding pending addr without UUID: tcp://[192.168.254.126:4567](http://192.168.254.126:4567)
2022-05-27T04:10:39.224599Z 0 [Note] WSREP: discarding pending addr proto entry 0x3a099e0
2022-05-27T04:10:39.224648Z 0 [Note] WSREP: discarding pending addr without UUID: tcp://[192.168.254.127:4567](http://192.168.254.127:4567)
2022-05-27T04:10:39.224665Z 0 [Note] WSREP: discarding pending addr proto entry 0x3a11f10
2022-05-27T04:10:39.224706Z 0 [Note] WSREP: gcomm: connected
2022-05-27T04:10:39.224797Z 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2022-05-27T04:10:39.224906Z 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1
2022-05-27T04:10:39.224999Z 0 [Note] WSREP: Waiting for SST/IST to complete.
2022-05-27T04:10:39.225213Z 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: f7a2205a-dd72-11ec-a520-6e50c1dd8219
2022-05-27T04:10:39.225354Z 0 [Note] WSREP: STATE EXCHANGE: sent state msg: f7a2205a-dd72-11ec-a520-6e50c1dd8219
2022-05-27T04:10:39.225375Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: f7a2205a-dd72-11ec-a520-6e50c1dd8219 from 0 (centosvm01)
2022-05-27T04:10:39.225389Z 0 [Note] WSREP: Quorum results:
version = 6,
component = PRIMARY,
conf_id = 0,
members = 1/1 (primary/total),
act_id = 8,
last_appl. = -1,
protocols = 0/9/3 (gcs/repl/appl),
group UUID = 165040b8-dbcf-11ec-895e-6b97581f251f
2022-05-27T04:10:39.225404Z 0 [Note] WSREP: Flow-control interval: [100, 100]
2022-05-27T04:10:39.225416Z 0 [Note] WSREP: Restored state OPEN -> JOINED (8)
2022-05-27T04:10:39.225455Z 0 [Note] WSREP: Member 0.0 (centosvm01) synced with group.
2022-05-27T04:10:39.225468Z 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 8)
2022-05-27T04:10:39.225501Z 2 [Note] WSREP: REPL Protocols: 9 (4, 2)
2022-05-27T04:10:39.225530Z 2 [Note] WSREP: REPL Protocols: 9 (4, 2)
2022-05-27T04:10:39.225551Z 2 [Note] WSREP: New cluster view: global state: 165040b8-dbcf-11ec-895e-6b97581f251f:8, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3
2022-05-27T04:10:39.225561Z 2 [Note] WSREP: Setting wsrep_ready to true
2022-05-27T04:10:39.225592Z 0 [Note] WSREP: SST complete, seqno: 8
2022-05-27T04:10:39.227449Z 0 [Note] InnoDB: PUNCH HOLE support available
2022-05-27T04:10:39.227477Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2022-05-27T04:10:39.227489Z 0 [Note] InnoDB: Uses event mutexes
2022-05-27T04:10:39.227499Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2022-05-27T04:10:39.227509Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.7
2022-05-27T04:10:39.227519Z 0 [Note] InnoDB: Using Linux native AIO
2022-05-27T04:10:39.227922Z 0 [Note] InnoDB: Number of pools: 1
2022-05-27T04:10:39.228099Z 0 [Note] InnoDB: Using CPU crc32 instructions
2022-05-27T04:10:39.230418Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2022-05-27T04:10:39.235909Z 0 [Note] InnoDB: Completed initialization of buffer pool
2022-05-27T04:10:39.238638Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2022-05-27T04:10:39.250298Z 0 [Note] InnoDB: Crash recovery did not find the parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite
2022-05-27T04:10:39.251903Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2022-05-27T04:10:39.300502Z 0 [Note] InnoDB: Created parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite, size 3932160 bytes
2022-05-27T04:10:39.343407Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2022-05-27T04:10:39.343498Z 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2022-05-27T04:10:39.431179Z 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2022-05-27T04:10:39.432251Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2022-05-27T04:10:39.432286Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2022-05-27T04:10:39.433098Z 0 [Note] InnoDB: Waiting for purge to start
2022-05-27T04:10:39.483302Z 0 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.7.37-40 started; log sequence number 2758228
2022-05-27T04:10:39.483808Z 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2022-05-27T04:10:39.484358Z 0 [Note] Plugin 'FEDERATED' is disabled.
2022-05-27T04:10:39.486757Z 0 [Note] InnoDB: Buffer pool(s) load completed at 220527 11:10:39
2022-05-27T04:10:39.570709Z 0 [Note] Found ca.pem, server-cert.pem and server-key.pem in data directory. Trying to enable SSL support using them.
2022-05-27T04:10:39.570742Z 0 [Note] Skipping generation of SSL certificates as certificate files are present in data directory.
2022-05-27T04:10:39.570755Z 0 [Warning] A deprecated TLS version TLSv1 is enabled. Please use TLSv1.2 or higher.
2022-05-27T04:10:39.570763Z 0 [Warning] A deprecated TLS version TLSv1.1 is enabled. Please use TLSv1.2 or higher.
2022-05-27T04:10:39.571463Z 0 [Warning] CA certificate ca.pem is self signed.
2022-05-27T04:10:39.571536Z 0 [Note] Skipping generation of RSA key pair as key files are present in data directory.
2022-05-27T04:10:39.571685Z 0 [Note] Server hostname (bind-address): '*'; port: 3306
2022-05-27T04:10:39.571730Z 0 [Note] IPv6 is available.
2022-05-27T04:10:39.571751Z 0 [Note] - '::' resolves to '::';
2022-05-27T04:10:39.571779Z 0 [Note] Server socket created on IP: '::'.
2022-05-27T04:10:39.594385Z 0 [Note] Failed to start slave threads for channel ''
2022-05-27T04:10:39.604293Z 0 [Note] Event Scheduler: Loaded 0 events
2022-05-27T04:10:39.605845Z 0 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.7.37-40-57-log' socket: '/var/lib/mysql/mysql.sock' port: 3306 Percona XtraDB Cluster (GPL), Release rel40, Revision ce3b993, WSREP version 31.57, wsrep_31.57
2022-05-27T04:10:39.605845Z 2 [Note] WSREP: Initialized wsrep sidno 2
2022-05-27T04:10:39.605898Z 2 [Note] WSREP: Auto Increment Offset/Increment re-align with cluster membership change (Offset: 1 -> 1) (Increment: 1 -> 1)
2022-05-27T04:10:39.605916Z 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2022-05-27T04:10:39.605949Z 2 [Note] WSREP: Assign initial position for certification: 8, protocol version: 4
2022-05-27T04:10:39.606011Z 0 [Note] WSREP: Service thread queue flushed.
2022-05-27T04:10:39.606064Z 2 [Note] WSREP: GCache history reset: 165040b8-dbcf-11ec-895e-6b97581f251f:0 -> 165040b8-dbcf-11ec-895e-6b97581f251f:8
2022-05-27T04:10:39.620663Z 2 [Note] WSREP: Synchronized with group, ready for connections
2022-05-27T04:10:39.620692Z 2 [Note] WSREP: Setting wsrep_ready to true
2022-05-27T04:10:39.620707Z 2 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
1 Like
13

Just curious can you check the ip address on both nodes?

1 Like
14

@hieu_nguyen,
In your log above of node1, I see no attempts from node2 to connect to node1. Under normal circumstances, there are many log entries each time a new node tries to connect to an existing member. You have some other issue with your networking, or IP address, that is preventing node2 from talking to node1.

1 Like