Node 1 : /var/log/mysqld.err ?============================================================================================ 2018-12-13T02:32:46.193962Z 2 [Note] WSREP: start_prim is enabled, turn off pc_recovery? 2018-12-13T02:32:46.195002Z 2 [Note] WSREP: Node 60218b50 state primary? 2018-12-13T02:32:46.195085Z 2 [Note] WSREP: Current view of cluster as seen by this node? view (view_id(PRIM,60218b50,1)? memb {? 60218b50,0? }? joined {? }? left {? }? partitioned {? }? )? 2018-12-13T02:32:46.195107Z 2 [Note] WSREP: Save the discovered primary-component to disk? 2018-12-13T02:32:46.200579Z 2 [Note] WSREP: gcomm: connected? 2018-12-13T02:32:46.200740Z 2 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)? 2018-12-13T02:32:46.200859Z 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 1? 2018-12-13T02:32:46.203549Z 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 60233e32-fe7f-11e8-a912-83622174bbd6? 2018-12-13T02:32:46.203595Z 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 60233e32-fe7f-11e8-a912-83622174bbd6? 2018-12-13T02:32:46.203610Z 0 [Note] WSREP: STATE EXCHANGE: got state msg: 60233e32-fe7f-11e8-a912-83622174bbd6 from 0 (devlsql03)? 2018-12-13T02:32:46.203628Z 0 [Note] WSREP: Quorum results:? version = 4,? component = PRIMARY,? conf_id = 0,? members = 1/1 (primary/total),? act_id = 58,? last_appl. = -1,? protocols = 0/9/3 (gcs/repl/appl),? group UUID = b9455b6d-fa00-11e8-8a11-f751cb54f21b? 2018-12-13T02:32:46.203644Z 0 [Note] WSREP: Flow-control interval: [100, 100]? 2018-12-13T02:32:46.203652Z 0 [Note] WSREP: Trying to continue unpaused monitor? 2018-12-13T02:32:46.203662Z 0 [Note] WSREP: Restored state OPEN -> JOINED (58)? 2018-12-13T02:32:46.203712Z 0 [Note] WSREP: Member 0.0 (devlsql03) synced with group.? 2018-12-13T02:32:46.203725Z 0 [Note] WSREP: Shifting JOINED -> SYNCED (TO: 58)? 2018-12-13T02:32:46.203817Z 4 [Note] WSREP: REPL Protocols: 9 (4, 2)? 2018-12-13T02:32:46.203870Z 4 [Note] WSREP: New cluster view: global state: b9455b6d-fa00-11e8-8a11-f751cb54f21b:58, view# 1: Primary, number of nodes: 1, my index: 0, protocol version 3? 2018-12-13T02:32:46.203883Z 4 [Note] WSREP: Setting wsrep_ready to true? 2018-12-13T02:32:46.222192Z 4 [Note] WSREP: Initialized wsrep sidno 2? 2018-12-13T02:32:46.222236Z 4 [Note] WSREP: Auto Increment Offset/Increment re-align with cluster membership change (Offset: 1 -> 1) (Increment: 1 -> 1)? 2018-12-13T02:32:46.222256Z 4 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.? 2018-12-13T02:32:46.222322Z 4 [Note] WSREP: Assign initial position for certification: 58, protocol version: 4? 2018-12-13T02:32:46.222505Z 0 [Note] WSREP: Service thread queue flushed.? 2018-12-13T02:32:46.222606Z 4 [Note] WSREP: GCache history reset: b9455b6d-fa00-11e8-8a11-f751cb54f21b:0 -> b9455b6d-fa00-11e8-8a11-f751cb54f21b:58? 2018-12-13T02:32:46.226061Z 4 [Note] WSREP: Synchronized with group, ready for connections? 2018-12-13T02:32:46.226091Z 4 [Note] WSREP: Setting wsrep_ready to true? 2018-12-13T02:32:46.226108Z 4 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.? ? ? ============================================================================================ Node 1 : my.cnf ?============================================================================================ -bash-4.1$ cat /etc/my.cnf? [mysqld]? server_id=1? binlog_format=ROW? pid-file=/var/run/mysqld/mysqld.pid? log-error=/var/log/mysqld.err? log_bin=mysql-bin? datadir=/var/lib/mysql? socket=/var/lib/mysql/mysql.sock? user=mysql? explicit_defaults_for_timestamp = 1? # Disabling symbolic-links is recommended to prevent assorted security risks? symbolic-links=0? ? [mysqld_safe]? log-error=/var/log/mysqld.err? pid-file=/var/run/mysqld/mysqld.pid? wsrep_on=ON? wsrep_provider=/usr/lib64/galera3/libgalera_smm.so? wsrep_provider_options="gcache.size=1G"? #wsrep_cluster_address=gcomm://172.18.23.12,172.18.23.13? wsrep_cluster_address=gcomm://? ? default_storage_engine=InnoDB? wsrep_slave_threads=4? wsrep_cluster_name=frameworkdb? wsrep_node_name=devlsql03? ? log_slave_updates=1? innodb_locks_unsafe_for_binlog=1? innodb_autoinc_lock_mode=2? innodb_doublewrite=1? innodb_log_files_in_group=2? innodb_buffer_pool_size=800M? innodb_log_file_size=64M? table_open_cache = 500? query_cache_size=0? ? # Node 1 address? wsrep_node_address=172.18.23.12? wsrep_node_incoming_address=172.18.23.12? ? # SST method? wsrep_sst_method=rsync? #wsrep_sst_method=xtrabackup-v2? #wsrep_slave_threads=4? wsrep_sst_auth=sstuser:sstpass? wsrep_sst_receive_address=172.18.23.12? ? # Cluster name? wsrep_cluster_name=frameworkdb? bind-address=0.0.0.0? ? pxc_strict_mode=ENFORCING? explicit_defaults_for_timestamp=1? wsrep_log_conflicts=ON? wsrep_debug=ON? wait_timeout=3300? interactive_timeout=3300? net_read_timeout=3300? net_write_timeout=3300? connection_timeout=3300? max-connections=2000? ? ============================================================================================ Node 2 : /var/log/mysqld.err ?============================================================================================ ? 2018-12-13T03:04:25.438399Z 7 [ERROR] WSREP: gcs connect failed: Connection timed out? 2018-12-13T03:04:25.438421Z 7 [ERROR] WSREP: Provider/Node (gcomm://172.18.23.12:4567) failed to establish connection with cluster (reason: 7)? 2018-12-13T03:07:30.277424Z 10 [Note] WSREP: Stop replication? 2018-12-13T03:07:30.277495Z 10 [Note] Giving 1 client threads a chance to die gracefully? 2018-12-13T03:07:32.277659Z 10 [Note] WSREP: Waiting for active wsrep applier to exit? 2018-12-13T03:07:32.277739Z 10 [Note] WSREP: Starting replication? 2018-12-13T03:07:32.277815Z 10 [Note] WSREP: Setting initial position to 6b530d8f-fc21-11e8-8dbf-921ed01a88d2:2? 2018-12-13T03:07:32.278341Z 10 [Note] WSREP: Using CRC-32C for message checksums.? 2018-12-13T03:07:32.278650Z 10 [Note] WSREP: gcomm thread scheduling priority set to other:0? 2018-12-13T03:07:32.278846Z 10 [Warning] WSREP: Fail to access the file (/var/lib/mysql//gvwstate.dat) error (No such file or directory). It is possible if node is booting for first time or re-booting after a graceful shutdown? 2018-12-13T03:07:32.278868Z 10 [Note] WSREP: Restoring primary-component from disk failed. Either node is booting for first time or re-booting after a graceful shutdown? 2018-12-13T03:07:32.280419Z 10 [Note] WSREP: GMCast version 0? 2018-12-13T03:07:32.280753Z 10 [Note] WSREP: (3b894146, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567? 2018-12-13T03:07:32.280777Z 10 [Note] WSREP: (3b894146, 'tcp://0.0.0.0:4567') multicast: , ttl: 1? 2018-12-13T03:07:32.281461Z 10 [Note] WSREP: EVS version 0? 2018-12-13T03:07:32.281656Z 10 [Note] WSREP: gcomm: connecting to group 'my_wsrep_cluster', peer '172.18.23.12:,172.18.23.13:'? 2018-12-13T03:07:32.284353Z 10 [Note] WSREP: (3b894146, 'tcp://0.0.0.0:4567') connection established to 3b894146 tcp://172.18.23.13:4567? 2018-12-13T03:07:32.284416Z 10 [Warning] WSREP: (3b894146, 'tcp://0.0.0.0:4567') address 'tcp://172.18.23.13:4567' points to own listening address, blacklisting? 2018-12-13T03:07:35.284498Z 10 [Note] WSREP: announce period timed out (pc.announce_timeout)? 2018-12-13T03:07:35.284948Z 10 [Warning] WSREP: no nodes coming from prim view, prim not possible? 2018-12-13T03:07:35.285513Z 10 [Note] WSREP: Current view of cluster as seen by this node? view (view_id(NON_PRIM,3b894146,1)? memb {? 3b894146,0? }? joined {? }? left {? }? partitioned {? }? )? 2018-12-13T03:07:35.785695Z 10 [Warning] WSREP: last inactive check more than PT1.5S (3*evs.inactive_check_period) ago (PT3.50425S), skipping check? 2018-12-13T03:08:05.296190Z 10 [Note] WSREP: Current view of cluster as seen by this node? view ((empty))? 2018-12-13T03:08:05.296529Z 10 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view (pc.wait_prim_timeout): 110 (Connection timed out)? at gcomm/src/pc.cpp:connect():159? 2018-12-13T03:08:05.296564Z 10 [ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():209: Failed to open backend connection: -110 (Connection timed out)? 2018-12-13T03:08:05.296666Z 10 [ERROR] WSREP: gcs/src/gcs.cpp:gcs_open():1514: Failed to open channel 'my_wsrep_cluster' at 'gcomm://172.18.23.12,172.18.23.13': -110 (Connection timed out)? 2018-12-13T03:08:05.296730Z 10 [ERROR] WSREP: gcs connect failed: Connection timed out? 2018-12-13T03:08:05.296743Z 10 [ERROR] WSREP: Provider/Node (gcomm://172.18.23.12,172.18.23.13) failed to establish connection with cluster (reason: 7)? ? ============================================================================================ Node 2 : my.cnf ?============================================================================================ [root@devlsql04 ~]# cat /etc/my.cnf? [mysqld]? server_id=2? binlog_format=ROW? pid-file=/var/run/mysqld/mysqld.pid? log-error=/var/log/mysqld.err? log_bin=mysql-bin? datadir=/var/lib/mysql? socket=/var/lib/mysql/mysql.sock? user=mysql? explicit_defaults_for_timestamp = 1? # Disabling symbolic-links is recommended to prevent assorted security risks? symbolic-links=0? ? [mysqld_safe]? log-error=/var/log/mysqld.err? pid-file=/var/run/mysqld/mysqld.pid? wsrep_on=ON? wsrep_provider=/usr/lib64/galera3/libgalera_smm.so? wsrep_provider_options="gcache.size=1G"? #wsrep_cluster_address=gcomm://172.18.23.12,172.18.23.13? wsrep_cluster_address=gcomm://172.18.23.12? ? default_storage_engine=InnoDB? wsrep_slave_threads=4? wsrep_cluster_name=frameworkdb? wsrep_node_name=devlsql04? ? log_slave_updates=1? innodb_locks_unsafe_for_binlog=1? innodb_autoinc_lock_mode=2? innodb_doublewrite=1? innodb_log_files_in_group=2? innodb_buffer_pool_size=800M? innodb_log_file_size=64M? table_open_cache = 500? query_cache_size=0? ? # Node 2 address? #wsrep_node_name=devlsql04? wsrep_node_address=172.18.23.13? wsrep_node_incoming_address=172.18.23.13? ? # SST method? wsrep_sst_method=rsync? #wsrep_sst_method=xtrabackup-v2? #wsrep_slave_threads=4? wsrep_sst_auth=sstuser:sstpass? wsrep_sst_receive_address=172.18.23.13? ? # Cluster name? #wsrep_cluster_name=frameworkdb? bind-address=0.0.0.0? ? pxc_strict_mode=ENFORCING? explicit_defaults_for_timestamp=1? wsrep_log_conflicts=ON? wsrep_debug=ON? wait_timeout=3300? interactive_timeout=3300? net_read_timeout=3300? net_write_timeout=3300? connection_timeout=3300? max-connections=2000? ?