I am setting up a new cluster that is IPv6 only. I have configured the wsrep.cnf file with the correct values. I have compared this file to another cluster that is IPv4 only and see no glaring differences. Is there a syntax change for formatting?
Here is my config.
[root@dbcl01-01 mysql]# more /etc/percona-xtradb-cluster.conf.d/wsrep.cnf [mysqld]wsrep_provider=/usr/lib64/galera3/libgalera_smm.sowsrep_cluster_address=gcomm://[2620:84:a001:6::3a],[2620:84:a001:6::3b],[2620:84:a001:6::3c]
binlog_format=ROW
default_storage_engine=InnoDBwsrep_slave_threads= 8
wsrep_log_conflictsinnodb_autoinc_lock_mode=2wsrep_node_address=[2620:84:a001:6::3a]
wsrep_cluster_name=prod-clusterwsrep_node_name=dbcl01-01pxc_strict_mode=ENFORCING
wsrep_sst_method=xtrabackup-v2
wsrep_sst_auth=“sstuser:XXXXXXXXXXX”
Is there something that I have misconfigured? MySQL will not start when this config is enabled. Remove it and it starts fine.
The following is the tail of the mysql.log:
2020-09-15T00:55:34.195972Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).2020-09-15T00:55:34.197616Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.30-33-57-log) starting as process 28024 …2020-09-15T00:55:34.200581Z 0 [Note] WSREP: Setting wsrep_ready to false2020-09-15T00:55:34.200603Z 0 [Note] WSREP: No pre-stored wsrep-start position found. Skipping position initialization.2020-09-15T00:55:34.200613Z 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib64/galera3/libgalera_smm.so’2020-09-15T00:55:34.203240Z 0 [Note] WSREP: wsrep_load(): Galera 3.43(ra60e019) by Codership Oy <info@codership.com> loaded successfully.2020-09-15T00:55:34.203333Z 0 [Note] WSREP: CRC-32C: using hardware acceleration.2020-09-15T00:55:34.203743Z 0 [Note] WSREP: Found saved state: a5797da6-f6cb-11ea-aa51-860d555bb2d6:3, safe_to_bootstrap: 12020-09-15T00:55:34.205198Z 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = [2620:84:a001:6::3a]; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 10; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 4; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.freeze_purge_at_seqno = -1; gcache.keep_pages_count = 0; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1; gcs.fc_limit = 100; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum = false; pc.ignore_sb = false; pc.npvo = false; pc.recovery = true; pc.version = 0; pc.wait_prim = true; pc.wait_prim_timeout = PT30S; pc.weight = 1; protonet.backend = asio; protonet.version = 0; repl.causal_read_timeout = PT30S; repl.commit_order = 3; repl.key_format = FLAT8; repl.max_ws_size = 2147483647; repl.proto_max = 9; socket.checksum = 2; socket.recv_buf_size = auto; socket.send_buf_size = auto; 2020-09-15T00:55:34.216611Z 0 [Note] WSREP: GCache history reset: a5797da6-f6cb-11ea-aa51-860d555bb2d6:0 -> a5797da6-f6cb-11ea-aa51-860d555bb2d6:32020-09-15T00:55:34.226551Z 0 [Note] WSREP: Assign initial position for certification: 3, protocol version: -12020-09-15T00:55:34.226585Z 0 [Note] WSREP: Preparing to initiate SST/IST2020-09-15T00:55:34.226593Z 0 [Note] WSREP: Starting replication2020-09-15T00:55:34.226608Z 0 [Note] WSREP: Setting initial position to a5797da6-f6cb-11ea-aa51-860d555bb2d6:32020-09-15T00:55:34.226849Z 0 [Note] WSREP: Using CRC-32C for message checksums.2020-09-15T00:55:34.226982Z 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 2020-09-15T00:55:34.227115Z 0 [Warning] WSREP: Fail to access the file (/var/lib/mysql//gvwstate.dat) error (No such file or directory). It is possible if node is booting for first time or re-booting after a graceful shutdown2020-09-15T00:55:34.227137Z 0 [Note] WSREP: Restoring primary-component from disk failed. Either node is booting for first time or re-booting after a graceful shutdown2020-09-15T00:55:34.227490Z 0 [Note] WSREP: GMCast version 02020-09-15T00:55:34.227823Z 0 [Note] WSREP: (2935f100, ‘tcp://0.0.0.0:4567’) listening at tcp://0.0.0.0:45672020-09-15T00:55:34.227835Z 0 [Note] WSREP: (2935f100, ‘tcp://0.0.0.0:4567’) multicast: , ttl: 12020-09-15T00:55:34.228302Z 0 [Note] WSREP: EVS version 02020-09-15T00:55:34.228444Z 0 [Note] WSREP: gcomm: connecting to group ‘prod-cluster’, peer '[2620:84:a001:6::3a]:,[2620:84:a001:6::3b]:,[2620:84:a001:6::3c]:'2020-09-15T00:55:37.232184Z 0 [Note] WSREP: announce period timed out (pc.announce_timeout)2020-09-15T00:55:37.232623Z 0 [Warning] WSREP: no nodes coming from prim view, prim not possible2020-09-15T00:55:37.232682Z 0 [Note] WSREP: Current view of cluster as seen by this nodeview (view_id(NON_PRIM,2935f100,1)memb { 2935f100,0 }joined { }left { }partitioned { })2020-09-15T00:55:37.733392Z 0 [Warning] WSREP: last inactive check more than PT1.5S (3*evs.inactive_check_period) ago (PT3.50511S), skipping check2020-09-15T00:56:07.253182Z 0 [Note] WSREP: Current view of cluster as seen by this nodeview ((empty))2020-09-15T00:56:07.253369Z 0 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view (pc.wait_prim_timeout): 110 (Connection timed out) at gcomm/src/pc.cpp:connect():1612020-09-15T00:56:07.253409Z 0 [ERROR] WSREP: gcs/src/gcs_core.cpp:gcs_core_open():209: Failed to open backend connection: -110 (Connection timed out)2020-09-15T00:56:07.253522Z 0 [ERROR] WSREP: gcs/src/gcs.cpp:gcs_open():1514: Failed to open channel ‘prod-cluster’ at ‘gcomm://[2620:84:a001:6::3a],[2620:84:a001:6::3b],[2620:84:a001:6::3c]’: -110 (Connection timed out)2020-09-15T00:56:07.253579Z 0 [ERROR] WSREP: gcs connect failed: Connection timed out2020-09-15T00:56:07.253603Z 0 [ERROR] WSREP: Provider/Node (gcomm://[2620:84:a001:6::3a],[2620:84:a001:6::3b],[2620:84:a001:6::3c]) failed to establish connection with cluster (reason: 7)2020-09-15T00:56:07.253611Z 0 [ERROR] Aborting
2020-09-15T00:56:07.253620Z 0 [Note] Giving 0 client threads a chance to die gracefully2020-09-15T00:56:07.253670Z 0 [Note] WSREP: Waiting for active wsrep applier to exit2020-09-15T00:56:07.253792Z 0 [Note] WSREP: Service disconnected.2020-09-15T00:56:07.253901Z 0 [Note] WSREP: Waiting to close threads…2020-09-15T00:56:12.254122Z 0 [Note] WSREP: Some threads may fail to exit.2020-09-15T00:56:12.254249Z 0 [Note] Binlog end2020-09-15T00:56:12.254588Z 0 [Note] /usr/sbin/mysqld: Shutdown complete
After further testing, I found that if the following is changed, MySQL will start.
Does not work: wsrep_cluster_address=gcomm://[2620:84:a001:6::3a]
Works: wsrep_cluster_address=gcomm://
@smalenshek When you set wsrep_cluster_address=gcomm://, you are telling this node to bootstrap and thus form a new, independent cluster. This should only be done on the first node when bootstrapping the entire cluster. As an alternative, you could have left the parameter as you had it and use systemctl start mysql@boostrap which would have started a new cluster without changing the my.cnf file. Then, for your remaining nodes, simply do systemctl start mysql and they will join.