Not the answer you need? Try our search, browse our categories, or register and ask your own question!

One node crash/hang, other node become non-primary

Hello,

I have a running cluster with 3 nodes. But if one node hang/crash the other node become non-primary and the cluster being failed. This case not happen were gracefully shut down.

This is the chronology crash :
- I'm losing one node A because of hardware/vm hang. 
- The other node B and node C become non-primary, and the cluster failed or cannot used.
- The cluster become normal again after the hanging node A restart and the MySQL service on that node started to join the cluster.

Node A : cluster-node1 = 10.30.1.43
Node B: cluster-node2  = 10.30.1.44
Node C: cluster-node3 = 10.30.1.30

Node B
2020-06-24T05:55:56.748727+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 1ceb3dad with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:55:56.753604+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://10.30.1.43:4567 
2020-06-24T05:55:57.754426+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 0
2020-06-24T05:56:00.771077+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:05.272699+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:08.691017+07:00 0 [Note] WSREP: evs::proto(2f81a4e3, GATHER, view_id(REG,1ceb3dad,620)) suspecting node: 1ceb3dad
2020-06-24T05:56:08.691668+07:00 0 [Note] WSREP: evs::proto(2f81a4e3, GATHER, view_id(REG,1ceb3dad,620)) suspected node without join message, declaring inactive
2020-06-24T05:56:09.692206+07:00 0 [Note] WSREP: view(view_id(NON_PRIM,1ceb3dad,620) memb {
	2f81a4e3,0
} joined {
} left {
} partitioned {
	1ceb3dad,0
	b48606bd,0
})
2020-06-24T05:56:09.693357+07:00 0 [Note] WSREP: view(view_id(NON_PRIM,2f81a4e3,621) memb {
	2f81a4e3,0
} joined {
} left {
} partitioned {
	1ceb3dad,0
	b48606bd,0
})
2020-06-24T05:56:09.694661+07:00 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2020-06-24T05:56:09.695104+07:00 0 [Note] WSREP: Flow-control interval: [256, 256]
2020-06-24T05:56:09.695240+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T05:56:09.695259+07:00 0 [Note] WSREP: Received NON-PRIMARY.
2020-06-24T05:56:09.695275+07:00 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 9582329)
2020-06-24T05:56:09.695356+07:00 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2020-06-24T05:56:09.695383+07:00 0 [Note] WSREP: Flow-control interval: [256, 256]
2020-06-24T05:56:09.695393+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T05:56:09.695401+07:00 0 [Note] WSREP: Received NON-PRIMARY.
2020-06-24T05:56:09.771723+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3
2020-06-24T05:56:09.771854+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T05:56:09.771980+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3
2020-06-24T05:56:09.771995+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T05:56:09.781823+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:10.696065+07:00 0 [Note] WSREP: declaring b48606bd at tcp://10.30.1.30:4567 stable
2020-06-24T05:56:10.697728+07:00 0 [Note] WSREP: view(view_id(NON_PRIM,2f81a4e3,622) memb {
	2f81a4e3,0
	b48606bd,0
} joined {
} left {
} partitioned {
	1ceb3dad,0
})
2020-06-24T05:56:10.697920+07:00 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 2
2020-06-24T05:56:10.697989+07:00 0 [Note] WSREP: Flow-control interval: [362, 362]
2020-06-24T05:56:10.698008+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T05:56:10.698018+07:00 0 [Note] WSREP: Received NON-PRIMARY.
2020-06-24T05:56:10.698139+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# -1: non-Primary, number of nodes: 2, my index: 0, protocol version 3
2020-06-24T05:56:10.698195+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T05:56:14.283533+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:18.786704+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:23.288202+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:27.291676+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:57:38.817692+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 30
2020-06-24T05:59:08.844661+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 60
2020-06-24T06:00:38.883278+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 90
2020-06-24T06:02:08.910682+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 120
2020-06-24T06:03:38.951557+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 150
2020-06-24T06:05:09.003140+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 180
2020-06-24T06:06:39.055554+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 210
2020-06-24T06:08:09.100981+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 240
2020-06-24T06:09:39.160631+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 270
2020-06-24T06:11:09.218993+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 300
2020-06-24T06:12:39.247609+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 330
2020-06-24T06:14:09.275575+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 360
2020-06-24T06:15:38.809586+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 390
2020-06-24T06:17:08.838642+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 420
2020-06-24T06:18:38.875385+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 450
2020-06-24T06:20:08.907983+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 480
2020-06-24T06:21:38.937346+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 510
2020-06-24T06:23:08.968237+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 540
2020-06-24T06:24:39.004672+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 570
2020-06-24T06:26:09.041725+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 600
2020-06-24T06:27:39.075586+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 630
2020-06-24T06:29:09.113035+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 660
2020-06-24T06:30:39.148766+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 690
2020-06-24T06:32:09.174982+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 720
2020-06-24T06:33:39.203873+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 750
2020-06-24T06:35:09.236552+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 780
2020-06-24T06:36:39.273287+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 810
2020-06-24T06:38:09.303512+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 840
2020-06-24T06:39:38.836719+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 870
2020-06-24T06:41:08.873322+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 900
2020-06-24T06:42:38.906965+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 930
2020-06-24T06:44:08.936437+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 960
2020-06-24T06:45:38.968792+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 990
2020-06-24T06:47:08.996032+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1020
2020-06-24T06:48:39.029468+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1050
2020-06-24T06:50:09.077618+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1080
2020-06-24T06:51:39.109591+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1110
2020-06-24T06:53:09.138939+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1140
2020-06-24T06:54:39.160600+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1170
2020-06-24T06:56:09.187998+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1200
2020-06-24T06:57:39.214927+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1230
2020-06-24T06:59:09.242261+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1260
2020-06-24T07:00:39.271220+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1290
2020-06-24T07:02:09.295768+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1320
2020-06-24T07:03:38.823272+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1350
2020-06-24T07:05:08.862910+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1380
2020-06-24T07:06:38.910103+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1410
2020-06-24T07:08:08.979765+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1440
2020-06-24T07:09:39.021933+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1470
2020-06-24T07:11:09.049762+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1500
2020-06-24T07:12:39.082209+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1530
2020-06-24T07:14:09.113021+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1560
2020-06-24T07:15:39.138512+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1590
2020-06-24T07:17:09.161037+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1620
2020-06-24T07:18:39.190899+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1650
2020-06-24T07:20:09.217369+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1680
2020-06-24T07:21:39.250440+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1710
2020-06-24T07:23:09.277725+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1740
2020-06-24T07:24:39.309504+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1770
2020-06-24T07:26:08.837326+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1800
2020-06-24T07:27:38.863784+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1830
2020-06-24T07:29:08.891861+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1860
2020-06-24T07:30:38.932764+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1890
2020-06-24T07:32:08.961812+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1920
2020-06-24T07:33:38.992976+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1950
2020-06-24T07:35:09.019870+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1980
2020-06-24T07:36:39.052431+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2010
2020-06-24T07:38:09.086935+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2040
2020-06-24T07:39:39.120607+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2070
2020-06-24T07:41:09.146785+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2100
2020-06-24T07:42:39.175952+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2130
2020-06-24T07:44:09.207314+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2160
2020-06-24T07:45:39.246078+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2190
2020-06-24T07:47:09.276147+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2220
2020-06-24T07:48:39.308084+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2250
2020-06-24T07:50:08.833436+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2280
2020-06-24T07:51:38.864809+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2310
2020-06-24T07:53:08.895314+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2340
2020-06-24T07:54:38.924018+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2370
2020-06-24T07:56:08.951747+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2400
2020-06-24T07:57:38.982124+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2430
2020-06-24T07:59:09.011313+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2460
2020-06-24T08:00:39.040498+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2490
2020-06-24T08:02:09.070890+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2520
2020-06-24T08:03:39.097539+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2550
2020-06-24T08:05:09.122272+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2580
2020-06-24T08:06:39.147057+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2610
2020-06-24T08:08:09.177213+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2640
2020-06-24T08:09:39.209591+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2670
2020-06-24T08:11:09.235046+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2700
2020-06-24T08:12:39.266450+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2730
2020-06-24T08:14:09.299597+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2760
2020-06-24T08:15:39.335109+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2790
2020-06-24T08:17:08.866290+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2820
2020-06-24T08:18:38.898406+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2850
2020-06-24T08:20:08.931198+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2880
2020-06-24T08:21:38.979701+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2910
2020-06-24T08:23:09.012747+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2940
2020-06-24T08:24:39.045387+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2970
2020-06-24T08:26:09.077435+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3000
2020-06-24T08:27:39.118057+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3030
2020-06-24T08:29:09.150750+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3060
2020-06-24T08:30:39.178534+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3090
2020-06-24T08:32:09.205511+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3120
2020-06-24T08:33:39.238279+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3150
2020-06-24T08:35:09.267332+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3180
2020-06-24T08:36:39.309557+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3210
2020-06-24T08:38:09.338580+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3240
2020-06-24T08:39:38.867370+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3270
2020-06-24T08:41:08.895040+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3300
2020-06-24T08:42:38.921690+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3330
2020-06-24T08:44:08.956410+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3360
2020-06-24T08:45:38.985736+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3390
2020-06-24T08:47:09.019994+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3420
2020-06-24T08:48:39.052765+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3450
2020-06-24T08:50:09.086833+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3480
2020-06-24T08:51:39.115205+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3510
2020-06-24T08:53:09.142277+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3540
2020-06-24T08:54:39.173699+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3570
2020-06-24T08:56:09.200240+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3600
2020-06-24T08:57:39.226815+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3630
2020-06-24T08:59:09.251952+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3660
2020-06-24T09:00:39.282128+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3690
2020-06-24T09:02:09.310223+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3720
2020-06-24T09:03:39.334642+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3750
2020-06-24T09:05:09.367011+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3780
2020-06-24T09:06:38.897112+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3810
2020-06-24T09:08:08.930110+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3840
2020-06-24T09:09:38.962121+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3870
2020-06-24T09:11:09.004927+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3900
2020-06-24T09:12:39.036694+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3930
2020-06-24T09:14:09.070101+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3960
2020-06-24T09:15:39.098808+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3990
2020-06-24T09:17:09.125590+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4020
2020-06-24T09:18:38.155955+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4050
2020-06-24T09:19:23.172451+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4080
2020-06-24T09:20:07.192763+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4110
2020-06-24T09:20:51.209724+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4140
2020-06-24T09:21:29.840562+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection established to 1ceb3dad tcp://10.30.1.43:4567
2020-06-24T09:21:30.341239+07:00 0 [Note] WSREP: declaring 1ceb3dad at tcp://10.30.1.43:4567 stable
2020-06-24T09:21:30.341323+07:00 0 [Note] WSREP: declaring b48606bd at tcp://10.30.1.30:4567 stable
2020-06-24T09:21:30.342225+07:00 0 [Note] WSREP: re-bootstrapping prim from partitioned components
2020-06-24T09:21:30.343045+07:00 0 [Note] WSREP: view(view_id(PRIM,1ceb3dad,623) memb {
	1ceb3dad,0
	2f81a4e3,0
	b48606bd,0
} joined {
} left {
} partitioned {
})
2020-06-24T09:21:30.343126+07:00 0 [Note] WSREP: save pc into disk
2020-06-24T09:21:30.443527+07:00 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3
2020-06-24T09:21:30.443632+07:00 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
2020-06-24T09:21:30.841290+07:00 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
2020-06-24T09:21:30.841927+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 0 (cluster-node1)
2020-06-24T09:21:30.842010+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 1 (cluster-node2)
2020-06-24T09:21:30.842070+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 2 (cluster57-node3)
2020-06-24T09:21:30.842174+07:00 0 [Warning] WSREP: Quorum: No node with complete state:

	Version      : 4
	Flags        : 0x1
	Protocols    : 0 / 9 / 3
	State        : NON-PRIMARY
	Desync count : 0
	Prim state   : NON-PRIMARY
	Prim UUID    : 00000000-0000-0000-0000-000000000000
	Prim  seqno  : -1
	First seqno  : -1
	Last  seqno  : 9582329
	Prim JOINED  : 0
	State UUID   : 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
	Group UUID   : 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
	Name         : 'cluster-node1'
	Incoming addr: '10.30.1.43:3306'

	Version      : 4
	Flags        : 0x2
	Protocols    : 0 / 9 / 3
	State        : NON-PRIMARY
	Desync count : 0
	Prim state   : SYNCED
	Prim UUID    : 1d3833af-ad80-11ea-b3bf-baf19666e63c
	Prim  seqno  : 10
	First seqno  : 9528528
	Last  seqno  : 9582329
	Prim JOINED  : 3
	State UUID   : 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
	Group UUID   : 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
	Name         : 'cluster-node2'
	Incoming addr: '10.30.1.44:3306'

	Version      : 4
	Flags        : 0x2
	Protocols    : 0 / 9 / 3
	State        : NON-PRIMARY
	Desync count : 0
	Prim state   : SYNCED
	Prim UUID    : 1d3833af-ad80-11ea-b3bf-baf19666e63c
	Prim  seqno  : 10
	First seqno  : 9544324
	Last  seqno  : 9582329
	Prim JOINED  : 3
	State UUID   : 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
	Group UUID   : 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
	Name         : 'cluster-node3'
	Incoming addr: '10.30.1.30:3306'

2020-06-24T09:21:30.842198+07:00 0 [Note] WSREP: Partial re-merge of primary 1d3833af-ad80-11ea-b3bf-baf19666e63c found: 2 of 3.
2020-06-24T09:21:30.842214+07:00 0 [Note] WSREP: Quorum results:
	version    = 4,
	component  = PRIMARY,
	conf_id    = 10,
	members    = 3/3 (joined/total),
	act_id     = 9582329,
	last_appl. = 9582287,
	protocols  = 0/9/3 (gcs/repl/appl),
	group UUID = 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
2020-06-24T09:21:30.842281+07:00 0 [Note] WSREP: Flow-control interval: [443, 443]
2020-06-24T09:21:30.842326+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T09:21:30.842338+07:00 0 [Note] WSREP: Restored state OPEN -> SYNCED (9582329)
2020-06-24T09:21:30.843178+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# 11: Primary, number of nodes: 3, my index: 1, protocol version 3
2020-06-24T09:21:30.843242+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T09:21:30.843341+07:00 5 [Note] WSREP: REPL Protocols: 9 (4, 2)
2020-06-24T09:21:30.844693+07:00 0 [Note] WSREP: Member 0.0 (cluster-node1) synced with group.
2020-06-24T09:21:30.988800+07:00 5 [Note] WSREP: Assign initial position for certification: 9582329, protocol version: 4
2020-06-24T09:21:30.989470+07:00 0 [Note] WSREP: Service thread queue flushed.
2020-06-24T09:21:30.989546+07:00 5 [Note] WSREP: Synchronized with group, ready for connections
2020-06-24T09:21:30.989563+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T09:21:33.222882+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') turning message relay requesting off


Node C
2020-06-24T05:55:56.748727+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 1ceb3dad with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:55:56.753604+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: tcp://10.30.1.43:4567 
2020-06-24T05:55:57.754426+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 0
2020-06-24T05:56:00.771077+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:05.272699+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:08.691017+07:00 0 [Note] WSREP: evs::proto(2f81a4e3, GATHER, view_id(REG,1ceb3dad,620)) suspecting node: 1ceb3dad
2020-06-24T05:56:08.691668+07:00 0 [Note] WSREP: evs::proto(2f81a4e3, GATHER, view_id(REG,1ceb3dad,620)) suspected node without join message, declaring inactive
2020-06-24T05:56:09.692206+07:00 0 [Note] WSREP: view(view_id(NON_PRIM,1ceb3dad,620) memb {
	2f81a4e3,0
} joined {
} left {
} partitioned {
	1ceb3dad,0
	b48606bd,0
})
2020-06-24T05:56:09.693357+07:00 0 [Note] WSREP: view(view_id(NON_PRIM,2f81a4e3,621) memb {
	2f81a4e3,0
} joined {
} left {
} partitioned {
	1ceb3dad,0
	b48606bd,0
})
2020-06-24T05:56:09.694661+07:00 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2020-06-24T05:56:09.695104+07:00 0 [Note] WSREP: Flow-control interval: [256, 256]
2020-06-24T05:56:09.695240+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T05:56:09.695259+07:00 0 [Note] WSREP: Received NON-PRIMARY.
2020-06-24T05:56:09.695275+07:00 0 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 9582329)
2020-06-24T05:56:09.695356+07:00 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2020-06-24T05:56:09.695383+07:00 0 [Note] WSREP: Flow-control interval: [256, 256]
2020-06-24T05:56:09.695393+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T05:56:09.695401+07:00 0 [Note] WSREP: Received NON-PRIMARY.
2020-06-24T05:56:09.771723+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3
2020-06-24T05:56:09.771854+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T05:56:09.771980+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# -1: non-Primary, number of nodes: 1, my index: 0, protocol version 3
2020-06-24T05:56:09.771995+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T05:56:09.781823+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:10.696065+07:00 0 [Note] WSREP: declaring b48606bd at tcp://10.30.1.30:4567 stable
2020-06-24T05:56:10.697728+07:00 0 [Note] WSREP: view(view_id(NON_PRIM,2f81a4e3,622) memb {
	2f81a4e3,0
	b48606bd,0
} joined {
} left {
} partitioned {
	1ceb3dad,0
})
2020-06-24T05:56:10.697920+07:00 0 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 2
2020-06-24T05:56:10.697989+07:00 0 [Note] WSREP: Flow-control interval: [362, 362]
2020-06-24T05:56:10.698008+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T05:56:10.698018+07:00 0 [Note] WSREP: Received NON-PRIMARY.
2020-06-24T05:56:10.698139+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# -1: non-Primary, number of nodes: 2, my index: 0, protocol version 3
2020-06-24T05:56:10.698195+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T05:56:14.283533+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:18.786704+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:23.288202+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:56:27.291676+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection to peer 00000000 with addr tcp://10.30.1.43:4567 timed out, no messages seen in PT3S
2020-06-24T05:57:38.817692+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 30
2020-06-24T05:59:08.844661+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 60
2020-06-24T06:00:38.883278+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 90
2020-06-24T06:02:08.910682+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 120
2020-06-24T06:03:38.951557+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 150
2020-06-24T06:05:09.003140+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 180
2020-06-24T06:06:39.055554+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 210
2020-06-24T06:08:09.100981+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 240
2020-06-24T06:09:39.160631+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 270
2020-06-24T06:11:09.218993+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 300
2020-06-24T06:12:39.247609+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 330
2020-06-24T06:14:09.275575+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 360
2020-06-24T06:15:38.809586+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 390
2020-06-24T06:17:08.838642+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 420
2020-06-24T06:18:38.875385+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 450
2020-06-24T06:20:08.907983+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 480
2020-06-24T06:21:38.937346+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 510
2020-06-24T06:23:08.968237+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 540
2020-06-24T06:24:39.004672+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 570
2020-06-24T06:26:09.041725+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 600
2020-06-24T06:27:39.075586+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 630
2020-06-24T06:29:09.113035+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 660
2020-06-24T06:30:39.148766+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 690
2020-06-24T06:32:09.174982+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 720
2020-06-24T06:33:39.203873+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 750
2020-06-24T06:35:09.236552+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 780
2020-06-24T06:36:39.273287+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 810
2020-06-24T06:38:09.303512+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 840
2020-06-24T06:39:38.836719+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 870
2020-06-24T06:41:08.873322+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 900
2020-06-24T06:42:38.906965+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 930
2020-06-24T06:44:08.936437+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 960
2020-06-24T06:45:38.968792+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 990
2020-06-24T06:47:08.996032+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1020
2020-06-24T06:48:39.029468+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1050
2020-06-24T06:50:09.077618+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1080
2020-06-24T06:51:39.109591+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1110
2020-06-24T06:53:09.138939+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1140
2020-06-24T06:54:39.160600+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1170
2020-06-24T06:56:09.187998+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1200
2020-06-24T06:57:39.214927+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1230
2020-06-24T06:59:09.242261+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1260
2020-06-24T07:00:39.271220+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1290
2020-06-24T07:02:09.295768+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1320
2020-06-24T07:03:38.823272+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1350
2020-06-24T07:05:08.862910+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1380
2020-06-24T07:06:38.910103+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1410
2020-06-24T07:08:08.979765+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1440
2020-06-24T07:09:39.021933+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1470
2020-06-24T07:11:09.049762+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1500
2020-06-24T07:12:39.082209+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1530
2020-06-24T07:14:09.113021+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1560
2020-06-24T07:15:39.138512+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1590
2020-06-24T07:17:09.161037+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1620
2020-06-24T07:18:39.190899+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1650
2020-06-24T07:20:09.217369+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1680
2020-06-24T07:21:39.250440+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1710
2020-06-24T07:23:09.277725+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1740
2020-06-24T07:24:39.309504+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1770
2020-06-24T07:26:08.837326+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1800
2020-06-24T07:27:38.863784+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1830
2020-06-24T07:29:08.891861+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1860
2020-06-24T07:30:38.932764+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1890
2020-06-24T07:32:08.961812+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1920
2020-06-24T07:33:38.992976+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1950
2020-06-24T07:35:09.019870+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 1980
2020-06-24T07:36:39.052431+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2010
2020-06-24T07:38:09.086935+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2040
2020-06-24T07:39:39.120607+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2070
2020-06-24T07:41:09.146785+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2100
2020-06-24T07:42:39.175952+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2130
2020-06-24T07:44:09.207314+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2160
2020-06-24T07:45:39.246078+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2190
2020-06-24T07:47:09.276147+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2220
2020-06-24T07:48:39.308084+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2250
2020-06-24T07:50:08.833436+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2280
2020-06-24T07:51:38.864809+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2310
2020-06-24T07:53:08.895314+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2340
2020-06-24T07:54:38.924018+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2370
2020-06-24T07:56:08.951747+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2400
2020-06-24T07:57:38.982124+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2430
2020-06-24T07:59:09.011313+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2460
2020-06-24T08:00:39.040498+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2490
2020-06-24T08:02:09.070890+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2520
2020-06-24T08:03:39.097539+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2550
2020-06-24T08:05:09.122272+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2580
2020-06-24T08:06:39.147057+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2610
2020-06-24T08:08:09.177213+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2640
2020-06-24T08:09:39.209591+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2670
2020-06-24T08:11:09.235046+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2700
2020-06-24T08:12:39.266450+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2730
2020-06-24T08:14:09.299597+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2760
2020-06-24T08:15:39.335109+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2790
2020-06-24T08:17:08.866290+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2820
2020-06-24T08:18:38.898406+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2850
2020-06-24T08:20:08.931198+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2880
2020-06-24T08:21:38.979701+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2910
2020-06-24T08:23:09.012747+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2940
2020-06-24T08:24:39.045387+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 2970
2020-06-24T08:26:09.077435+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3000
2020-06-24T08:27:39.118057+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3030
2020-06-24T08:29:09.150750+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3060
2020-06-24T08:30:39.178534+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3090
2020-06-24T08:32:09.205511+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3120
2020-06-24T08:33:39.238279+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3150
2020-06-24T08:35:09.267332+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3180
2020-06-24T08:36:39.309557+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3210
2020-06-24T08:38:09.338580+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3240
2020-06-24T08:39:38.867370+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3270
2020-06-24T08:41:08.895040+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3300
2020-06-24T08:42:38.921690+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3330
2020-06-24T08:44:08.956410+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3360
2020-06-24T08:45:38.985736+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3390
2020-06-24T08:47:09.019994+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3420
2020-06-24T08:48:39.052765+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3450
2020-06-24T08:50:09.086833+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3480
2020-06-24T08:51:39.115205+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3510
2020-06-24T08:53:09.142277+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3540
2020-06-24T08:54:39.173699+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3570
2020-06-24T08:56:09.200240+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3600
2020-06-24T08:57:39.226815+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3630
2020-06-24T08:59:09.251952+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3660
2020-06-24T09:00:39.282128+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3690
2020-06-24T09:02:09.310223+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3720
2020-06-24T09:03:39.334642+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3750
2020-06-24T09:05:09.367011+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3780
2020-06-24T09:06:38.897112+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3810
2020-06-24T09:08:08.930110+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3840
2020-06-24T09:09:38.962121+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3870
2020-06-24T09:11:09.004927+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3900
2020-06-24T09:12:39.036694+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3930
2020-06-24T09:14:09.070101+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3960
2020-06-24T09:15:39.098808+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 3990
2020-06-24T09:17:09.125590+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4020
2020-06-24T09:18:38.155955+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4050
2020-06-24T09:19:23.172451+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4080
2020-06-24T09:20:07.192763+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4110
2020-06-24T09:20:51.209724+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') reconnecting to 1ceb3dad (tcp://10.30.1.43:4567), attempt 4140
2020-06-24T09:21:29.840562+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') connection established to 1ceb3dad tcp://10.30.1.43:4567
2020-06-24T09:21:30.341239+07:00 0 [Note] WSREP: declaring 1ceb3dad at tcp://10.30.1.43:4567 stable
2020-06-24T09:21:30.341323+07:00 0 [Note] WSREP: declaring b48606bd at tcp://10.30.1.30:4567 stable
2020-06-24T09:21:30.342225+07:00 0 [Note] WSREP: re-bootstrapping prim from partitioned components
2020-06-24T09:21:30.343045+07:00 0 [Note] WSREP: view(view_id(PRIM,1ceb3dad,623) memb {
	1ceb3dad,0
	2f81a4e3,0
	b48606bd,0
} joined {
} left {
} partitioned {
})
2020-06-24T09:21:30.343126+07:00 0 [Note] WSREP: save pc into disk
2020-06-24T09:21:30.443527+07:00 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 3
2020-06-24T09:21:30.443632+07:00 0 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
2020-06-24T09:21:30.841290+07:00 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
2020-06-24T09:21:30.841927+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 0 (cluster-node1)
2020-06-24T09:21:30.842010+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 1 (cluster-node2)
2020-06-24T09:21:30.842070+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 2 (cluster-node3)
2020-06-24T09:21:30.842174+07:00 0 [Warning] WSREP: Quorum: No node with complete state:

	Version      : 4
	Flags        : 0x1
	Protocols    : 0 / 9 / 3
	State        : NON-PRIMARY
	Desync count : 0
	Prim state   : NON-PRIMARY
	Prim UUID    : 00000000-0000-0000-0000-000000000000
	Prim  seqno  : -1
	First seqno  : -1
	Last  seqno  : 9582329
	Prim JOINED  : 0
	State UUID   : 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
	Group UUID   : 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
	Name         : 'cluster-node1'
	Incoming addr: '10.30.1.43:3306'

	Version      : 4
	Flags        : 0x2
	Protocols    : 0 / 9 / 3
	State        : NON-PRIMARY
	Desync count : 0
	Prim state   : SYNCED
	Prim UUID    : 1d3833af-ad80-11ea-b3bf-baf19666e63c
	Prim  seqno  : 10
	First seqno  : 9528528
	Last  seqno  : 9582329
	Prim JOINED  : 3
	State UUID   : 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
	Group UUID   : 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
	Name         : 'cluster-node2'
	Incoming addr: '10.30.1.44:3306'

	Version      : 4
	Flags        : 0x2
	Protocols    : 0 / 9 / 3
	State        : NON-PRIMARY
	Desync count : 0
	Prim state   : SYNCED
	Prim UUID    : 1d3833af-ad80-11ea-b3bf-baf19666e63c
	Prim  seqno  : 10
	First seqno  : 9544324
	Last  seqno  : 9582329
	Prim JOINED  : 3
	State UUID   : 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
	Group UUID   : 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
	Name         : 'cluster-node3'
	Incoming addr: '10.30.1.30:3306'

2020-06-24T09:21:30.842198+07:00 0 [Note] WSREP: Partial re-merge of primary 1d3833af-ad80-11ea-b3bf-baf19666e63c found: 2 of 3.
2020-06-24T09:21:30.842214+07:00 0 [Note] WSREP: Quorum results:
	version    = 4,
	component  = PRIMARY,
	conf_id    = 10,
	members    = 3/3 (joined/total),
	act_id     = 9582329,
	last_appl. = 9582287,
	protocols  = 0/9/3 (gcs/repl/appl),
	group UUID = 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33
2020-06-24T09:21:30.842281+07:00 0 [Note] WSREP: Flow-control interval: [443, 443]
2020-06-24T09:21:30.842326+07:00 0 [Note] WSREP: Trying to continue unpaused monitor
2020-06-24T09:21:30.842338+07:00 0 [Note] WSREP: Restored state OPEN -> SYNCED (9582329)
2020-06-24T09:21:30.843178+07:00 5 [Note] WSREP: New cluster view: global state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329, view# 11: Primary, number of nodes: 3, my index: 1, protocol version 3
2020-06-24T09:21:30.843242+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T09:21:30.843341+07:00 5 [Note] WSREP: REPL Protocols: 9 (4, 2)
2020-06-24T09:21:30.844693+07:00 0 [Note] WSREP: Member 0.0 (cluster-node1) synced with group.
2020-06-24T09:21:30.988800+07:00 5 [Note] WSREP: Assign initial position for certification: 9582329, protocol version: 4
2020-06-24T09:21:30.989470+07:00 0 [Note] WSREP: Service thread queue flushed.
2020-06-24T09:21:30.989546+07:00 5 [Note] WSREP: Synchronized with group, ready for connections
2020-06-24T09:21:30.989563+07:00 5 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2020-06-24T09:21:33.222882+07:00 0 [Note] WSREP: (2f81a4e3, 'tcp://0.0.0.0:4567') turning message relay requesting off

After hang, on node A 
2020-06-24T09:21:27.767684+07:00 0 [Note] /usr/sbin/mysqld (mysqld 5.7.23) starting as process 1405 ...
2020-06-24T09:21:27.774060+07:00 0 [Note] WSREP: Read nil XID from storage engines, skipping position init
2020-06-24T09:21:27.774106+07:00 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/galera/libgalera_smm.so'
2020-06-24T09:21:27.918050+07:00 0 [Note] WSREP: wsrep_load(): Galera 3.24(rf216443) by Codership Oy <[email protected]> loaded successfully.
2020-06-24T09:21:27.918137+07:00 0 [Note] WSREP: CRC-32C: using hardware acceleration.
2020-06-24T09:21:27.932343+07:00 0 [Note] WSREP: Found saved state: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:-1, safe_to_bootstrap: 0
2020-06-24T09:21:29.715429+07:00 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.30.1.43; base_port = 4567; cert.log_conflicts = YES; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.recover = no; gcache.size = 128M; gcomm.thread_prio = ; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ig
2020-06-24T09:21:29.789860+07:00 0 [Note] WSREP: GCache history reset: 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:0 -> 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329
2020-06-24T09:21:29.813620+07:00 0 [Note] WSREP: Assign initial position for certification: 9582329, protocol version: -1
2020-06-24T09:21:29.813663+07:00 0 [Note] WSREP: wsrep_sst_grab()
2020-06-24T09:21:29.813674+07:00 0 [Note] WSREP: Start replication
2020-06-24T09:21:29.813693+07:00 0 [Note] WSREP: Setting initial position to 28b84a9c-b8b0-11e7-bcc1-ab2b6eeffa33:9582329
2020-06-24T09:21:29.813845+07:00 0 [Note] WSREP: protonet asio version 0
2020-06-24T09:21:29.814070+07:00 0 [Note] WSREP: Using CRC-32C for message checksums.
2020-06-24T09:21:29.816491+07:00 0 [Note] WSREP: backend: asio
2020-06-24T09:21:29.816634+07:00 0 [Note] WSREP: gcomm thread scheduling priority set to other:0 
2020-06-24T09:21:29.838599+07:00 0 [Note] WSREP: restore pc from disk successfully
2020-06-24T09:21:29.839001+07:00 0 [Note] WSREP: GMCast version 0
2020-06-24T09:21:29.839471+07:00 0 [Note] WSREP: (1ceb3dad, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
2020-06-24T09:21:29.839496+07:00 0 [Note] WSREP: (1ceb3dad, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
2020-06-24T09:21:29.840169+07:00 0 [Note] WSREP: EVS version 0
2020-06-24T09:21:29.840354+07:00 0 [Note] WSREP: gcomm: connecting to group 'cluster-mysql', peer '10.30.1.43:,10.30.1.44:,10.30.1.30:'
2020-06-24T09:21:29.843474+07:00 0 [Note] WSREP: (1ceb3dad, 'tcp://0.0.0.0:4567') connection established to 1ceb3dad tcp://10.30.1.43:4567
2020-06-24T09:21:29.843559+07:00 0 [Warning] WSREP: (1ceb3dad, 'tcp://0.0.0.0:4567') address 'tcp://10.30.1.43:4567' points to own listening address, blacklisting
2020-06-24T09:21:29.845197+07:00 0 [Note] WSREP: (1ceb3dad, 'tcp://0.0.0.0:4567') connection established to 2f81a4e3 tcp://10.30.1.44:4567
2020-06-24T09:21:29.845290+07:00 0 [Note] WSREP: (1ceb3dad, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers: 
2020-06-24T09:21:29.845369+07:00 0 [Note] WSREP: (1ceb3dad, 'tcp://0.0.0.0:4567') connection established to b48606bd tcp://10.30.1.30:4567
2020-06-24T09:21:30.345175+07:00 0 [Note] WSREP: declaring 2f81a4e3 at tcp://10.30.1.44:4567 stable
2020-06-24T09:21:30.345216+07:00 0 [Note] WSREP: declaring b48606bd at tcp://10.30.1.30:4567 stable
2020-06-24T09:21:30.346162+07:00 0 [Note] WSREP: re-bootstrapping prim from partitioned components
2020-06-24T09:21:30.347135+07:00 0 [Note] WSREP: view(view_id(PRIM,1ceb3dad,623) memb {
	1ceb3dad,0
	2f81a4e3,0
	b48606bd,0
} joined {
} left {
} partitioned {
})
2020-06-24T09:21:30.347175+07:00 0 [Note] WSREP: save pc into disk
2020-06-24T09:21:30.390407+07:00 0 [Note] WSREP: clear restored view
2020-06-24T09:21:30.842526+07:00 0 [Note] WSREP: gcomm: connected
2020-06-24T09:21:30.842643+07:00 0 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
2020-06-24T09:21:30.842809+07:00 0 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2020-06-24T09:21:30.842829+07:00 0 [Note] WSREP: Opened channel 'cluster-mysql'
2020-06-24T09:21:30.842935+07:00 0 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 3
2020-06-24T09:21:30.843040+07:00 0 [Note] WSREP: Waiting for SST to complete.
2020-06-24T09:21:30.843923+07:00 0 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
2020-06-24T09:21:30.845347+07:00 0 [Note] WSREP: STATE EXCHANGE: sent state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6
2020-06-24T09:21:30.846068+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 0 (cluster-node1)
2020-06-24T09:21:30.846099+07:00 0 [Note] WSREP: STATE EXCHANGE: got state msg: 6a81d310-b5c1-11ea-90ec-7eeb19ee9aa6 from 1
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.