why when i restart all node ,they need to resync with primary node 1

why when i restart all node ,they need to resync with primary node 1.it will take so longtime to sync.
removed /data/mysql/maoye/biz_contract_leaseassociated.ibd' removed /data/mysql/maoye/act_ru_job.ibd’
removed /data/mysql/maoye/biz_stop_contract.ibd' removed /data/mysql/maoye/biz_pospay_sale_temp.frm’
removed /data/mysql/maoye/ztfi_xscb_rh.frm' removed /data/mysql/maoye/custom_query.ibd’
removed /data/mysql/maoye/biz_statement_main_detail1.ibd' removed /data/mysql/maoye/biz_project_ts.frm’
removed /data/mysql/maoye/biz_brand_synexception.ibd' removed /data/mysql/maoye/biz_contract_marketassmonthguarantee.ibd’
removed /data/mysql/maoye/bpm_task_turnassign.frm' removed /data/mysql/maoye/bpm_form_inst.frm’
removed /data/mysql/maoye/biz_contract_leaseasssmallproprietor.frm' removed /data/mysql/maoye/biz_contract_advertassociated.frm’
removed /data/mysql/maoye/biz_brand_cust.ibd' removed /data/mysql/maoye/t_auth_documents_permission#P#p5.ibd’
removed /data/mysql/maoye/sys_base_shop_temp.ibd' removed /data/mysql/maoye/biz_pospay_parm_terminal.frm’
removed /data/mysql/maoye/sys_base_contractatttype.ibd' removed /data/mysql/maoye/bpm_pro_status.frm’
removed /data/mysql/maoye/biz_intent_assbaseitems.frm' removed /data/mysql/maoye/act_ru_identitylink.ibd’
removed /data/mysql/maoye/biz_pospay_supplier_goods.ibd' removed /data/mysql/maoye/t_auth_structure.frm’
removed /data/mysql/maoye/biz_standard_stdsupportcondition.frm' removed /data/mysql/maoye/t_auth_general_post.frm’
removed directory: /data/mysql/maoye' removed /data/mysql/masterbin-log.000001’
dirname: extra operand /data/mysql/masterbin-log' Try dirname --help’ for more information.
WSREP_SST: [ERROR] Cleanup after exit with status:1 (20170816 12:05:09.337)
WSREP_SST: [INFO] Removing the sst_in_progress file (20170816 12:05:09.340)
2017-08-16 12:05:09 15605 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role ‘joiner’ --address ‘172.29.9.42’ --auth ‘sstuser:s3cret’ --datadir ‘/data/mysql/’ --defaults-file ‘/etc/my.cnf’ --parent ‘15605’ ‘’ : 1 (Operation not permitted)
2017-08-16 12:05:09 15605 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2017-08-16 12:05:09 15605 [ERROR] WSREP: SST failed: 1 (Operation not permitted)
2017-08-16 12:05:09 15605 [ERROR] Aborting

There is good probability that something is going wrong on restart that is out of PXC control.

Generally on restart PXC node will initiate IST from the cluster node if IST is not possible it will turn to SST.
There are multiple reason why IST is not possible

  • Cluster has made good progress before the restarted node tried to join the cluster again.
  • Shutdown was not successful and process was killed. Check what are content of grastate.dat. (It should have valid seqno and -1 if shutdown was successful).