Hi!
/etc/init.d/mysql bootstrap-pxc
Everything is ok and bootstrap the cluster have done.
But when i tried to start the second node, i see this errors in log file:
2015-09-28 14:19:48 9432 [ERROR] InnoDB: ./ibdata1 can't be opened in read-write mode
2015-09-28 14:19:48 9432 [ERROR] InnoDB: The system tablespace must be writable!
2015-09-28 14:19:48 9432 [ERROR] Plugin 'InnoDB' init function returned error.
2015-09-28 14:19:48 9432 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2015-09-28 14:19:48 9432 [ERROR] Unknown/unsupported storage engine: InnoDB
2015-09-28 14:19:48 9432 [ERROR] Aborting
And i see, that owner on files in mysql_data_dir is root, but it must be mysql
-rw------- 1 mysql mysql 129M Sep 28 16:33 galera.cache
-rw-rw---- 1 mysql mysql 104 Sep 28 16:33 grastate.dat
-rw-rw---- 1 root root 48M Sep 28 16:34 ib_logfile0
-rw-rw---- 1 root root 48M Sep 28 16:34 ib_logfile1
-rw-rw---- 1 root root 12M Sep 28 16:34 ibdata1
-rw-rw---- 1 root root 20K Sep 28 16:34 innobackup.move.log
-rw-rw---- 1 root root 5.3K Sep 28 16:34 innobackup.prepare.log
drwxrwx--x 2 root root 4.0K Sep 28 16:34 mysql
-rw-r----- 1 mysql mysql 6.9K Sep 28 15:26 mysql-cluster-02.err
drwxrwx--x 2 root root 4.0K Sep 28 16:34 performance_schema
drwxrwx--x 2 root root 19 Sep 28 16:34 test
-rw-rw---- 1 root root 38 Sep 28 16:34 xtrabackup_galera_info
-rw-rw---- 1 root root 671 Sep 28 16:34 xtrabackup_info
and during the sst process i see that all files in temporary dir has root as owner
/var/lib/mysql# ls -lah .sst/
total 126M
drwxrwx--x 5 root root 4.0K Sep 28 17:12 .
drwx------ 3 mysql mysql 4.0K Sep 28 17:12 ..
-rw-rw---- 1 root root 358 Sep 28 17:12 backup-my.cnf
-rw-rw---- 1 root root 48M Sep 28 17:12 ib_logfile1
-rw-rw---- 1 root root 48M Sep 28 17:12 ib_logfile101
-rw-rw---- 1 root root 12M Sep 28 17:12 ibdata1
-rw-rw---- 1 root root 4.7K Sep 28 17:12 innobackup.prepare.log
drwx------ 2 root root 4.0K Sep 28 17:12 mysql
drwx------ 2 root root 4.0K Sep 28 17:12 performance_schema
drwx------ 2 root root 19 Sep 28 17:12 test
-rw-rw---- 1 root root 89 Sep 28 17:12 xtrabackup_checkpoints
-rw-rw---- 1 root root 38 Sep 28 17:12 xtrabackup_galera_info
-rw-rw---- 1 root root 671 Sep 28 17:12 xtrabackup_info
-rw-rw---- 1 root root 2.0M Sep 28 17:12 xtrabackup_logfile
Please, anybody, tell me how can i fix it?
Some lines from my.cnf on both nodes:
[mysqld]
user = mysql
datadir = /var/lib/mysql
# Authentication for SST method
wsrep_sst_auth="cluster:password"
# This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
innodb_autoinc_lock_mode=2
# SST method
wsrep_sst_method=xtrabackup-v2
# In order for Galera to work correctly binlog format should be ROW
binlog_format=ROW
Dear Community. Please help me with my problem. I suffer second day. Just drop a hint at list…
revin
September 30, 2015, 5:09pm
4
Can you paste here the full error log section when you start MySQL on the second node until the end.
rolls, revin, thank you for your answers!!!
This is mysq log.
Part 1:
2015-09-29 16:21:55 0 [Note] /usr/sbin/mysqld (mysqld 5.6.25-73.1-56) starting as process 12720 ...
2015-09-29 16:21:55 12720 [Note] WSREP: Read nil XID from storage engines, skipping position init
2015-09-29 16:21:55 12720 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/libgalera_smm.so'
2015-09-29 16:21:55 12720 [Note] WSREP: wsrep_load(): Galera 3.12(r9921e73) by Codership Oy <info@codership.com> loaded successfully.
2015-09-29 16:21:55 12720 [Note] WSREP: CRC-32C: using hardware acceleration.
2015-09-29 16:21:55 12720 [Note] WSREP: Found saved state: 00000000-0000-0000-0000-000000000000:-1
2015-09-29 16:21:55 12720 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 192.168.3.48; base_port = 4567; cert.log_conflicts = no; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_keep_period = PT30S; evs.inactive_check_period = PT0.5S; evs.inactive_timeout = PT15S; evs.join_retrans_period = PT1S; evs.max_install_timeouts = 3; evs.send_window = 4; evs.stats_report_period = PT1M; evs.suspect_timeout = PT5S; evs.user_send_window = 2; evs.view_forget_timeout = PT24H; gcache.dir = /var/lib/mysql/; gcache.keep_pages_count = 0; gcache.keep_pages_size = 0; gcache.mem_size = 0; gcache.name = /var/lib/mysql//galera.cache; gcache.page_size = 128M; gcache.size = 128M; gcs.fc_debug = 0; gcs.fc_factor = 1.0; gcs.fc_limit = 16; gcs.fc_master_slave = no; gcs.max_packet_size = 64500; gcs.max_throttle = 0.25; gcs.recv_q_hard_limit = 9223372036854775807; gcs.recv_q_soft_limit = 0.25; gcs.sync_donor = no; gmcast.segment = 0; gmcast.version = 0; pc.announce_timeout = PT3S; pc.checksum = false; pc.ignore_quorum =
2015-09-29 16:21:55 12720 [Note] WSREP: Service thread queue flushed.
2015-09-29 16:21:55 12720 [Note] WSREP: Assign initial position for certification: -1, protocol version: -1
2015-09-29 16:21:55 12720 [Note] WSREP: wsrep_sst_grab()
2015-09-29 16:21:55 12720 [Note] WSREP: Start replication
2015-09-29 16:21:55 12720 [Note] WSREP: Setting initial position to 00000000-0000-0000-0000-000000000000:-1
2015-09-29 16:21:55 12720 [Note] WSREP: protonet asio version 0
2015-09-29 16:21:55 12720 [Note] WSREP: Using CRC-32C for message checksums.
2015-09-29 16:21:55 12720 [Note] WSREP: backend: asio
2015-09-29 16:21:55 12720 [Warning] WSREP: access file(/var/lib/mysql//gvwstate.dat) failed(No such file or directory)
2015-09-29 16:21:55 12720 [Note] WSREP: restore pc from disk failed
2015-09-29 16:21:55 12720 [Note] WSREP: GMCast version 0
2015-09-29 16:21:55 12720 [Note] WSREP: (0dd2ccfa, 'tcp://0.0.0.0:4567') listening at tcp://0.0.0.0:4567
2015-09-29 16:21:55 12720 [Note] WSREP: (0dd2ccfa, 'tcp://0.0.0.0:4567') multicast: , ttl: 1
2015-09-29 16:21:55 12720 [Note] WSREP: EVS version 0
2015-09-29 16:21:55 12720 [Note] WSREP: gcomm: connecting to group 'group', peer '192.168.3.49:,192.168.3.48:'
2015-09-29 16:21:55 12720 [Warning] WSREP: (0dd2ccfa, 'tcp://0.0.0.0:4567') address 'tcp://192.168.3.48:4567' points to own listening address, blacklisting
2015-09-29 16:21:55 12720 [Note] WSREP: (0dd2ccfa, 'tcp://0.0.0.0:4567') turning message relay requesting on, nonlive peers:
2015-09-29 16:21:55 12720 [Note] WSREP: declaring 6f1395d2 at tcp://192.168.3.49:4567 stable
2015-09-29 16:21:55 12720 [Note] WSREP: Node 6f1395d2 state prim
2015-09-29 16:21:55 12720 [Note] WSREP: view(view_id(PRIM,0dd2ccfa,4) memb {
0dd2ccfa,0
6f1395d2,0
} joined {
} left {
} partitioned {
})
2015-09-29 16:21:55 12720 [Note] WSREP: save pc into disk
2015-09-29 16:21:55 12720 [Note] WSREP: gcomm: connected
2015-09-29 16:21:55 12720 [Note] WSREP: Changing maximum packet size to 64500, resulting msg size: 32636
2015-09-29 16:21:55 12720 [Note] WSREP: Shifting CLOSED -> OPEN (TO: 0)
2015-09-29 16:21:55 12720 [Note] WSREP: Opened channel 'group'
2015-09-29 16:21:55 12720 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 0, memb_num = 2
2015-09-29 16:21:55 12720 [Note] WSREP: Waiting for SST to complete.
2015-09-29 16:21:55 12720 [Note] WSREP: STATE_EXCHANGE: sent state UUID: 0e22d11e-66ad-11e5-8bab-fa1f2d1cf12b
2015-09-29 16:21:55 12720 [Note] WSREP: STATE EXCHANGE: sent state msg: 0e22d11e-66ad-11e5-8bab-fa1f2d1cf12b
2015-09-29 16:21:55 12720 [Note] WSREP: STATE EXCHANGE: got state msg: 0e22d11e-66ad-11e5-8bab-fa1f2d1cf12b from 1 (mysql-cluster-01)
2015-09-29 16:21:55 12720 [Note] WSREP: STATE EXCHANGE: got state msg: 0e22d11e-66ad-11e5-8bab-fa1f2d1cf12b from 0 (mysql-cluster-02)
2015-09-29 16:21:55 12720 [Note] WSREP: Quorum results:
version = 3,
component = PRIMARY,
conf_id = 3,
members = 1/2 (joined/total),
act_id = 3,
last_appl. = -1,
protocols = 0/7/3 (gcs/repl/appl),
group UUID = 662ed9a7-66a0-11e5-a831-fa74c0e8b135
2015-09-29 16:21:55 12720 [Note] WSREP: Flow-control interval: [23, 23]
2015-09-29 16:21:55 12720 [Note] WSREP: Shifting OPEN -> PRIMARY (TO: 3)
2015-09-29 16:21:55 12720 [Note] WSREP: State transfer required:
Group state: 662ed9a7-66a0-11e5-a831-fa74c0e8b135:3
Local state: 00000000-0000-0000-0000-000000000000:-1
2015-09-29 16:21:55 12720 [Note] WSREP: New cluster view: global state: 662ed9a7-66a0-11e5-a831-fa74c0e8b135:3, view# 4: Primary, number of nodes: 2, my index: 0, protocol version 3
2015-09-29 16:21:55 12720 [Warning] WSREP: Gap in state sequence. Need state transfer.
2015-09-29 16:21:55 12720 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '192.168.3.48' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --defaults-group-suffix '' --parent '12720' '' '
WSREP_SST: [INFO] Streaming with xbstream (20150929 16:21:56.384)
WSREP_SST: [INFO] Using socat as streamer (20150929 16:21:56.391)
WSREP_SST: [INFO] Evaluating timeout -k 110 100 socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20150929 16:21:56.448)
2015-09-29 16:21:56 12720 [Note] WSREP: Prepared SST request: xtrabackup-v2|192.168.3.48:4444/xtrabackup_sst//1
2015-09-29 16:21:56 12720 [Note] WSREP: wsrep_notify_cmd is not defined, skipping notification.
2015-09-29 16:21:56 12720 [Note] WSREP: REPL Protocols: 7 (3, 2)
2015-09-29 16:21:56 12720 [Note] WSREP: Service thread queue flushed.
2015-09-29 16:21:56 12720 [Note] WSREP: Assign initial position for certification: 3, protocol version: 3
2015-09-29 16:21:56 12720 [Note] WSREP: Service thread queue flushed.
2015-09-29 16:21:56 12720 [Warning] WSREP: Failed to prepare for incremental state transfer: Local state UUID (00000000-0000-0000-0000-000000000000) does not match group state UUID (662ed9a7-66a0-11e5-a831-fa74c0e8b135): 1 (Operation not permitted)
at galera/src/replicator_str.cpp:prepare_for_IST():482. IST will be unavailable.
2015-09-29 16:21:56 12720 [Note] WSREP: Member 0.0 (mysql-cluster-02) requested state transfer from '*any*'. Selected 1.0 (mysql-cluster-01)(SYNCED) as donor.
2015-09-29 16:21:56 12720 [Note] WSREP: Shifting PRIMARY -> JOINER (TO: 3)
2015-09-29 16:21:56 12720 [Note] WSREP: Requesting state transfer: success, donor: 1
WSREP_SST: [INFO] Proceeding with SST (20150929 16:21:58.683)
WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20150929 16:21:58.684)
WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20150929 16:21:58.690)
2015-09-29 16:21:58 12720 [Note] WSREP: (0dd2ccfa, 'tcp://0.0.0.0:4567') turning message relay requesting off
Part 2:
removed '/var/lib/mysql/ibdata1'
removed '/var/lib/mysql/xtrabackup_info'
removed '/var/lib/mysql/mysql/tables_priv.frm'
removed '/var/lib/mysql/mysql/db.frm'
removed '/var/lib/mysql/mysql/time_zone_name.frm'
removed '/var/lib/mysql/mysql/proc.MYI'
removed '/var/lib/mysql/mysql/help_topic.MYD'
removed '/var/lib/mysql/mysql/time_zone.MYD'
removed '/var/lib/mysql/mysql/help_topic.MYI'
removed '/var/lib/mysql/mysql/time_zone_leap_second.MYD'
removed '/var/lib/mysql/mysql/func.MYD'
removed '/var/lib/mysql/mysql/time_zone.MYI'
removed '/var/lib/mysql/mysql/slave_relay_log_info.frm'
removed '/var/lib/mysql/mysql/help_relation.frm'
removed '/var/lib/mysql/mysql/help_category.frm'
removed '/var/lib/mysql/mysql/func.MYI'
removed '/var/lib/mysql/mysql/help_relation.MYD'
removed '/var/lib/mysql/mysql/general_log.CSM'
removed '/var/lib/mysql/mysql/help_category.MYD'
removed '/var/lib/mysql/mysql/help_category.MYI'
removed '/var/lib/mysql/mysql/ndb_binlog_index.MYD'
removed '/var/lib/mysql/mysql/procs_priv.MYD'
removed '/var/lib/mysql/mysql/procs_priv.MYI'
removed '/var/lib/mysql/mysql/innodb_index_stats.frm'
removed '/var/lib/mysql/mysql/time_zone_transition_type.frm'
removed '/var/lib/mysql/mysql/servers.MYI'
removed '/var/lib/mysql/mysql/user.MYI'
removed '/var/lib/mysql/mysql/innodb_table_stats.ibd'
removed '/var/lib/mysql/mysql/event.frm'
removed '/var/lib/mysql/mysql/time_zone_leap_second.MYI'
removed '/var/lib/mysql/mysql/user.MYD'
removed '/var/lib/mysql/mysql/columns_priv.MYD'
removed '/var/lib/mysql/mysql/proxies_priv.MYD'
removed '/var/lib/mysql/mysql/time_zone_transition.frm'
removed '/var/lib/mysql/mysql/slow_log.CSV'
removed '/var/lib/mysql/mysql/servers.MYD'
removed '/var/lib/mysql/mysql/time_zone_transition_type.MYD'
removed '/var/lib/mysql/mysql/help_keyword.MYI'
removed '/var/lib/mysql/mysql/event.MYD'
removed '/var/lib/mysql/mysql/help_keyword.MYD'
removed '/var/lib/mysql/mysql/help_relation.MYI'
removed '/var/lib/mysql/mysql/proc.MYD'
removed '/var/lib/mysql/mysql/proc.frm'
removed '/var/lib/mysql/mysql/columns_priv.MYI'
removed '/var/lib/mysql/mysql/general_log.frm'
removed '/var/lib/mysql/mysql/user.frm'
removed '/var/lib/mysql/mysql/time_zone_leap_second.frm'
removed '/var/lib/mysql/mysql/func.frm'
removed '/var/lib/mysql/mysql/time_zone_name.MYD'
removed '/var/lib/mysql/mysql/slave_master_info.frm'
removed '/var/lib/mysql/mysql/columns_priv.frm'
removed '/var/lib/mysql/mysql/ndb_binlog_index.MYI'
removed '/var/lib/mysql/mysql/slave_worker_info.frm'
removed '/var/lib/mysql/mysql/tables_priv.MYI'
removed '/var/lib/mysql/mysql/slave_worker_info.ibd'
removed '/var/lib/mysql/mysql/proxies_priv.frm'
removed '/var/lib/mysql/mysql/slave_master_info.ibd'
removed '/var/lib/mysql/mysql/servers.frm'
removed '/var/lib/mysql/mysql/ndb_binlog_index.frm'
removed '/var/lib/mysql/mysql/procs_priv.frm'
removed '/var/lib/mysql/mysql/help_topic.frm'
removed '/var/lib/mysql/mysql/time_zone_transition_type.MYI'
removed '/var/lib/mysql/mysql/event.MYI'
removed '/var/lib/mysql/mysql/time_zone.frm'
removed '/var/lib/mysql/mysql/plugin.frm'
removed '/var/lib/mysql/mysql/slow_log.CSM'
removed '/var/lib/mysql/mysql/slave_relay_log_info.ibd'
removed '/var/lib/mysql/mysql/time_zone_transition.MYD'
removed '/var/lib/mysql/mysql/tables_priv.MYD'
removed '/var/lib/mysql/mysql/help_keyword.frm'
removed '/var/lib/mysql/mysql/time_zone_transition.MYI'
removed '/var/lib/mysql/mysql/innodb_table_stats.frm'
removed '/var/lib/mysql/mysql/time_zone_name.MYI'
removed '/var/lib/mysql/mysql/plugin.MYI'
removed '/var/lib/mysql/mysql/innodb_index_stats.ibd'
removed '/var/lib/mysql/mysql/db.MYI'
removed '/var/lib/mysql/mysql/db.MYD'
removed '/var/lib/mysql/mysql/plugin.MYD'
removed '/var/lib/mysql/mysql/proxies_priv.MYI'
removed '/var/lib/mysql/mysql/general_log.CSV'
removed '/var/lib/mysql/mysql/slow_log.frm'
removed directory: '/var/lib/mysql/mysql'
removed '/var/lib/mysql/performance_schema/events_statements_summary_global_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_statements_history.frm'
removed '/var/lib/mysql/performance_schema/host_cache.frm'
removed '/var/lib/mysql/performance_schema/events_stages_summary_by_account_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_stages_history_long.frm'
removed '/var/lib/mysql/performance_schema/events_stages_summary_by_host_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/file_instances.frm'
removed '/var/lib/mysql/performance_schema/events_waits_history.frm'
removed '/var/lib/mysql/performance_schema/setup_instruments.frm'
removed '/var/lib/mysql/performance_schema/file_summary_by_instance.frm'
removed '/var/lib/mysql/performance_schema/events_waits_summary_by_host_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/table_io_waits_summary_by_table.frm'
removed '/var/lib/mysql/performance_schema/socket_summary_by_instance.frm'
removed '/var/lib/mysql/performance_schema/events_waits_summary_by_account_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/session_connect_attrs.frm'
removed '/var/lib/mysql/performance_schema/setup_timers.frm'
removed '/var/lib/mysql/performance_schema/mutex_instances.frm'
removed '/var/lib/mysql/performance_schema/file_summary_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/socket_summary_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_statements_summary_by_thread_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_stages_summary_by_user_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/rwlock_instances.frm'
removed '/var/lib/mysql/performance_schema/table_lock_waits_summary_by_table.frm'
removed '/var/lib/mysql/performance_schema/events_waits_history_long.frm'
removed '/var/lib/mysql/performance_schema/table_io_waits_summary_by_index_usage.frm'
removed '/var/lib/mysql/performance_schema/setup_objects.frm'
removed '/var/lib/mysql/performance_schema/db.opt'
removed '/var/lib/mysql/performance_schema/threads.frm'
removed '/var/lib/mysql/performance_schema/session_account_connect_attrs.frm'
removed '/var/lib/mysql/performance_schema/setup_consumers.frm'
removed '/var/lib/mysql/performance_schema/hosts.frm'
removed '/var/lib/mysql/performance_schema/socket_instances.frm'
removed '/var/lib/mysql/performance_schema/events_waits_summary_global_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/users.frm'
removed '/var/lib/mysql/performance_schema/events_stages_summary_by_thread_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_statements_summary_by_account_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_statements_current.frm'
removed '/var/lib/mysql/performance_schema/events_stages_history.frm'
removed '/var/lib/mysql/performance_schema/accounts.frm'
removed '/var/lib/mysql/performance_schema/events_waits_summary_by_instance.frm'
removed '/var/lib/mysql/performance_schema/events_waits_summary_by_user_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/objects_summary_global_by_type.frm'
removed '/var/lib/mysql/performance_schema/performance_timers.frm'
removed '/var/lib/mysql/performance_schema/events_statements_summary_by_host_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_statements_history_long.frm'
removed '/var/lib/mysql/performance_schema/cond_instances.frm'
removed '/var/lib/mysql/performance_schema/events_waits_current.frm'
removed '/var/lib/mysql/performance_schema/setup_actors.frm'
removed '/var/lib/mysql/performance_schema/events_stages_current.frm'
removed '/var/lib/mysql/performance_schema/events_statements_summary_by_digest.frm'
removed '/var/lib/mysql/performance_schema/events_statements_summary_by_user_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_waits_summary_by_thread_by_event_name.frm'
removed '/var/lib/mysql/performance_schema/events_stages_summary_global_by_event_name.frm'
removed directory: '/var/lib/mysql/performance_schema'
removed '/var/lib/mysql/test/db.opt'
removed directory: '/var/lib/mysql/test'
removed '/var/lib/mysql/ib_logfile1'
removed '/var/lib/mysql/ib_logfile0'
Part 3:
WSREP_SST: [INFO] Waiting for SST streaming to complete! (20150929 16:21:59.190)
2015-09-29 16:22:13 12720 [Note] WSREP: 1.0 (mysql-cluster-01): State transfer to 0.0 (mysql-cluster-02) complete.
2015-09-29 16:22:13 12720 [Note] WSREP: Member 1.0 (mysql-cluster-01) synced with group.
WSREP_SST: [INFO] Preparing the backup at /var/lib/mysql//.sst (20150929 16:22:13.568)
WSREP_SST: [INFO] Evaluating innobackupex --no-version-check --apply-log $rebuildcmd ${DATA} &>${DATA}/innobackup.prepare.log (20150929 16:22:13.574)
WSREP_SST: [INFO] Moving the backup to /var/lib/mysql/ (20150929 16:22:20.547)
WSREP_SST: [INFO] Evaluating innobackupex --defaults-file=/etc/mysql/my.cnf --defaults-group=mysqld --no-version-check --move-back --force-non-empty-directories ${DATA} &>${DATA}/innobackup.move.log (20150929 16:22:20.553)
WSREP_SST: [INFO] Move successful, removing /var/lib/mysql//.sst (20150929 16:22:21.326)
WSREP_SST: [INFO] Galera co-ords from recovery: 662ed9a7-66a0-11e5-a831-fa74c0e8b135:3 (20150929 16:22:21.362)
WSREP_SST: [INFO] Total time on joiner: 0 seconds (20150929 16:22:21.370)
WSREP_SST: [INFO] Removing the sst_in_progress file (20150929 16:22:21.376)
2015-09-29 16:22:21 12720 [Note] WSREP: SST complete, seqno: 3
2015-09-29 16:22:21 12720 [Note] Plugin 'FEDERATED' is disabled.
/usr/sbin/mysqld: Can't find file: './mysql/plugin.frm' (errno: 13 - Permission denied)
2015-09-29 16:22:21 12720 [ERROR] Can't open the mysql.plugin table. Please run mysql_upgrade to create it.
2015-09-29 16:22:21 12720 [Note] InnoDB: Using atomics to ref count buffer pool pages
2015-09-29 16:22:21 12720 [Note] InnoDB: The InnoDB memory heap is disabled
2015-09-29 16:22:21 12720 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2015-09-29 16:22:21 12720 [Note] InnoDB: Memory barrier is not used
2015-09-29 16:22:21 12720 [Note] InnoDB: Compressed tables use zlib 1.2.8
2015-09-29 16:22:21 12720 [Note] InnoDB: Using Linux native AIO
2015-09-29 16:22:21 12720 [Note] InnoDB: Using CPU crc32 instructions
2015-09-29 16:22:21 12720 [Note] InnoDB: Initializing buffer pool, size = 50.0G
2015-09-29 16:22:25 12720 [Note] InnoDB: Completed initialization of buffer pool
2015-09-29 16:22:26 12720 [ERROR] InnoDB: ./ibdata1 can't be opened in read-write mode
2015-09-29 16:22:26 12720 [ERROR] InnoDB: The system tablespace must be writable!
2015-09-29 16:22:26 12720 [ERROR] Plugin 'InnoDB' init function returned error.
2015-09-29 16:22:26 12720 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2015-09-29 16:22:26 12720 [ERROR] Unknown/unsupported storage engine: InnoDB
2015-09-29 16:22:26 12720 [ERROR] Aborting
2015-09-29 16:22:28 12720 [Note] WSREP: Closing send monitor...
2015-09-29 16:22:28 12720 [Note] WSREP: Closed send monitor.
2015-09-29 16:22:28 12720 [Note] WSREP: gcomm: terminating thread
2015-09-29 16:22:28 12720 [Note] WSREP: gcomm: joining thread
2015-09-29 16:22:28 12720 [Note] WSREP: gcomm: closing backend
2015-09-29 16:22:29 12720 [Note] WSREP: view(view_id(NON_PRIM,0dd2ccfa,4) memb {
0dd2ccfa,0
} joined {
} left {
} partitioned {
6f1395d2,0
})
2015-09-29 16:22:29 12720 [Note] WSREP: view((empty))
2015-09-29 16:22:29 12720 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2015-09-29 16:22:29 12720 [Note] WSREP: gcomm: closed
2015-09-29 16:22:29 12720 [Note] WSREP: Flow-control interval: [16, 16]
2015-09-29 16:22:29 12720 [Note] WSREP: Received NON-PRIMARY.
2015-09-29 16:22:29 12720 [Note] WSREP: Shifting JOINER -> OPEN (TO: 3)
2015-09-29 16:22:29 12720 [Note] WSREP: Received self-leave message.
2015-09-29 16:22:29 12720 [Note] WSREP: Flow-control interval: [0, 0]
2015-09-29 16:22:29 12720 [Note] WSREP: Received SELF-LEAVE. Closing connection.
2015-09-29 16:22:29 12720 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 3)
2015-09-29 16:22:29 12720 [Note] WSREP: RECV thread exiting 0: Success
2015-09-29 16:22:29 12720 [Note] WSREP: recv_thread() joined.
2015-09-29 16:22:29 12720 [Note] WSREP: Closing replication queue.
2015-09-29 16:22:29 12720 [Note] WSREP: Closing slave action queue.
2015-09-29 16:22:29 12720 [Note] WSREP: Service disconnected.
2015-09-29 16:22:29 12720 [Note] WSREP: rollbacker thread exiting
2015-09-29 16:22:30 12720 [Note] WSREP: Some threads may fail to exit.
2015-09-29 16:22:30 12720 [Note] Binlog end
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'partition'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'ARCHIVE'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_CHANGED_PAGES'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_DATAFILES'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_TABLESPACES'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN_COLS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_FOREIGN'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_FIELDS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_COLUMNS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_INDEXES'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_TABLESTATS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_SYS_TABLES'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_FT_INDEX_TABLE'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_FT_INDEX_CACHE'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_FT_CONFIG'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_FT_BEING_DELETED'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_FT_DELETED'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_FT_DEFAULT_STOPWORD'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_METRICS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_BUFFER_POOL_STATS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE_LRU'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_BUFFER_PAGE'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX_RESET'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_CMP_PER_INDEX'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_CMPMEM_RESET'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_CMPMEM'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_CMP_RESET'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_CMP'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_LOCK_WAITS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_LOCKS'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'INNODB_TRX'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'XTRADB_RSEG'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'XTRADB_INTERNAL_HASH_TABLES'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'XTRADB_READ_VIEW'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'BLACKHOLE'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'PERFORMANCE_SCHEMA'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'MyISAM'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'CSV'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'MEMORY'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'MRG_MYISAM'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'sha256_password'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'mysql_old_password'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'mysql_native_password'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'wsrep'
2015-09-29 16:22:30 12720 [Note] Shutting down plugin 'binlog'
2015-09-29 16:22:31 12720 [Note] /usr/sbin/mysqld: Shutdown complete
rolls
October 5, 2015, 4:11am
8
Make mysql user owner of /var/lib/mysql with #chown -R mysql.mysql /var/lib/mysql