tar: -: Cannot write: Broken pipe error

My third galera mariadb node (out of 3) will not start and I get this:

150428 15:11:27 innobackupex: Connecting to MySQL server with DSN ‘dbi:mysql:;mysql_read_default_file=/etc/my.cnf;mysql_read_default_group=xtrabackup;mysql_socket=/var/lib/mysql/mysql.sock’ as ‘root’ (using password: YES).
150428 15:11:27 innobackupex: Connected to MySQL server
150428 15:11:27 innobackupex: Executing a version check against the server…
150428 15:11:27 innobackupex: Done.
150428 15:11:27 innobackupex: Starting the backup operation

IMPORTANT: Please check that the backup run completes successfully.
At the end of a successful backup run innobackupex
prints “completed OK!”.

innobackupex: Using server version 10.0.17-MariaDB-wsrep

innobackupex: Created backup directory /tmp
tar: -: Cannot write: Broken pipe
tar: Error is not recoverable: exiting now
innobackupex: ‘tar chf -’ returned with exit code 2.
innobackupex: got a fatal error with the following stacktrace: at /usr//bin/innobackupex line 4894.
main::backup_file_via_stream(’/tmp’, ‘backup-my.cnf’) called at /usr//bin/innobackupex line 4943
main::backup_file(’/tmp’, ‘backup-my.cnf’, ‘/tmp/backup-my.cnf’) called at /usr//bin/innobackupex line 4967
main::write_to_backup_file(’/tmp/backup-my.cnf’, ‘# This MySQL options file was generated by innobackupex.\x{a}\x{a}# T…’) called at /usr//bin/innobackupex line 3774
main::write_backup_config_file(’/tmp/backup-my.cnf’) called at /usr//bin/innobackupex line 3701
main::init() called at /usr//bin/innobackupex line 1566
innobackupex: Error: Failed to stream ‘/tmp/backup-my.cnf’: 2 at /usr//bin/innobackupex line 4894.

Any ideas? Thanks

Try cleaning up mysql’s datadir on the third node and restart mysql. Can you paste my.cnf of the third node?

When you say cleanup the datadir, what does that mean? Sorry I am new to this. Here is my server.cnf:

[server]

this is only for the mysqld standalone daemon

[mysqld]
log_error=/var/log/mysql/error.log

* Galera-related settings

[galera]

Mandatory settings

wsrep_provider=/usr/lib64/galera/libgalera_smm.so
wsrep_cluster_address=gcomm://10.10.61.146,10.10.61.147,10.10.61.148
binlog_format=row
default_storage_engine=InnoDB
innodb_autoinc_lock_mode=2
bind-address=0.0.0.0
user=root

Optional setting

wsrep_slave_threads=1
innodb_flush_log_at_trx_commit=0
wsrep_provider_options=“gcache.size=32G;gcache.page_size=1G;”
wsrep_cluster_name=‘example_cluster’
wsrep_node_address=‘10.10.61.148’
wsrep_node_name=‘10.10.61.148’
wsrep_sst_method=xtrabackup
wsrep_sst_auth=root:password
innodb_log_file_size=100M

this is only for embedded server

[embedded]

This group is only read by MariaDB servers, not by MySQL.

If you use the same .cnf file for MySQL and MariaDB,

you can put MariaDB-only options here

[mariadb]

This group is only read by MariaDB-10.0 servers.

If you use the same .cnf file for MariaDB of different versions,

use this group for options that older servers don’t understand

[mariadb-10.0]

Thanks

Move or delete everything inside /var/lib/mysql on node 3 (granting you use default MySQL datadir) then change

wsrep_sst_method=xtrabackup
to
wsrep_sst_method=xtrabackup-v2

Restart node 3.