Not the answer you need?
Register and ask your own question!

New setup fails to join

evrynetevrynet ContributorCurrent User Role Beginner
Hello,

I recently deployed a new xtradb cluster setup on Ubuntu 14.04 however they are fails to join. From the logs it seems something is missing from the server however there is nothing specifics in logs and I couldn't locate whats missing. I have installed libssl,rsync etc...

2014-12-28 08:09:09 4142 [Note] WSREP: New cluster view: global state: 2cda4783-8e85-11e4-b8e9-a278e296bb8d:9543, view# 16: Primary, number of nodes: 2, my index: 1, protocol version 3
2014-12-28 08:09:09 4142 [Warning] WSREP: Gap in state sequence. Need state transfer.
2014-12-28 08:09:09 4142 [Note] WSREP: Running: 'wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.2.2.2' --auth 'sstuser:Xgbx6X' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '4142' '' '
2014-12-28 08:09:09 4142 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.2.2.2' --auth 'sstuser:Xgbx6X' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '4142' ''
Read: '(null)'
2014-12-28 08:09:09 4142 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.2.2.2' --auth 'sstuser:Xgbx6X' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '4142' '' : 2 (No such file or directory)
2014-12-28 08:09:09 4142 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
2014-12-28 08:09:09 4142 [ERROR] Aborting

Both nodes has the same versions.

[email protected]:/var/lib/mysql# mysql --version
mysql Ver 14.14 Distrib 5.6.21-70.1, for debian-linux-gnu (x86_64) using EditLine wrapper
[email protected]:/var/lib/mysql#

They are running xtradb version 5.6 and both has wsrep_sst_xtrabackup-v2 as well. Please advise.

Thanks

Comments

  • wagnerbianchiwagnerbianchi Remote DBA Current User Role Patron
    Is there on the donor's mysql the user and password configured as the value of wsrep_sst_auth (I believe so as the sst process completed successfully)? Can you check if any firewalls are restricting access on used ports (UFW/AppArmor)?
  • evrynetevrynet Contributor Current User Role Beginner
    Is there on the donor's mysql the user and password configured as the value of wsrep_sst_auth (I believe so as the sst process completed successfully)? Can you check if any firewalls are restricting access on used ports (UFW/AppArmor)?

    Yes the user/pass is configured correctly. AppArmor has been removed however there is an active firewall configured on the donor but the necessary ports (3306,4444,4567,4568) are enabled.
  • evrynetevrynet Contributor Current User Role Beginner
    Hi,

    Ok. It seems it wasn't enough just to disable and remove AppArmor but we needed an actual reboot too. Now the issue is as follows:

    WSREP_SST: [INFO] Stale sst_in_progress file: /var/lib/mysql//sst_in_progress (20141228 13:54:19.016)
    touch: cannot touch ‘/var/lib/mysql//sst_in_progress’: Permission denied
    2014-12-28 13:54:19 19751 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.2.2.2' --auth 'sstuser:Xgbx6X' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '19751' ''
    Read: '(null)'
    2014-12-28 13:54:19 19751 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '10.2.2.2' --auth 'sstuser:Xgbx6X' --datadir '/var/lib/mysql/' --defaults-file '/etc/mysql/my.cnf' --parent '19751' '' : 1 (Operation not permitted)

    the folder is owned by mysql on both servers:

    drwx
    10 mysql mysql 4096 Dec 28 13:52 mysql

    The mysql is running as mysql user defined in /etc/mysql/my.cnf

    [mysqld]

    datadir=/var/lib/mysql
    user=mysql

    Any advise?

    Thanks
  • evrynetevrynet Contributor Current User Role Beginner
    Okey removing the sst_in_progress file from the slave solved the issue.
  • wagnerbianchiwagnerbianchi Remote DBA Current User Role Patron
    It's good to hear from you that everything is running well.
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.