# Percona Cluster: 2nd node can't start on "Ubuntu 12.04.1 LTS" **URL:** \ **Category:** Percona XtraDB Cluster 5.x\ **Created:** [July 2, 2013, 7:22am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785 "2013-07-02T07:22:32Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![torotry](https://avatars.discourse-cdn.com/v4/letter/t/65b543/32.png) [@torotry](https://forums.percona.com/u/torotry)\ **Post date:** [July 2, 2013, 7:22am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/1 "2013-07-02T07:22:32Z")
Hi, I have 3 servers with “Ubuntu 12.04.1 LTS” and I have instaled Percona XtraDB Cluster server from the binaries repositories using apt-get method. It appears that everything is installed correctly, because it runs the mysql server standalone from each node. The first node (with wsrep\_cluster\_address=gcomm://) starts without problem. And I can run mysql --defaults-file=/etc/mysql/debian.cnf -e “show status like ‘wsrep%’;” and get the spected output saying that there is a cluster initialized with wsrep\_cluster\_size=1. But when I start the second (or third node) the service mysql start finish with [[COLOR=#FF0000]fail] message. When i tried to start mysql with the command : “/etc/init.d/mysql start” or “service mysql start”, i get the message start [COLOR=#FF0000]failed. “[my.cn](http://my.cn)f” file for the first node is: [client] port = 3306 socket = /var/run/mysqld/mysqld.sock [mysqld] user = mysql default\_storage\_engine = InnoDB port = 3306 pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock # MyISAM key\_buffer\_size = 32M # SAFETY max\_allowed\_packet = 16M max\_connect\_errors = 1000 # DATA STORAGE datadir = /var/opt/hosting/db # BINARY LOGGING log\_bin = /var/opt/hosting/tmp/mysql-bin-log/log-bin-node01.log expire\_logs\_days = 10 # INNODB innodb\_flush\_method = ALL\_O\_DIRECT innodb\_log\_files\_in\_group = 2 innodb\_log\_file\_size = 150M innodb\_import\_table\_from\_xtrabackup = 1 innodb\_flush\_log\_at\_trx\_commit = 1 innodb\_file\_per\_table = 1 innodb\_buffer\_pool\_size = 1G # LOGGING log-error = /var/opt/hosting/db/node1.err long\_query\_time = 2 slow-query-log-file = /var/opt/hosting/log/mysql/mysql-slow-queries.node1.log # GALERA # Path to Galera library wsrep\_provider = /usr/lib/libgalera\_smm.so # Cluster connection URL contains the IPs of node#1, node#2 and node#3 wsrep\_cluster\_address = gcomm:// # In order for Galera to work correctly binlog format should be ROW binlog\_format = ROW # This is a recommended tuning variable for performance innodb\_locks\_unsafe\_for\_binlog = 1 # This changes how InnoDB autoincrement locks are managed and is a requirement for Galera innodb\_autoinc\_lock\_mode = 2 # Node address wsrep\_node\_address = node1\_IP # SST method wsrep\_sst\_method = xtrabackup # Cluster name wsrep\_cluster\_name = my\_first\_node # Authentication for SST method ## wsrep\_sst\_auth = “sst\_user:password” “my.cnf” file for the 2nd node [client] port = 3306 socket = /var/run/mysqld/mysqld.sock [mysqld] user = mysql default\_storage\_engine = InnoDB port = 3306 pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock # MyISAM key\_buffer\_size = 32M # SAFETY max\_allowed\_packet = 16M max\_connect\_errors = 1000 # DATA STORAGE datadir = /var/opt/hosting/db # BINARY LOGGING log\_bin = /var/opt/hosting/tmp/mysql-bin-log/log-bin-node02.log expire\_logs\_days = 10 # INNODB innodb\_flush\_method = ALL\_O\_DIRECT innodb\_log\_files\_in\_group = 2 innodb\_log\_file\_size = 150M innodb\_import\_table\_from\_xtrabackup = 1 innodb\_flush\_log\_at\_trx\_commit = 1 innodb\_file\_per\_table = 1 innodb\_buffer\_pool\_size = 1G # LOGGING log-error = /var/opt/hosting/db/poolm/node2.err long\_query\_time = 2 slow-query-log-file = /var/opt/hosting/log/mysql/mysql-slow-queries.node2.log # GALERA # Path to Galera library wsrep\_provider = /usr/lib/libgalera\_smm.so # Cluster connection URL contains the IPs of node#1, node#2 and node#3 wsrep\_cluster\_address = gcomm://node1\_IP # In order for Galera to work correctly binlog format should be ROW binlog\_format = ROW # This is a recommended tuning variable for performance innodb\_locks\_unsafe\_for\_binlog = 1 # This changes how InnoDB autoincrement locks are managed and is a requirement for Galera innodb\_autoinc\_lock\_mode = 2 # Node address wsrep\_node\_address = node2\_IP # SST method wsrep\_sst\_method = xtrabackup # Cluster name wsrep\_cluster\_name = my\_first\_node # Authentication for SST method wsrep\_sst\_auth = “sst\_user:password” I followed the documentation from the Percona XtraDB Cluster : [http://www.percona.com/doc/percona-xtradb-cluster/index.html](http://www.percona.com/doc/percona-xtradb-cluster/index.html) During the process of starting up with “/etc/init.d/mysql start” command on “node2”, it looks like the synchronization from this node(node2) with “node1” set more time than the 14seconds which were defined. I think “/etc/init.d/mysql start” command, don’t wait that this synchronization is ended before going out in error. What do you think about it? Can you help me with this setup, I need it urgently. Thanks in advance. PS : PERCONA SERVER VERSION : 5.5.30-30.2-log Percona Server (GPL), Release 30.2, wsrep\_23.7.4.r3843 ---
**Author:** ![percona.jayj](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/percona.jayj/32/927_2.png) [@percona.jayj](https://forums.percona.com/u/percona.jayj)\ **Post date:** [July 4, 2013, 7:58am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/2 "2013-07-04T07:58:43Z")
So – what does the log say on the 2nd node? My guess would be this is some SST error, so you can also check the innobackup.backup.log file in the datadir on the first node for clues. ---
**Author:** ![torotry](https://avatars.discourse-cdn.com/v4/letter/t/65b543/32.png) [@torotry](https://forums.percona.com/u/torotry)\ **Post date:** [July 4, 2013, 9:28am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/3 "2013-07-04T09:28:16Z")
Thank you for your reply. For the log, please see attached files : “node1log.txt” I noticed that “/etc/init.d/mysql” deployed during installation of “percona-xtradb-cluster-server.5.5” was erroneous. See attached file “etc\_initd\_mysql\_deployed”. I replaced it by “etc\_initd\_mysql\_replaced” file before my installation and the 2nd node started well and joined the cluster maybe because of in this script timeout waiting is set to “service\_startup\_timeout=900” seconds. Expect 15 minutes that 2nd node synchronizes with the cluster don’t seem to me very determinist. If size of my data in cluster make 10Go,15Go,50Go or 1To…, 15 minutes seems to me too few. Wouldn’t it be better to wait that the synchronization of the second node is ended instead of waiting 15minutes? [etc\_initd\_mysql\_deployed.sh](https://forums.percona.com/uploads/short-url/ztbxPqiy7cWIa1cztxTseqt9XGK.txt) (8.95 KB) [etc\_initd\_mysql\_replaced.sh](https://forums.percona.com/uploads/short-url/aelcILirxCTNQ8m8V3W6RLg9Ess.txt) (17.3 KB) ---
**Author:** ![torotry](https://avatars.discourse-cdn.com/v4/letter/t/65b543/32.png) [@torotry](https://forums.percona.com/u/torotry)\ **Post date:** [July 4, 2013, 9:31am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/4 "2013-07-04T09:31:26Z")
node2log.txt ==\> [node2log.txt](https://forums.percona.com/uploads/short-url/aelcILirxCTNQ8m8V3W6RLg9Ess.txt) (17.3 KB) ---
**Author:** ![percona.jayj](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/percona.jayj/32/927_2.png) [@percona.jayj](https://forums.percona.com/u/percona.jayj)\ **Post date:** [July 4, 2013, 1:11pm UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/5 "2013-07-04T13:11:58Z")
Based on your logs, I’m not sure I agree with the assessment that your problems are due to init script timeouts – there are some clear crashes with stack traces in the log. I’d try clearing the datadir on your second node to see if that helps at all. I’d also check the innobackup.apply.log on that node to see if there were any errors recovering the Xtrabackup SST before the node starts up. ---
**Author:** ![torotry](https://avatars.discourse-cdn.com/v4/letter/t/65b543/32.png) [@torotry](https://forums.percona.com/u/torotry)\ **Post date:** [July 5, 2013, 3:36am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/6 "2013-07-05T03:36:10Z")
Before every installation i clean my datadir on 2ndnode but that doesn’t seem to solve my problem…Even before “2ndnode” tries to connect to the cluster, i already have an error in /var/opt/hosting/db/poolm/node2.err log log : 130704 16:27:40 Percona XtraDB ([http://www.percona.com](http://www.percona.com)) 5.5.30-rel30.2 started; log sequence number 1597945 ERROR: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ALTER TABLE user ADD column Show\_view\_priv enum(‘N’,‘Y’) CHARA 130704 16:27:40 [ERROR] Aborting What do you think of the difference between “etc\_initd\_mysql\_deployed” and “etc\_initd\_mysql\_replaced” (I got back etc\_initd\_mysql\_replaced here : [url][Download Percona XtraDB Cluster 5.5](http://www.percona.com/downloads/Percona-XtraDB-Cluster/5.5.30-23.7.4/binary/linux/x86_64/%5B/url%5D)) files. Why during installation of “percona-xtradb-cluster-server.5.5” “etc\_initd\_mysql\_deployed” was deployed in place of “etc\_initd\_mysql\_replaced”? Compared to my previous post, if size of my data in cluster make 10Go,15Go,50Go or 1To wouldn’t it be better to wait that the synchronization of the second node is ended instead of waiting 15minutes? ---
**Author:** ![percona.jayj](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/percona.jayj/32/927_2.png) [@percona.jayj](https://forums.percona.com/u/percona.jayj)\ **Post date:** [July 8, 2013, 7:58am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/7 "2013-07-08T07:58:44Z")
I’m not a huge fan of all the “help” the init scripts try to give you in Ubuntu/Debian. In my experience errors like “ERROR: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ALTER TABLE user ADD column Show\_view\_priv enum(‘N’,‘Y’) CHARA” happen on newly created datadirs and you may be better off with leaving a working datadir on node2 before you start it. I can’t comment on the inner workings of the init script. If you think there’s something wrong there, then I’d direct you to: [URL=“[Percona XtraDB Cluster](http://www.percona.com/doc/percona-xtradb-cluster/howtos/bugreport.html)”][http://www.percona.com/doc/percona-x...bugreport.html[/URL]](http://www.percona.com/doc/percona-x...bugreport.html%5B/URL%5D) ---
**Author:** ![torotry](https://avatars.discourse-cdn.com/v4/letter/t/65b543/32.png) [@torotry](https://forums.percona.com/u/torotry)\ **Post date:** [July 8, 2013, 11:09am UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/8 "2013-07-08T11:09:05Z")
OK. Thank You. For information, I tried to install the lastest version of “percona-xtradb-cluster-server-5.5 (5.5.31-23.7.5-438.precise)” and I haven’t errors on node2 during her synchronization with node1. However something wrong in “/etc/init.d/mysql” script and i am going to report it. ---
**Author:** ![przemek](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/przemek/32/3_2.png) [@przemek](https://forums.percona.com/u/przemek)\ **Post date:** [July 9, 2013, 2:52pm UTC](https://forums.percona.com/t/percona-cluster-2nd-node-cant-start-on-ubuntu-12-04-1-lts/2785/9 "2013-07-09T14:52:50Z")
A hardcoded timeout in the init script can be too short and this was addressed in Percona XtraDB Cluster version 5.5.31 - see this bug: [url][https://bugs.launchpad.net/percona-x...r/+bug/1099428[/url]](https://bugs.launchpad.net/percona-x...r/+bug/1099428%5B/url%5D) The other thing you should make sure is that newly joined nodes have the same credentials in /etc/mysql/debian.cnf as primary node. This is also needed for init script to work properly.