WSREP: no nodes coming from prim view, prim not possible

Setting up a 4 node XtraDB cluster for eval at work. The first three notes went pretty smoothly, but number 4 is another story. It was working then I ran apt-get upgrade and it wont restart. Well it did restart once I ran another apt-get to install a utility and boom mysql won’t restart.

So a few questions

I can’t find a answer to how wsrep_cluster_address=gcomm://192.168.0.83:3306 should be set see lot of different answers on the internet.

I have:
node 1 ip .81
wsrep_cluster_address=gcomm://
node 2 ip .82
wsrep_cluster_address=gcomm://192.168.0.83:3306
node 3 ip .83
wsrep_cluster_address=gcomm://192.168.0.81:3306
node 4 ip .100
wsrep_cluster_address=gcomm://192.168.0.82:3306

I’ve seen on net saying point 2-4 at node 1, others saying daisy chain them.

Then when a node won’t restart no errors just the warning " WSREP: no nodes coming from prim view, prim not possible"

Any idea what’s wrong things were working until the apt-get upgrade and now everytime I run apt-get it stop mysql and restart it.

the my.cnf I’m using…

Generated by Percona Configuration Wizard

[mysql]

CLIENT

port = 3306
socket = /mysql/mysql.sock

[mysqld]

GENERAL

user = mysql
default_storage_engine = InnoDB
socket = /mysql/mysql.sock
pid_file = /mysql/mysql.pid

MyISAM

key_buffer_size = 32M
myisam_recover = FORCE,BACKUP

SAFETY

max_allowed_packet = 16M
max_connect_errors = 1000000

DATA STORAGE

datadir = /mysql/

BINARY LOGGING

log_bin = /mysql/mysql-bin
expire_logs_days = 14
sync_binlog = 1

REPLICATION

log_slave_updates = 1
relay_log = /mysql/relay-bin
slave_net_timeout = 60
sync_master_info = 1
sync_relay_log = 1
sync_relay_log_info = 1

CACHES AND LIMITS

tmp_table_size = 32M
max_heap_table_size = 32M
query_cache_type = 0
query_cache_size = 0
max_connections = 500
thread_cache_size = 50
open_files_limit = 65535
table_definition_cache = 4096
table_open_cache = 4096

INNODB

innodb_flush_method = O_DIRECT
innodb_log_files_in_group = 2
innodb_log_file_size = 512M
innodb_flush_log_at_trx_commit = 1
innodb_file_per_table = 1
innodb_buffer_pool_size = 26G

LOGGING

log_error = /mysql/mysql-error.log
log_queries_not_using_indexes = 1
slow_query_log = 1
slow_query_log_file = /mysql/mysql-slow.log

Can’t someone at least say how the gcomm should be configured?

I have:
node 1 ip .81
wsrep_cluster_address=gcomm://
node 2 ip .82
wsrep_cluster_address=gcomm://192.168.0.83:3306
node 3 ip .83
wsrep_cluster_address=gcomm://192.168.0.81:3306
node 4 ip .100
wsrep_cluster_address=gcomm://192.168.0.82:3306

I’ve seen on net saying point 2-4 at node 1, others saying daisy chain them.

first node works when set to nothing, but other info say set it to and address once 2nd and 3th nodes are up.

Please advise.