Cluster goes down with 1 node offline

MySQL & MariaDB Percona XtraDB Cluster 5.x
1

I was testing XtraDB for our new database-setup and I’m running into trouble after testing a fail-over scenario.
Right now I have 2 nodes in this cluster, both working and syncing fine.
When I shut the network down on 1 of the nodes, the other node keeps reconnecting and the database-server is hanging:

At this time I can not log in to the node that is online, it’s waiting for the other side to come back online, it seems:

mysql -uroot -pxxxxxxx database_name

mysql: [Warning] Using a password on the command line interface can be insecure.
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Is this expecting behavior for a 2-node cluster perhaps?

The plan was to have 1 server in the datacenter and the other node locally at the office, but right now this seems a bad idea when both go offline like this.

2

The clean way to shutdown the node is to execute “mysqladmin shutdown” on that node.
If you just put network down, the working node can’t figure out what happened and to prevent the split-brain it disables itself.

3

Ok, but what if the server crashes?
Or is the only way to fix this, use more than 2 servers?

4

Server crash is also different from the network timeout, so it should be handled properly.
But in general the typical cluster recommendation is to have 3 or more servers.