Disabling AppArmor for installing PXC on ubuntu 16.04

[tom256]

Hi, today I tried to start a Percona XtraDB Cluster on Ubuntu 16.04, I’m using vestacp control panel, so Mysql is already installed.
In the first step of installing written in the percona website in third note of Prerequisites, I saw this note:

Note
If you previously had MySQL installed on the server, there might be an AppArmor profile which will prevent Percona XtraDB Cluster nodes from communicating with each other. The best solution is to remove the apparmor package entirely:

$ sudo apt-get remove apparmor

If you need to have AppArmor enabled due to security policies or for other reasons, it is possible to disable or extend the MySQL profile.

Very bad, dangerous and unacceptable solution by Percona!!!
As mentioned in comment sections of this tutorial, Disabling AppArmor in a wrong way will break your Ubuntu 14x -16x system and will remove a lot of programs!!

I tried sudo apt-get remove apparmor which was the solution of Percona, and I got this:

The following packages were automatically installed and are no longer required:
libcgi-fast-perl libcgi-pm-perl libevent-core-2.0-5 libfcgi-perl libhtml-template-perl linux-headers-4.4.0-131
linux-headers-4.4.0-131-generic linux-image-4.4.0-131-generic linux-image-extra-4.4.0-131-generic mysql-server-core-5.7
python3-apparmor python3-libapparmor
Use ‘sudo apt autoremove’ to remove them.
The following packages will be REMOVED:
apparmor apparmor-utils liblxc1 lxc-common lxd mysql-server mysql-server-5.7 snapd
0 upgraded, 0 newly installed, 8 to remove and 0 not upgraded.
After this operation, 150 MB disk space will be freed.

As you can see it is going to uninstall the Mysql, and as a result, it will break my whole vestacp server! Fortunately, I answered No and was able to save my life!

My question is what is the right solution to install PXC when Mysql is already installed?
[HR][/HR]PS: I was googling the whole day and found this tutorial:
https://www.percona.com/blog/2018/08/09/lock-down-enforcing-apparmor-with-percona-xtradb-cluster/

But I couldn’t understand it completely, it was a little confusing.
1- First of all, Is it the solution I need for installing PXC when Mysql is already installed
2- What is that Start iterating title mean? Is it a mandatory step?
3- In top of the tutorial is written “you can just grab my profile”
Can I skip all the steps for even first node, and do these below steps for all nodes (without doing anything else! none of the other steps!)
(first installing apparmor-utils which is not mentioned in “Copy the profile” section):

apt install apparmor-utils
cd /etc/apparmor.d
wget https://gist.githubusercontent.com/y-trudeau/dc62a324817df643eae0ccb39d719c91/raw/94e7c4416de0eb26cbd29fe638a6740a70b90485/usr.bin.mysqld_safe
aa-enforce usr.bin.mysqld_safe

[tom256]

UPDATE2: My server is broken after trying to install PXC! Urgent help is needed!

I was installing Percona XtraDB Cluster by this tutorial:
https://www.percona.com/doc/percona-xtradb-cluster/LATEST/install/apt.html#apt

after installing with: [PHP] sudo apt-get install percona-xtradb-cluster-57 [/PHP]
it showed me it is going to install some packages and removing some packages like mysql-server:

The following packages were automatically installed and are no longer required: libcgi-fast-perl libcgi-pm-perl libevent-core-2.0-5 libfcgi-perl libhtml-template-perl linux-headers-4.4.0-131 linux-headers-4.4.0-131-generic linux-image-4.4.0-131-generic linux-image-extra-4.4.0-131-generic Use 'sudo apt autoremove' to remove them. The following additional packages will be installed: debsums iproute libdbd-mysql-perl libdbi-perl libdpkg-perl libev4 libfile-fcntllock-perl libfile-fnmatch-perl libmecab2 percona-xtrabackup-24 percona-xtradb-cluster-client-5.7 percona-xtradb-cluster-common-5.7 percona-xtradb-cluster-server-5.7 qpress socat Suggested packages: libclone-perl libmldbm-perl libnet-daemon-perl libsql-statement-perl debian-keyring tinyca pv The following packages will be REMOVED: mysql-client-5.7 mysql-client-core-5.7 mysql-server-5.7 mysql-server-core-5.7 The following NEW packages will be installed: debsums iproute libdbd-mysql-perl libdbi-perl libdpkg-perl libev4 libfile-fcntllock-perl libfile-fnmatch-perl libmecab2 percona-xtrabackup-24 percona-xtradb-cluster-57 percona-xtradb-cluster-client-5.7 percona-xtradb-cluster-common-5.7 percona-xtradb-cluster-server-5.7 qpress socat 0 upgraded, 16 newly installed, 4 to remove and 0 not upgraded. Need to get 19.9 MB of archives. After this operation, 18.9 MB of additional disk space will be used. Do you want to continue? [Y/n] Y 

I’ve entered yes, and then it asked me there are some data in /var/lib/mysql ,backup it, and I cp the directory to somewhere else in my server, then it asked me the new mysql root password, which I skiped and entered nothing.
This is the installation log:

Preconfiguring packages ... /usr/bin/locale: Cannot set LC_ALL to default locale: No such file or directory Selecting previously unselected package libdpkg-perl. (Reading database ... 153043 files and directories currently installed.) Preparing to unpack .../libdpkg-perl_1.18.4ubuntu1.4_all.deb ... Unpacking libdpkg-perl (1.18.4ubuntu1.4) ... Selecting previously unselected package libfile-fnmatch-perl. Preparing to unpack .../libfile-fnmatch-perl_0.02-2build2_amd64.deb ... Unpacking libfile-fnmatch-perl (0.02-2build2) ... Selecting previously unselected package debsums. Preparing to unpack .../archives/debsums_2.1.2_all.deb ... Unpacking debsums (2.1.2) ... Processing triggers for man-db (2.7.5-1) ... Setting up libdpkg-perl (1.18.4ubuntu1.4) ... Setting up libfile-fnmatch-perl (0.02-2build2) ... Setting up debsums (2.1.2) ... locale: Cannot set LC_ALL to default locale: No such file or directory Selecting previously unselected package percona-xtradb-cluster-common-5.7. (Reading database ... 153202 files and directories currently installed.) Preparing to unpack .../percona-xtradb-cluster-common-5.7_5.7.22-29.26-1.xenial_amd64.deb ... Unpacking percona-xtradb-cluster-common-5.7 (5.7.22-29.26-1.xenial) ... dpkg: mysql-server-5.7: dependency problems, but removing anyway as you requested: mysql-server depends on mysql-server-5.7. (Reading database ... 153212 files and directories currently installed.) Removing mysql-server-5.7 (5.7.23-0ubuntu0.16.04.1) ... locale: Cannot set LC_ALL to default locale: No such file or directory locale: Cannot set LC_ALL to default locale: No such file or directory update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode Removing mysql-server-core-5.7 (5.7.23-0ubuntu0.16.04.1) ... Processing triggers for man-db (2.7.5-1) ... Setting up percona-xtradb-cluster-common-5.7 (5.7.22-29.26-1.xenial) ... update-alternatives: using /etc/mysql/percona-xtradb-cluster.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode Selecting previously unselected package percona-xtradb-cluster-server-5.7. (Reading database ... 153041 files and directories currently installed.) Preparing to unpack .../percona-xtradb-cluster-server-5.7_5.7.22-29.26-1.xenial_amd64.deb ... . Unpacking percona-xtradb-cluster-server-5.7 (5.7.22-29.26-1.xenial) ... dpkg: error processing archive /var/cache/apt/archives/percona-xtradb-cluster-server-5.7_5.7.22-29.26-1.xenial_amd64.deb (--unpack): trying to overwrite '/usr/share/man/man1/mysqlman.1.gz', which is also in package mysql-client-5.7 5.7.23-0ubuntu0.16.04.1 dpkg-deb: error: subprocess paste was killed by signal (Broken pipe) Processing triggers for man-db (2.7.5-1) ... Errors were encountered while processing: /var/cache/apt/archives/percona-xtradb-cluster-server-5.7_5.7.22-29.26-1.xenial_amd64.deb E: Sub-process /usr/bin/dpkg returned an error code (1) 

And now I can’t even login to a DB with PHPmyadmin.
Mysql service is stopped and all my website on my server are down

what happend? How to fix it?

Please help.

[lorraine.pocklington]

Hello there
Sorry to hear that you are in this predicament.
Could you upload your my.cnf files from all nodes please?
Also any error logs that you are receiving.
Thanks

[tom256]

UPDATE 3:
If you pay attention to my previous logs I gave, it seems that there was a packages conflicting when installing PXC. after I occurred that Error, I tried apt-get upgrade , then I got:

Fetched 1372 kB in 1s (1274 kB/s)
Reading package lists… Done
Reading package lists… Done
Building dependency tree
Reading state information… Done
You might want to run ‘apt-get -f install’ to correct these.
The following packages have unmet dependencies:
mysql-server : Depends: mysql-server-5.7
E: Unmet dependencies. Try using -f.

real 0m0.294s
user 0m0.284s
sys 0m0.000s

So I tried: apt-get -f install

It installed some lost packages like mysql-server-5.7
Then I tried to install PXC again by --force-overwrite option:

sudo apt-get -o Dpkg::Options::="--force-overwrite" install percona-xtradb-cluster-57

Now PXC is installed and fortunately my websites are up (but with a problem I’ll mention later! DB is read-only now!)

then I didn’t completed the next steps of this tutorial:
https://www.percona.com/doc/percona-xtradb-cluster/LATEST/install/apt.html#apt

it told $ sudo service mysql stop but I started it! to see if my website can be back online again!
I’m not interested to continue the installation! I’m very disappointed for installing PXC, disappointed a lot!
I’m now scared to broke my server more and more! I don’t know how to uninstall PXC and revert everything to my default mysql, it seems a lot of ppl out there have the same problem and unanswered yet:
https://dba.stackexchange.com/questions/210568/dpkg-error-uninstall-percona-xtradb-server-5-7

Now I have several problems, PHPmyadmin is full of errors, it shows me my previous databases but it can’t even show me the Global variables.
A more severe problem is that my PHP applications and websites are able to only read from DB, when I try to write or update something, I get:

#1105 - Percona-XtraDB-Cluster prohibits use of DML command on a table ( * ) that resides in non-transactional storage engine with pxc_strict_mode = ENFORCING or MASTER

It seems that it’s because I didn’t complete the installation and my single node is not able to do WRITE operation:
https://www.percona.com/doc/percona-xtradb-cluster/LATEST/features/pxc-strict-mode.html

So at final, please help me! What Should I do now?
1- Uninstall Percona-XtraDB-Cluster and using the previous mysql? How can it be done?
2- still using it with this 1 single node mode? How? SET GLOBAL pxc_strict_mode=PERMISSIVE; or MASTER ?
3- Completing the installation steps for all 3 nodes? (As I said I’m very disappointed, I don’t want to see these errors anymore in all my other nodes! I Prefer saving my life instead of HA by 3node!)

Or is there anything wrong in my installation tasks? I don’t think so! Because I just entered:
sudo apt-get install percona-xtradb-cluster-57 PXC itself broke everything!

[tom256]

@lorraine.pocklington;52408

Hello there
Sorry to hear that you are in this predicament.
Could you upload your my.cnf files from all nodes please?
Also any error logs that you are receiving.
Thanks

I haven’t installed PXC on other nodes yet! Fortunately!

/etc/mysql/my.cnf

The Percona XtraDB Cluster 5.7 configuration file.

* IMPORTANT: Additional settings that can override those from this file!

The files must end with ‘.cnf’, otherwise they’ll be ignored.

Please make any edits and changes to the appropriate sectional files

included below.

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/percona-xtradb-cluster.conf.d/

/etc/mysql/percona-xtradb-cluster.conf.d/mysqld.cnf

# Template my.cnf for PXC
# Edit to your requirements.
[mysqld]
server-id=1
datadir=/var/lib/mysql
socket=/var/run/mysqld/mysqld.sock
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
log-bin
log_slave_updates
expire_logs_days=7

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

thanks for your help.

[lorraine.pocklington]

Hello there Tom

Yes the original install failed which can be seen from this:

dpkg: error processing archive /var/cache/apt/archives/percona-xtradb-cluster-server-5.7_5.7.22-29.26-1.xenial_amd64.deb
(--unpack): trying to overwrite '/usr/share/man/man1/mysqlman.1.gz', which is also in package mysql-client-5.7 5.7.23-0ubuntu0.16.04.1
dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)

The team here say that it looks like on that node you have the mysql-client installed, and this conflicts with Percona XtraDB Cluster. Meanwhile I will see if any of the team are free to add to this.

[lorraine.pocklington]

Hello Tom256, I have had someone look at the whole of your post now. Unfortunately there is no way to unpick your situation and you need to start from a fresh install.

You can’t have Oracle MySQL and Percona XtraDB Cluster code resident on the same machine, and this may have been the start of your issues here. Hindsight is easy, I know, but if you can spin up a development/test environment somewhere to try things out that would be the best thing to do.

I will refer the documentation / advice about AppArmor to the team for review, to see if there is more clarity we can add there.

[tom256]

@lorraine.pocklington;52415

Hello there Tom

Yes the original install failed which can be seen from this:

Yes, As I’ve mentioned before there was a packages conflict in for writing in ‘/usr/share/man/man1/mysqlman.1.gz’
I’ve solved this problem by “–force-overwrite”
[PHP]sudo apt-get -o Dpkg::Options::="–force-overwrite" install percona-xtradb-cluster-57[/PHP]

@lorraine.pocklington;52415

The team here say that it looks like on that node you have the mysql-client installed, and this conflicts with Percona XtraDB Cluster.

Yes, I think so, the vestacp may installed it before. By the way being mysql-client installed or not is something PXC installation process must check it before installing anything.
Please tell the team to check it, Can they install PXC on a vestacp server?

@lorraine.pocklington;52416

You can’t have Oracle MySQL and Percona XtraDB Cluster code resident on the same machine, and this may have been the start of your issues here.

Now phpMyAdmin shows me I’m using Percona Server, So I don’t think I’m using MySQL anymore, Is there any way to fix this single node and use it again? for example by using:
[PHP]SET GLOBAL pxc_strict_mode=MASTER;[/PHP]

@lorraine.pocklington;52416

I will refer the documentation / advice about AppArmor to the team for review, to see if there is more clarity we can add there.

I fixed the AppArmor issue by this commands:

sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/

sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld

Only disabling MySQL profile is better than removing the whole AppArmor which was mentioned in PXC’s installation tutorial!