xtrabackup has messed up my MySQL installation

I’m posting here at the recommendation of Lorraine@Percona, after a 1st post at StackExchange.

Last Thursday I experimented a bit with Percona xtrabackup on a MySQL 5.5.23 server (we’re stuck at this very old version, that’s another discussion). I installed the Percona repo as part of the task. I uninstalled the tool before the end of the day, but didn’t uninstall the repo.

On Friday I noticed that when I ran “mysql -u root -p” I got a segmentation fault!

I investigated and found (using the rpm -qa --last command) that at 5:19am Friday morning, when we were all asleep, somehow, Percona-Server-Shared was installed (by itself??), and /usr/lib64/libmysqlclient.so.16.0.0 is now part of that package, as I confirm with the rpm -qf command. I suspect that’s why the plain vanilla mysql client is dumping core, it can’t find the libraries it expects.

When I attempt to uninstall Percona server, I see that it cascades to remove the mysql-server package itself (!), so I can’t do it.

At StackExchange I was told that this is a known bug that plagues the 5.5.23 MySQL version and I received the recommendation to uninstall Percona-Server-Shared using the --nodeps option, and then install mysql-libs.

I must fix this problem ASAP, the server is a staging server (no I didn’t do it on a production server :slight_smile: ) It’s running Redhat 6.9.

Thanks for your help,

George

Hello George, thanks for joining the Forum. It’s not to obfuscate - I’ll happily update the StackExchange post if we can bottom this out - but many of our engineers don’t feedback on that site so you are more likely to get the answer over here.

OK … the advice on the last comment is what we’ll need to help you here, so that’s a good start. I’ve reproduced that here, though if you could provide the requested output that might move us forward. :

Please provide - output of

rpm -qa --last
  • output of the
yum install xtrabackup

libmysqlclient.so.16.0.0 is now part of that package, Percona no longer provides libmysqlclient.

This should fix:

rpm -ev --nodeps Percona-Server-shared
yum install mysql-libs 

As everyone else advised: - spin up free-tier aws - setup 5.5.23 - setup xtrabackup - remove xtrabackup - check if same problem - apply the steps suggested above.

Hi Lorraine,

Attached is the output of rpm -qa --last on the problematic host. I can’t go back far enough to show you the output of yum install xtrabackup.
I have launched a CentOS 6.9 AWS instance with MySQL 5.5.60 (I don’t think I can find 5.5.23). I installed the Percona repo and then xtrabackup-2.4 Then I removed xtrabackup. Now I’ll have to wait till tomorrow to see if the host (I guess through yum auto tasks) will install Percona Shared Server during the night. That’s what worries me, if this host does not run whatever it was that the other host ran with yum and therefore does not install PSS overnight, this test we’re running will fail. Do you know more about this, i.e. how PSS was installed “on its own”?

Thanks,
George

rpm -qa --last |grep Perco

Percona-Server-shared-51-5.1.73-rel14.12.625.rhel6.x86_64 Fri 01 Jun 2018 03:19:49 AM BST

perc.txt (93.2 KB)

Hi George, thanks for the update.
I can’t personally advance this (not enough knowledge), but I will bring it to the attention of the tech team, and hopefully they will have some insight for you on this issue.
What is your current situation?
If you prefer you are welcome to email me direct with that info lorraine.pocklington@percona.com

My current situation is as I have described it above. If something is unclear I’ll be happy to clarify.

I’m looking forward to any feedback from your eng team.

Hi again George

OK, I have some feedback. The suggestions provided by Marcos and that I reproduced above are sound, and should help you out of this.

Unfortunately (or fortunately, whichever way you look at it) the version that you are trialling on, 5.5.60, won’t reproduce the issue since it was fixed in 5.5.26 (and your version I understand is 5.5.23). So to reproduce the problem, you’d need 5.5.23

To explain what’s happened: during the set up of XtraBackup that you did, there would have been some questions that you had to answer Y or N to, and one of those would have been whether to “replace mysql-libs with Percona-Server-shared-51-5.1.73-rel14.12.625.rhel6.x86_64” - we can see that from the logs. So that is likely to be where the problem arose.

In the original post on DBA Stack Exchange you mentioned that you were currently stuck with this old version of MySQL. If you you are in a position to and want to get help with that, our support or consultancy services might well be of use to you. Likewise if you need help on setting up a rigorous back up strategy for your current set up. I realize not everyone can or wants to do that, but I’m throwing it out there in case it’s of benefit.

Meanwhile, I hope the suggested steps work for you!

OK, thanks, I’ll try to find 5.5.23 for my trial.

You should be able to get it from here … https://downloads.mysql.com/archives/community/ … ?

Are you perhaps aware of a repository where old versions are archived? I can’t find it anywhere…

What if I go ahead and explicitly install using yum the PSS, wouldn’t I effectively reproduce the problem?

Oh I see I missed your post, yes I’ll get it from there tomorrow, thanks.

Good morning,
Here is my update:

I removed 5.5.60 and installed 5.5.23 from the link you kindly provided.
I ensured I can login as root with mysql client.
I then installed xtrabackup-24 (log below).
I was not asked to replace libs as you wrote above.
I checked and found I can still login as root with mysql client, there is no problem now.
Conclusion: no problem this time, the problematic situation was not replicated

yum install percona-xtrabackup-24

Loaded plugins: presto, security
Setting up Install Process
epel/metalink | 19 kB 00:00
percona-release-noarch | 2.9 kB 00:00
percona-release-x86_64 | 2.9 kB 00:00
remi-safe | 2.9 kB 00:00
ultra-centos-6.9-base | 3.4 kB 00:00
ultra-centos-6.9-extras | 3.3 kB 00:00
ultra-centos-6.9-updates | 3.4 kB 00:00
ultra-centos-6.x-glusterfs | 2.9 kB 00:00
ultra-centos-6.x-ultrarepo | 2.9 kB 00:00
Resolving Dependencies
–> Running transaction check
—> Package percona-xtrabackup-24.x86_64 0:2.4.11-1.el6 will be installed
–> Finished Dependency Resolution

Dependencies Resolved

================================================== ==============================
Package Arch Version Repository Size
================================================== ==============================
Installing:
percona-xtrabackup-24 x86_64 2.4.11-1.el6 percona-release-x86_64 8.1 M

Transaction Summary
================================================== ==============================
Install 1 Package(s)

Total download size: 8.1 M
Installed size: 8.1 M
Is this ok [y/N]: y
Downloading Packages:
Setting up and reading Presto delta metadata
Processing delta metadata
Package(s) data still to download: 8.1 M
percona-xtrabackup-24-2.4.11-1.el6.x86_64.rpm | 8.1 MB 00:00
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
Installing : percona-xtrabackup-24-2.4.11-1.el6.x86_64 1/1
Verifying : percona-xtrabackup-24-2.4.11-1.el6.x86_64 1/1

Installed:
percona-xtrabackup-24.x86_64 0:2.4.11-1.el6

Complete!

Could you let us have the output of this, please?

rpm -qa |egrep -i "mysql|maria|percona";

Thanks!