RHEL9 XtraDB Cluster 8 Generic Binaries Segfault

MySQL & MariaDB Percona XtraDB Cluster 8.x
1

We have a proprietary internal server monitoring system that uses XtraDB cluster. Previously, we were running on OEL7 with version 5.6 of Percona. We’re looking at new installations on RHEL9 and we’re trying to duplicate what we were doing before with XtraDB 8.

We need pxc to be isolated to our application with a minimal of system level components installed. So we use the generic linux tarballs and unpack them to our own folder, create our inititalization files then attempt to initialize the db, but mysqld is causing an immediate segfault. I’ve run ldd on the mysqld binary and all the dynamic link libraries are satisfied so I’m guessing it is probably some system level component(s) that are missing.

What I’m trying to use install and use is the Percona-XtraDB-Cluster_8.0.34-26.1_Linux.x86_64.glibc2.34-minimal.tar.gz. We are indeed using glibc 2.34.

I started up a docker container with just our company base RHEL9 image and installed the various packages that we install during our normal installation. I then downloaded the RHEL9 RPMs for the same version of pxc. I then ran dnf install to just so I could see the dependency list it wants to install. I didn’t go through with the installation, I just wanted to see the list of dependencies.

First question is why is qpress still necessary? I thought I read that it has been deprecated but the RPM’s are still requiring it. I installed your repo and installed that single package so I could get where I wanted to be. Is there any way to get that package in another way via download? I really don’t want to have to configure the Percona repo and install that at the system level if it truly is going to be necessary for our installation.

That aside, the next question is based on the following list of dependencies that the RPMs wanted to install. Does someone know if all of these should be required for us to run our own standalone minimal version of pxc, or are there just a few key components we need?

Last metadata expiration check: 0:12:20 ago on Tue Nov  7 13:08:17 2023.
Dependencies resolved.
=================================================================================================================================================================
 Package                                                 Architecture             Version                               Repository                          Size
=================================================================================================================================================================
Installing:
 percona-xtradb-cluster-client                           x86_64                   8.0.34-26.1.el9                       @commandline                       2.9 M
 percona-xtradb-cluster-icu-data-files                   x86_64                   8.0.34-26.1.el9                       @commandline                       2.2 M
 percona-xtradb-cluster-server                           x86_64                   8.0.34-26.1.el9                       @commandline                       101 M
     replacing  mariadb-connector-c-config.noarch 3.2.6-1.el9_0
 percona-xtradb-cluster-shared                           x86_64                   8.0.34-26.1.el9                       @commandline                       1.4 M
Installing dependencies:
 checkpolicy                                             x86_64                   3.5-1.el9                             rhel-9-appstream                   351 k
 diffutils                                               x86_64                   3.7-12.el9                            rhel-9-baseos                      402 k
 libev                                                   x86_64                   4.33-5.el9                            rhel-9-baseos                       56 k
 libselinux-utils                                        x86_64                   3.5-1.el9                             rhel-9-baseos                      188 k
 libtirpc                                                x86_64                   1.3.3-2.el9                           rhel-9-baseos                       96 k
 mariadb-connector-c                                     x86_64                   3.2.6-1.el9_0                         rhel-9-appstream                   203 k
 perl-DBD-MySQL                                          x86_64                   4.050-13.el9                          rhel-9-appstream                   151 k
 perl-DBI                                                x86_64                   1.643-9.el9                           rhel-9-appstream                   729 k
 perl-DynaLoader                                         x86_64                   1.47-480.el9                          rhel-9-appstream                    27 k
 perl-Math-BigInt                                        noarch                   1:1.9998.18-460.el9                   rhel-9-appstream                   194 k
 perl-Math-Complex                                       noarch                   1.59-480.el9                          rhel-9-appstream                    49 k
 policycoreutils                                         x86_64                   3.5-2.el9                             rhel-9-baseos                      242 k
 policycoreutils-python-utils                            noarch                   3.5-2.el9                             rhel-9-appstream                    82 k
 python3-audit                                           x86_64                   3.0.7-104.el9                         rhel-9-appstream                    85 k
 python3-distro                                          noarch                   1.5.0-7.el9                           rhel-9-appstream                    40 k
 python3-libselinux                                      x86_64                   3.5-1.el9                             rhel-9-appstream                   192 k
 python3-libsemanage                                     x86_64                   3.5-2.el9                             rhel-9-appstream                    82 k
 python3-policycoreutils                                 noarch                   3.5-2.el9                             rhel-9-appstream                   2.1 M
 python3-setools                                         x86_64                   4.4.3-1.el9                           rhel-9-baseos                      608 k
 rpm-plugin-selinux                                      x86_64                   4.16.1.3-25.el9                       rhel-9-baseos                       18 k
 selinux-policy                                          noarch                   38.1.23-1.el9                         rhel-9-baseos                       60 k
 selinux-policy-targeted                                 noarch                   38.1.23-1.el9                         rhel-9-baseos                      6.8 M
 socat                                                   x86_64                   1.7.4.1-5.el9                         rhel-9-appstream                   309 k

We actually build our own version of socat and place it in the pxc/bin folder, so we won’t need to install that to the system.

2

qpress is needed for compatibility with pxc-57.
Could you please share an error from mysql log when sigfault happened?

3

Since we are using pxc 8, that’s why I wondered why qpress was still required. In answer to your question, the program segfaults before it has a chance to write anything to the MySQL log file, so there’s nothing there.

Since I posted originally, I have done the following:
I started with one of our RHEL9 base images, then added the percona repo and installed pxc 8 via RPM without any of our additional files. After that, I was able to run mysqld --version without a segfault and it did indeed output the version in use as expected.

However, if I take that same clean base image and then unpack the generic pxc tarball Percona-XtraDB-Cluster_8.0.34-26.1_Linux.x86_64.glibc2.34.tar.gz to a folder like /pxctest/pxc then add the /pxctest/pxc/bin folder to the path and try to run mysqld I get the segfault. I then install every dependency that the RPM install added via dnf, but still get the segfault. mysql also generates a segfault. I’ve tried to run strace, but it segfaults so quickly there’s no output from strace except the segfault. I’ve installed the source RPMs and tried to run gdb, but because it’s running under Rosetta on an M2 mac, that doesn’t work. There’s a workaround for that where you can tell Rosetta to open a port for debugging and launch your app, then try to connect via gdb from another terminal, but it segfaults when launched from Rosetta directly even though it’s supposed to hold the execution at the entry point until a gdb connection is established.

At this point, I’m just stuck and don’t know what else to try. It’s important that we be able to just unpack the pxc binaries to a folder structure and run it manually as we’ve been doing with 5.6 for years under EL6/7. If I could just figure out what it’s missing or what it thinks it needs that causes this immediate segfault.

As far as qpress, is there any way to download that as a tarball. I’ve looked through most of the things on the download page and I don’t see any way to get it other than to install the Percona repo and install it via dnf. I want to try and avoid that if possible.

4

This appears to be an issue with Rosetta on my M2 Macbook. When turning off Rosetta support in docker and trying this again, the MySQL binaries do not segfault anymore. However, it doesn’t explain why using the RPM’s to install didn’t exhibit the problem. But for now we have a workaround.