Single node PXC Fails to recover due to ibdata1 on NFS Storage

Hi All,

Looking for some insight here. I’m having issues with Percona recovering from power failures on a VSphere VM. We are running PXC with MySQL Innodb configured. Deploying using Helm into a K8s cluster. Whats happening is that when we test a power failure, the containers will restart and try to start PXC back up. The new container MySQL Service tries to connect to the PVC, and due to complications I have assumed is because us having used NFS as the storage backend, the db files are locked and Percona cannot recover. The error itself is that the ibdata1 file is locked, error 11. So MySQL cannot start up. This does not occur on other storage backend, just NFS. We are using NFSv4.

We can recover this by deleting the Percona Operator, the PXC deployments, and the pods, and then the datdir PVC thats mounted onto the pod which contains the MySQL data, however this is extremely bad design practice and we don’t want to go with this approach.

Is there any formal instruction on not using NFS storage for Percona XtraDB Cluster? We are wondering if due to MySQL warnings about using NFS and MySQL this is explaining our variability of PXC. We are using 5.7.

1 Like

Hello @WatermelonPXC,
It’s pretty common knowledge to not use NFS for anything database related due to the issues you are experiencing (among others). It is certainly part of our training material not to use NFS.

1 Like

Thanks @matthewb , yes this is what we figured after investigation. I will close this ticket. Thanks for sanity checking.

1 Like