Hi,
I’ve configured a pxc cluster with keyring vault enable via PXC Operator
Vault is available, and pxc pods can write keys in the secret path
At start up of the first pxc pod, some keys are written in Vault, but then the pod crashes with the following log :
{“log”:“2023-12-01T09:59:45.224081Z 0 [Warning] [MY-013712] [Server] No suitable ‘keyring_load’ service implementation found to fulfill the request.\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:45.238191Z 0 [Warning] [MY-011197] [Server] Plugin keyring_vault reported: ‘Probing mt-rke2percona-tst for being a mount point successful - identified kv-v2 secret engine.’\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:45.256559Z 0 [Warning] [MY-048031] [InnoDB] Setting Percona-specific INNODB_PARALLEL_DBLWR_ENCRYPT is deprecated and has no effect.\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:45.274082Z 0 [System] [MY-011197] [Server] Plugin keyring_vault reported: ‘Vault Server response data is not an Object’\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:45.274108Z 0 [System] [MY-011197] [Server] Plugin keyring_vault reported: ‘Could not read key from Vault.’\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:45.288819Z 3 [System] [MY-013576] [InnoDB] InnoDB initialization has started.\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:45.957707Z 3 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.186689Z 3 [Note] [MY-000000] [WSREP] wsrep_init_schema_and_SR (nil)\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.279646Z 3 [System] [MY-000000] [WSREP] PXC upgrade completed successfully\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.837698Z 0 [System] [MY-011197] [Server] Plugin keyring_vault reported: ‘Vault Server response data is not an Object’\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.837743Z 0 [System] [MY-011197] [Server] Plugin keyring_vault reported: ‘Could not read key from Vault.’\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.869275Z 0 [System] [MY-011197] [Server] Plugin keyring_vault reported: ‘Vault Server response data is not an Object’\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.898417Z 0 [System] [MY-011197] [Server] Plugin keyring_vault reported: ‘Could not read key from Vault.’\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.898478Z 0 [ERROR] [MY-013285] [Repl] Failed to store key, please check if keyring is loaded.\n”,“file”:“/var/lib/mysql/mysqld-error.log”} {“log”:“2023-12-01T09:59:46.898509Z 0 [ERROR] [MY-013288] [Server] Failed to initialize binlog encryption, please check if keyring is loaded.\n”,“file”:“/var/lib/mysql/mysqld-error.log”}
Here is the vault policy used :
path “mt-rke2percona-tst/config”
{ capabilities = [“read”] }
path “mt-rke2percona-tst/metadata/pxc-cluster2/*”
{ capabilities = [“list”] }
path “mt-rke2percona-tst/data/pxc-cluster2/*”
{ capabilities = [“create”, “read”, “delete”, “update”, “list”] }
Here is the vault secret used in the CRD pxc-cluster for vault secret :
keyring_vault.conf :
token =
vault_url = https://
secret_mount_point = mt-rke2percona-tst/pxc-cluster2/keyring-vault-keys
secret_mount_point_version = 2
vault_ca = /etc/mysql/vault-keyring-secret/ca.cert
The secret-engine is kv-2.
thank you for the help that you could provide