Not able to scale PXC on kubernetes

MySQL & MariaDB Percona XtraDB Cluster 8.x
,
1

Hi there,

I tried to configure PCX with 15 nodes on AWS EKS, but it just started 5 pcx pods. My goal is to load a WordPress application on Kubernetes and log 5k user during 3 minutes. I did a load test but I wasn’t able run more than 500 requests simultaneously, I got timeout for 59% of the requests.

If someone has any hint to my help me out I appreciate.

1 Like
2

Hello @Amadeu_Bonfante,
First off, PXC will not support 15 nodes. Once you go more than 5, you will start to suffer performance penalty from all of the node synchronization.

A 3 node PXC will easily support over 20,000 QPS per node. You must ensure that your K8S can support pods utilizing 100% of each nodes CPU and Memory because it all must be allocated to MySQL. At minimum, such a configuration would require 32 CPUs and 96GB of memory per PXC node. Then you must also properly configured MySQL within the pod to utilize that memory for InnoDB’s buffer pool.

2 Likes
3

Hello Matthew,

Thanks for your quick answer, it clarified a little bit more my understand about kubernetes and databases. I’ll tried your recommendation.

1 Like
4

@Amadeu_Bonfante my 5 cents here. As @matthewb mentioned, once you go for more than 5 nodes you might face severe performance issues.

Our Operators have safe-guard measures, where we do not allow users to apply unsafe or harmful configurations. This is controlled with spec.allowUnsafeConfigurations flag. If it is set to true, you will not be able to spin up less than 3 or more than 5 nodes.

You can find more details here: Custom Resource options - Percona Operator for MySQL based on Percona XtraDB Cluster

Prevents users from configuring a cluster with unsafe parameters such as starting the cluster with the number of Percona XtdaDB Cluster instances which is less than 3, more than 5, or is an even number, with less than 2 ProxySQL or HAProxy Pods, or without TLS/SSL certificates (if false, unsafe parameters will be automatically changed to safe defaults)

If for some reason you want more than 5 nodes, just set spec.allowUnsafeConfigurations' to true`.

1 Like
5

Hi @Sergey_Pronin , Thanks for you reply.

I tried to follow what @matthewb propose and I decided and move my PCX setup outside Kubernetes.
I’m following the documentation Installing Percona XtraDB Cluster - Percona XtraDB Cluster, and I believed that I missed something that cannot figure it out.
I opened a new topic with what got so far here Extra Node on PCX not starting (not sure if it’s already available).

1 Like