Hi team, Good day, I’ve been spending over a month in percona documentation for installation of pxc-operator and pxc-db with proxysql, and haproxy recently, I have not find any document which has clear instructions that is end to end with k8s operator(from operator deployment to sysbench testing and result analysis) . All of them are related to non-k8s approaches. Could you please help me in finding the right documentation to make my life easier with percona setup. Please help me here.
To be specific:
when I implemented pxc-operator and pxc-db with proxysql (disable Tls true) then after testing with sysbench I could able to analyse the results but query rules are not syncing at the first attempt of insert, verify , load and save. I had to repeat several times to make it stick to table. And am not sure how to backup the configuration that I made from proxysql admin interface such as adding query rules. And also when I make the disable tls false then second pod of the pxc is going to crash loop by stating ssl issues.
That’s why I tried to use haproxy.
I implemented operator and pxc-db with haproxy I did the sysbench testing but don’t know what to analyse and what is the next step since I did not find any clear documentation. Percona has something but that was from 2012 and not related to operator, as of my understanding operator does all the configuration with haproxy during initialization by running scripts, and the default configuration taking from "percona-docker/haproxy-global.cfg at main · percona/percona-docker · GitHub " but this is not having anything on how to access statistics report url/haproxy/stats. Any help would be much appreciated.
Many thanks in advance.
I’ve run a lot of sysbench experiments and will try to help you, but I need to understand more what issues related to sysbench you have.
Are you able to prepare data and run some simple tests against operator?
Hi @vadimtk, yes i could able to do the sysbench testings (preparing the database and run the tests)with both proxysql and haproxy. But in-case of Haproxy, i dont really know how to analyse the results or where to find the stats. Where as in proxysql there is stat schema to look at the performance and read/write split through Proxysql admin interface.
And by default my Haproxy configuration is percona-docker/haproxy.cfg at main · percona/percona-docker · GitHub and percona-docker/haproxy-global.cfg at main · percona/percona-docker · GitHub. do i have to add anymore configuration manually or operator takes care of the configuration required?
Hi @vadimtk - i’ve attached my test results. and it looks like failover is not happening with haproxy. should i have to change any of haproxy config?
there are two scenarios
- tested with oltp_read_write.lua script and host is pxc-db-haproxy.pxc.svc.cluster.local
while the test is running deleted my primary(pxc-0) and the test got failed with error “Deadlock found when trying to get lock; try restarting transaction” for more details see the attachement.
- tested with oltp_read_only.lua and host is pxc-db-haproxy-replicas.pxc.svc.cluster.local
while the test is running deleted my replica(pxc-1) and the test got failed with error “WSREP has not yet prepared node for application use”.
haproxy-sysbench.txt (6.4 KB)
@ajay in this case I think failover happens, but sysbench is not instructed to ignore errors - and this will happen around failover, your application need to be ready to handle this.
There is a command line options in sysbench to ignore specific errors.
@vadimtk, i’ve passed this option “–mysql-ignore-errors=1062” and --skip-trx=true. is there any other option for ignoring the errors?
in your log I also see 1047 and 1213 errors, you may want to skip these
@vadimtk - I will try with --mysql-ignore-errors=1062, 1047, 1213 and let you know. Thank you so much for quick respnce.
@vadimtk - when i used --mysql-ignore-errors=1062, 1047, 1213 i got “FATAL: mysql_stmt_execute() returned error 2013 (Lost connection to MySQL server during query)” .
then i used “–mysql-ignore-errors=ALL” then the test got succeeded.
Now ,how can i see the stats regarding the tests that i’ve done(for eg: no.of queries directed to replicas when the primary was down during the test)? in case of proxysql i could able to check stats schema tables for how database reacted to these tests.