# Poor Galera performance
**URL:** https://forums.percona.com/t/poor-galera-performance/3650
**Category:** Percona XtraDB Cluster 5.x
**Created:** [August 2, 2014, 2:57pm UTC](https://forums.percona.com/t/poor-galera-performance/3650 "2014-08-02T14:57:11Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Sergey\_Kurkin](https://forums.percona.com/u/Sergey_Kurkin)
#### Post date: [August 2, 2014, 2:57pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/1 "2014-08-02T14:57:11Z")
Hi,
I’m new to Galera, however I was able to setup 3 node cluster
(using MariaDB or Percona). My problem is that when I go from single node to Galera (for example
single node with wsrep\* in my.cnf) performance have decreased about 10 fold (without wsrep node
1000% faster).
What may be wrong with my Galera setup ?
thank you,
Sergey
---
### Author:  [@dbennett455](https://forums.percona.com/u/dbennett455)
#### Post date: [August 4, 2014, 4:17pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/2 "2014-08-04T16:17:05Z")
Hi Sergey,
Can you post your my.cnf/wsrep.cnf configurations?
–Dave
---
### Author:  [@Sergey\_Kurkin](https://forums.percona.com/u/Sergey_Kurkin)
#### Post date: [August 5, 2014, 1:32am UTC](https://forums.percona.com/t/poor-galera-performance/3650/3 "2014-08-05T01:32:21Z")
Of course. I forgot to say - this slow down only for write queries. Galera reading performance is good. In production environment I have good hardware (Xeon cpu, RAID and so on). I made experiments on Virtual machines (usual Celeron CPU, SATA HDD) and have found the same slow down at transition from stand-alone server to Galera.
[client]
port = 3306
socket = /var/run/mysqld/mysqld.sock
[mysqld\_safe]
socket = /var/run/mysqld/mysqld.sock
#nice = 0
[mysqld]
#CLUSTER
wsrep\_provider=/usr/lib/galera/libgalera\_smm.so
wsrep\_provider\_options=“gcache.size=2G; gcache.page\_size=1G;gcs.fc\_limit = 256; gcs.fc\_factor = 0.99;”
wsrep\_slave\_threads= 32
innodb\_doublewrite=1
query\_cache\_size=0
query\_cache\_type=0
wsrep\_node\_name=“Gandalf”
#wsrep\_node\_incoming\_address=10.0.0.1
#wsrep\_sst\_receive\_address=10.0.0.1
innodb\_locks\_unsafe\_for\_binlog=1
wsrep\_cluster\_address=gcomm://10.0.0.1,10.0.0.6,10.0.0.3
binlog\_format=ROW
auto\_increment\_offset = 1
auto\_increment\_increment = 1
wsrep\_auto\_increment\_control = 0
default\_storage\_engine=InnoDB
innodb\_autoinc\_lock\_mode=2
wsrep\_node\_address=10.0.0.1
wsrep\_sst\_method=xtrabackup-v2
wsrep\_cluster\_name=DB\_CLUSTER
wsrep\_sst\_auth=“sstuserroot:842655”
#wsrep\_sst\_method=rsync
wsrep\_replicate\_myisam=1
#GENERAL
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
lc-messages-dir = /usr/share/mysql
skip-name-resolve
default\_time\_zone=‘+04:00’
collation-server = utf8\_general\_ci
init-connect=‘SET NAMES utf8;’
#init-connect=‘SET GLOBAL sql\_mode=“”;’
#init-connect=‘SET SESSION sql\_mode=“”;’
#init-connect=‘SET sql\_mode=“”;’
default-storage-engine = InnoDB
character-set-server = utf8
bind-address = 0.0.0.0
#bind-address = 127.0.0.1
#THREADS
thread\_handling = pool-of-threads
#thread\_pool\_high\_prio\_mode = transactions
thread\_pool\_size = 16
thread-cache-size = 400
# MyISAM
key-buffer-size = 32M
myisam-recover = FORCE,BACKUP
# SAFETY
max-allowed-packet = 16M
max-connect-errors = 1000000
# CACHES AND LIMITS
tmp-table-size = 32M
max-heap-table-size = 32M
query-cache-type = 0
query-cache-size = 0
max-connections = 1000
open-files-limit = 65535
table-definition-cache = 4096
table-open-cache = 1900
# INNODB
innodb-log-files-in-group = 2
innodb-log-file-size = 128M
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table = 1
innodb-buffer-pool-size = 32G
# LOGGING
log-error = /var/log/mysql/mysql-error.log
#log-queries-not-using-indexes = 1
#slow-query-log = 1
#slow-query-log-file = /var/log/mysql/mysql-slow.log
[mysqldump]
quick
quote-names
max\_allowed\_packet = 16M
[mysql]
#no-auto-rehash # faster start of mysql but no tab completition
[isamchk]
key\_buffer = 16M
#
# \* IMPORTANT: Additional settings that can override those from this file!
# The files must end with ‘.cnf’, otherwise they’ll be ignored.
#
!includedir /etc/mysql/conf.d/
---
### Author:  [@dbennett455](https://forums.percona.com/u/dbennett455)
#### Post date: [August 5, 2014, 7:34pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/4 "2014-08-05T19:34:01Z")
Hi Sergey,
Thanks for posting. Here’s a couple of performance tips for starters:
- wsrep\_slave\_threads= 32
From the README-wsrep file, this number should be low per CPU core. To be safe, I’d say start with 2 threads per core, Use
cat /proc/cpuinfo | grep -c ‘^processor\s\*:’
and multiply by 2 for a starting value. You may want to reduce this to 1 thread per core or lower.
- wsrep\_provider\_options=“gcache.size=2G; gcache.page\_size=1G;gcs.fc\_limit = 256; gcs.fc\_factor = 0.99;”
Refer to this guide [url][http://www.fromdual.com/gcache\_size\_in\_galera\_cluster[/url]](http://www.fromdual.com/gcache_size_in_galera_cluster%5B/url%5D) for these values.
Also:
- rule out a network bandwidth issue, use the iperf tool ([url][http://iperf.sourceforge.net/[/url]](http://iperf.sourceforge.net/%5B/url%5D)) to measure the bandwidth between your nodes.
- make sure that values aren’t being overridden by configuration files in your /etc/mysql/conf.d directory.
---
### Author:  [@Sergey\_Kurkin](https://forums.percona.com/u/Sergey_Kurkin)
#### Post date: [August 5, 2014, 11:21pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/5 "2014-08-05T23:21:18Z")
Thank you, Dave.
All is ok on my system, including bandwith issue.
No one can explain me why Galera so slow on write queries…
(at least for query “insert into test.tmptable values (‘test1’,‘test2’,‘test3’)”)
---
### Author:  [@dbennett455](https://forums.percona.com/u/dbennett455)
#### Post date: [August 6, 2014, 1:05pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/6 "2014-08-06T13:05:24Z")
Hi Sergey,
Can you provide some general specifications on the write operation? table size, row length, column definition, indexes and avg/max write operations per second would help. I could put together a randgen simulation with your configuration above.
–Dave
---
### Author:  [@dbennett455](https://forums.percona.com/u/dbennett455)
#### Post date: [August 6, 2014, 8:41pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/7 "2014-08-06T20:41:46Z")
One other question, what is your overall transaction size?
---
### Author:  [@taka-h](https://forums.percona.com/u/taka-h)
#### Post date: [August 8, 2014, 2:06am UTC](https://forums.percona.com/t/poor-galera-performance/3650/8 "2014-08-08T02:06:26Z")
Hi
Within my environment, the most dominant tuning parameter was repl.commit\_order(although we may not change this parameter)
[url][Index of wsrep\_provider options](http://www.percona.com/doc/percona-xtradb-cluster/5.6/wsrep-provider-index.html#repl.commit_order%5B/url%5D)
This configuration seems to serialize the transaction in each replicated nodes, scalability for cuncurrent transaction may lost significantly
This occurred even when the cluster node is just one node.
To judge whether the problem is caused by galera cluster or not, just to turn off all the wsrep configuration(comment out or so on) and benchmark may help you.
thank you.
---
### Author:  [@Sergey\_Kurkin](https://forums.percona.com/u/Sergey_Kurkin)
#### Post date: [August 8, 2014, 3:37pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/9 "2014-08-08T15:37:11Z")
Thank you very much.
Yes, I made benchmarks (before posting) with VMWare environment (Ubuntu and so on).
Expert from SeveralNines made his benchmark with sysbench (olpt), also using VMWare and he
have got ~400% slow down.
So, from my experience I can’t view database clustering as good solution for performance increase.
For example if I will setup 4 nodes I will have performance of 1 node.
If I will setup for example 20 nodes I don’t know value of performance I will have. May be I will have 4xperformance of 1 node,
but may be less than 4x.
---
### Author:  [@taka-h](https://forums.percona.com/u/taka-h)
#### Post date: [August 8, 2014, 5:57pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/10 "2014-08-08T17:57:02Z")
Hi
- scaling for read traffic is very effective and easy(this almost depends on CPU resource)
- scaling for write seems difficult and I also need the effective and simple strategy.
thank you
---
### Author:  [@dbennett455](https://forums.percona.com/u/dbennett455)
#### Post date: [August 10, 2014, 1:16pm UTC](https://forums.percona.com/t/poor-galera-performance/3650/11 "2014-08-10T13:16:04Z")
In most scenarios, moving from a virtualized environment to dedicated hardware would be a logical first scaling step prior to considering multi-master replication. So a performance benchmark of a pxc/galera multi-master virtualized environment may not be practical. Aside from overall performace scaling, there is also the high availability use case in which vitalization may play a role. That being said, there are some tuning steps regarding threading and transaction size that may also be effecting your test results.
If you’re running a PXC cluster as a VM guest you should definitely reduce your slave threads to match 2x max cores (or lower) defined in the guest. Also, if you are running very large write transactions such as multiple inserts, updates or [B] to see if there’s a large memory allocation and/or swapping occurring.
There are some tuning parameters for this case. If a large writeset is created by a [B] which will set limits on the transaction size. These parameters will cause replication rejections which will be logged. If they are, you need to modify your code to split up the apps large writesets.
---
### Author:  [@Sergey\_Kurkin](https://forums.percona.com/u/Sergey_Kurkin)
#### Post date: [August 11, 2014, 2:40am UTC](https://forums.percona.com/t/poor-galera-performance/3650/12 "2014-08-11T02:40:33Z")
Thank you. It is very interesting.
---
### Author:  [@krunalbauskar](https://forums.percona.com/u/krunalbauskar)
#### Post date: [April 24, 2017, 11:30am UTC](https://forums.percona.com/t/poor-galera-performance/3650/13 "2017-04-24T11:30:19Z")
Check the new performance optimized pxc-5.7
[url][https://www.percona.com/blog/2017/04/19/performance-improvements-percona-xtradb-cluster-5-7-17/[/url]](https://www.percona.com/blog/2017/04/19/performance-improvements-percona-xtradb-cluster-5-7-17/%5B/url%5D)