Innodb bufferpool continuous flush storm

I am using 5.7.28-31-57-log - Percona XtraDB Cluster (GPL), Release rel31 in production enviroment

one of the nodes started for no apparent reason agresively flushing bufferpool …

what I see as strange is that only file under pressure xb_doublewrite but no other file have heavy IO

EDIT:
innodb_adaptive_flushing_lwm is at 15
disabling innodb_adaptive_flushing had no effect
incereasing innodb_buffer_pool_size by several GB had no effect

If somebody met similar issue … I would be happy for any pointers where to look further

1 Like

I suspect InnoDB detected idle state and decided to flush dirty pages. Can you show charts for InnoDB buffer pool dirty pages during this time?

1 Like


you are right all dirty pages are gone … problem is that only xb_doublewrite is stressed, .idb data files are not stressed at all…

Bytes Filename

30.97M /xtradbcluster/xb_doublewrite
1.04M /innodblogs/ib_logfile2
17.89KB /foo-slow.log
16.00KB /tablespacefilex.ibd
8.57KB /proc/stat
8.00KB /tablespacefiley.ibd
2.51KB /proc/mounts
2.50KB /proc/vmstat
2.33KB /proc/net/netstat
2.08KB /usr/local/percona/pmm-client/textfile-collector/example.prom
^C

normaly I have folowing options:
innodb_io_capacity=1000
innodb_io_capacity_max=2000
innodb_purge_batch_size=4608
innodb_adaptive_flushing_lwm=15

so innodb IO is at innodb_io_capacity_max … when I attempted to change innodb_io_capacity to 100 nothing changed

1 Like