# Alleged memory leak in MySQL master on a huge database **URL:** \ **Category:** MySQL & MariaDB\ **Created:** [June 18, 2024, 10:59am UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069 "2024-06-18T10:59:49Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![mag1kan1n](https://avatars.discourse-cdn.com/v4/letter/m/22d042/32.png) [@mag1kan1n](https://forums.percona.com/u/mag1kan1n)\ **Post date:** [June 18, 2024, 10:59am UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/1 "2024-06-18T10:59:49Z")
Hi everyone. I am having a trouble with the MySQL installation that causes slow but steady memory consumption increase on the production database. I am attaching the config and a screenshot that shows MySQL memory usage has been increasing by 20GB over the course of the last two days. Eventually I will have to restart the MySQL (as I have done many times by now) which will cause some downtime as the database size is over 850 GB and it usually takes about half an hour to restart. It would be ideal to figure what causes this but I have no idea where to start. I would grealy appreciate any help. Thanks! ![Screenshot 2024-06-18 at 14.13.51](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/3X/1/9/1994c6ed6713ff54511489db5dbe02277a877b18.png) ```auto [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock bind-address = 0.0.0.0 user=mysql symbolic-links=0 log_timestamps = SYSTEM log_error = /var/log/mysql/error.log log_bin_trust_function_creators = 1 sql_mode = "ERROR_FOR_DIVISION_BY_ZERO,NO_ZERO_IN_DATE,NO_ENGINE_SUBSTITUTION" log_bin = binlog log_bin_index = binlog_list max_binlog_size = 1G expire_logs_days = 5 binlog_row_image = MINIMAL binlog-do-db = admin server-id = 1 max_connections = 10000 open_files_limit = 10000 wait_timeout=150 interactive_timeout=7200 back_log = 4096 key_buffer_size = 512M max_allowed_packet = 1024M table_open_cache = 20000 #table_open_cache = 64 table_definition_cache = 10000 sort_buffer_size = 2M read_buffer_size = 1M read_rnd_buffer_size = 1M net_buffer_length = 32K group_concat_max_len = 4K join_buffer_size = 320M max_join_size = 1024M tmp_table_size = 4G max_heap_table_size = 8G thread_cache_size = 80 thread_stack = 4M innodb_buffer_pool_size = 450G innodb_buffer_pool_instances = 80 innodb_log_file_size = 6G innodb_log_buffer_size = 64M innodb_lock_wait_timeout = 60 innodb_file_per_table = 1 innodb_page_cleaners = 8 innodb_read_io_threads = 32 innodb_write_io_threads = 8 innodb_flush_method = O_DSYNC innodb_flush_log_at_trx_commit = 2 innodb_io_capacity = 100 skip-external-locking slow_query_log=ON long_query_time = 0 log_output=FILE slow_query_log_file = /var/log/mysql/slow.log log_slow_admin_statements=OFF log_slow_slave_statements=OFF log_slow_rate_limit=100 log_slow_rate_type='query' slow_query_log_always_write_time=1 log_slow_verbosity='full' slow_query_log_use_global_control='all' log_slow_extra=ON innodb_temp_data_file_path=ibtmp1:100M:autoextend:max:10G ``` ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [June 18, 2024, 1:29pm UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/2 "2024-06-18T13:29:15Z")
Is this MySQL 5.7 or 8.0? ---
**Author:** ![mag1kan1n](https://avatars.discourse-cdn.com/v4/letter/m/22d042/32.png) [@mag1kan1n](https://forums.percona.com/u/mag1kan1n)\ **Post date:** [June 19, 2024, 7:04am UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/3 "2024-06-19T07:04:08Z")
Hi Matthew. It’s 8.0. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [June 19, 2024, 5:17pm UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/4 "2024-06-19T17:17:05Z")
Are you closing connections after each query/txn or are you using a connection pooler which is keeping connections open for extended periods of time? ---
**Author:** ![mag1kan1n](https://avatars.discourse-cdn.com/v4/letter/m/22d042/32.png) [@mag1kan1n](https://forums.percona.com/u/mag1kan1n)\ **Post date:** [June 24, 2024, 12:10pm UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/5 "2024-06-24T12:10:57Z")
Thank you for your reply. We use PDO for PHP to work with the database, and it does close connections. I am attaching the graph that shows that connection count varies over the course of the day (the graph depicts last 7 days) while memory consumption steadily grows, so it doesn’t seem to be the issue. ![Screenshot 2024-06-24 at 16.08.11](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/3X/6/b/6be3820917302bb77bf8573fe29f1536c4edc326.png) ---
**Author:** ![Vit\_Bartunek1](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vit_bartunek1/32/23500_2.png) [@Vit\_Bartunek1](https://forums.percona.com/u/Vit_Bartunek1)\ **Post date:** [April 7, 2026, 11:58am UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/6 "2026-04-07T11:58:44Z")
Hi, we observe the same behavior with our instances. Even after reducing the buffer pool in runtime, the memory of the entire mysql process gradually increases. It happens mainly on the instance that has read\_only=0. On slave instances it is growing much more slowly. The blue line tag is time when we switched the instance to be writable and server as a master to others. ![Screenshot_20260407_135818](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/3X/5/0/50fdd87df666032ab8e3ee21db77943ef4fcf3fb.png) ---
**Author:** ![Vit\_Bartunek1](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vit_bartunek1/32/23500_2.png) [@Vit\_Bartunek1](https://forums.percona.com/u/Vit_Bartunek1)\ **Post date:** [April 7, 2026, 12:00pm UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/7 "2026-04-07T12:00:11Z")
Hello @mag1kan1n. Did you find out how to stabilize memory consumption? Thank you.. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [April 7, 2026, 3:40pm UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/8 "2026-04-07T15:40:31Z")
Hey @Vit_Bartunek1, There are several memory-related table inside the `sys` schema that I suggest you take a look at to see where exactly memory is being used, and not being free’d. I would also recommend that you try out the `jemalloc` memory manager. MySQL should auto-detect jemalloc when installed and load it automatically on restart. ---
**Author:** ![Vit\_Bartunek](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vit_bartunek/32/23499_2.png) [@Vit\_Bartunek](https://forums.percona.com/u/Vit_Bartunek)\ **Post date:** [April 9, 2026, 9:37am UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/9 "2026-04-09T09:37:21Z")
> [@matthewb](#): > > jemalloc Thank you for the tips. I restarted instances with jemalloc and will investigate sys tables on servers with the standard memory allocation method. ---
**Author:** ![Vit\_Bartunek](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vit_bartunek/32/23499_2.png) [@Vit\_Bartunek](https://forums.percona.com/u/Vit_Bartunek)\ **Post date:** [August 24, 2026, 11:09am UTC](https://forums.percona.com/t/alleged-memory-leak-in-mysql-master-on-a-huge-database/31069/10 "2026-08-24T11:09:06Z")
FYI The memory in glibc allocator was not returned to the kernel. Switching to jemalloc and settings background\_thread:true solved the issue. Memory footprint is stable. @matthewb Thank you very much for informations! Systemd override settings: `[Service]` `Environment="MALLOC_CONF=prof:true,prof_active:false,lg_prof_sample:19,background_thread:true"`