Too much memory consumption with myRocks engine

MySQL & MariaDB
1

Hi all,

I have some concerns about the memory usage of myrocks. I have a server with a percona mysql server version 8.0.29 and all production tables are using MyRocks. The specific configuration of myrocks has not been modified for the moment. This environment does not have a high usage, due to it contains some historical data, but I have realized that memory usage grows continuosly, also swap, and finally, two days ago it crashed due to an OOM.
This situation started after doing a backup from some tables with mysqldump. During the backup, the usage of memory grew up a lot and, some time later, the oom took place. I have added 4 GB of RAM (now it has 12) but after another backup, it has grown up too and it has barely freed space since yesterday and after every query it grows (Normally queries are a SUM of values from a table)

I cannot find any myrocks configuration parameter related to this memory problem. Could you please provide any clue about how to identify the problem or how to reduce the memory usage? when using InnoDB this problem was usually solved by adjusting innodb_buffer_pool_size, but I cannot locate anything here.

Thanks in advance.
Regards

2

Hello,
Can you SELECT * FROM sys.memory_global_by_current_bytes during one of these events and see what is using all the memory? MyRocks and InnoDB will compete for memory so make sure you have reduced the InnoDB buffer pool down to minimal levels if you plan to be 100% MyRocks. rocksdb_block_cache_size is the approximate equivalent to InnoDB buffer pool.

1 Like
3

Hi Matthew,
thank you very much for yor reply. I have already checked innodb_buffer_pool_size, but the DB has the default settings and it is set in 128MB with just one instance :slight_smile:

After doing some research, I found that there is some kind of issue related to the library glibc malloc, which seems not to manage memory well and causes oom.

Finally yesterday I installed the library in a test DB and noticed a better memory management. Today I have set it in production and, for the moment, RAM consumption is still reasonable. I am going to monitor it these days and will reply if everything is fine.

Thank you very much.

Kind regards

1 Like
4

I would suggest you take a look at the jemalloc library. It is a very popular replacement for the standard glibc.

1 Like
5

Hi Matthew,
sorry, I forgot saying that the library I have installed is jemalloc, which seems to solve this issue. For the moment it is working fine.
Thanks for your attention.
Kind regards

1 Like