Percona Docker Image + RocksDB + Server Variables

MySQL & MariaDB
,
1

I’m writing a small application for fun that uses Docker Compose to orchestrate the deployment of an app and DB. I’m using the official Percona image and wanting to make use of the RocksDB storage engine (env variable INIT_ROCKSDB=1).

My issue seems to be that the database fails to initialize if it encounters RocksDB server variables in my.cnf that it doesn’t recognize, including variables related to plugins that it refuses to load during initialization.

Here’s a small compose.yaml file I can replicate the issue with:

services:
  db:
    image: percona:8.0
    restart: "no"
    ports:
      - "3306:3306"
    environment:
      - MYSQL_ALLOW_EMPTY_PASSWORD=1  # not a prod configuration
      - INIT_ROCKSDB=1
    volumes:
      - ./my.cnf:/etc/my.cnf

A simple my.cnf that includes rocksdb server variables:

[mysqld]
host_cache_size = 0
plugin-load-add=ha_rocksdb.so
rocksdb_db_write_buffer_size = 9600000000

And then logs that the resulting containers outputs:

db-1  | Initializing database
db-1  | 2024-06-14T16:03:27.175743Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.36-28) initializing of server in progress as process 14
db-1  | 2024-06-14T16:03:27.183376Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
db-1  | 2024-06-14T16:03:27.591317Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
db-1  | 2024-06-14T16:03:28.162971Z 0 [Warning] [MY-013501] [Server] Ignoring --plugin-load[_add] list as the server is running with --initialize(-insecure).
db-1  | 2024-06-14T16:03:29.412124Z 0 [ERROR] [MY-000067] [Server] unknown variable 'rocksdb_db_write_buffer_size=9600000000'.
db-1  | 2024-06-14T16:03:29.412136Z 0 [ERROR] [MY-013236] [Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
db-1  | 2024-06-14T16:03:29.412139Z 0 [ERROR] [MY-010119] [Server] Aborting
db-1  | 2024-06-14T16:03:31.932918Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.36-28)  Percona Server (GPL), Release 28, Revision 47601f19.

It appears to me that the problem is that MySQL refuses to load plugins on init, which makes sense, but will fail to initialize if it doesn’t recognize variables associated w/ plugins.

(Note: I’ve tested, and it doesn’t matter whether you give the superuser a password or not, just so no one thinks it’s associated w/ the MYSQL_ALLOW_EMPTY_PASSWORD environment variable.)

All that is understandable… I’m just curious if I can get the convenience of a declarative configuration of server variables for Percona+RocksDB without having to launch twice or run queries after launch. (I’ve attempted briefly, and the DB refused to let me set the server variables after things were running, but I didn’t try too hard.)

Any help is greatly appreciated!

2

@Matt_Youngberg,
If RocksDB has not yet loaded (such as on first init), then you need to prefix all rocksdb_ variables with loose- So it looks like loose-rocksdb_db_write_buffer_size=96M This tells mysql that the variable may be usable later on.

Side curiosity, why are you using RocksDB? Are you expecting high writes, or needing high compression? What’s making you pick this engine over the standard InnoDB engine?

3

@matthewb That worked! Thank you. I had tried loose before, but I either applied just dashes or just underscores to the value, I didn’t mix them, so that’s what it was.

I’m just barely below a write-performance threshold I’m trying to hit, and way over my requirement on read speed. It’s a super simple two-column table and the write is a simple INSERT. After some query analysis w/ InnoDB backing it, it wasn’t spending too much time with acquiring metadata locks or anything, it was literally just the write speed to the table that was creating the bulk of the time spent.

Admittedly, I’m just an app developer with not much exposure to DB configuration, but I just picked up a copy of “Efficient MySQL Performance” by Daniel Nitcher, and he mentioned RocksDB as a good backing engine for high writes and good SSD write management if you wanted to attempt that before changing how you handle inserts in the app or sharding.

4

Ha! For transparency, Daniel used to work at Percona. Yes, RocksDB is a good storage engine… when you’re above 20,000 writes per second. It’s a niche engine; not general purpose like InnoDB.

For simplicity, I would suggest that you stay with the default InnoDB until you start pushing the limits of InnoDB.

5

@matthewb It clicks now why he mentions Percona so much in the book! Haha.

That’s a good threshold to be aware of. I’m only approaching 1,000 writes of 107 bytes a second, but it’s much more than the ~700 I was getting out of InnoDB after buffer configurations. But with your suggestion, I’m much more likely to chalk it up to something I don’t understand to align with your understanding of the products.

I’ll give InnoDB another try to see how it does. Thanks for the help!