I am running into an issue with TokuDB and replication lag.
I am wondering if this is expected or if there is a setting I am missing somewhere?
We have tables with billions of rows, I can successfully create hot indexes on the master with no issues. It all happens in the background as expected.
The issue occurs when we have an index that takes time to build and it replicates out.
Say I have an index that takes 6 hours to build.
On the master I issue the command and it kicks off in the background and builds over the next 6 hours.
Everything is fine. There is no replication lag during master index creation.
Upon completion on the master, the new index is replicated out to the slaves.
I now have replication lag for the next 6 hours.
I.E. the slave stays on Relay_Log_Pos X for 6 hours while building the index, then catches up once the index is built.
(Seconds behind master builds up to 6+ hours and when the index is finished creating, then it catches up.)
This causes problems with something being saved and the change doesn’t show up on the slaves for 6 hours.
MySQL/TokuDB version: 5.7.21-21
tokudb_create_index_online = ON is on all the servers.
CREATE INDEX thing_id ON thing (thing_id);
Is there a setting that will allow the slaves to pick up the index creation statement and kick it to a background job like it does on the master allowing replication to continue normally and not get behind so bad?