Backup present:
Full
Incr1
Incr2
Backup created using Xtrbackup and Xbcloud
To restore incremental backups (Incr1 and Incr2), as per documentation we need to download (xbcloud get) it to a directory and apply it with full.
Incremental backup size is the equal to the data change (consider only large updates to a table) since previous backup. To apply this Incremental backup with full backup, disk requirement is higher than original data size i.e. Full backup + Incr1.
Are there any ways to do this in-memory or on the fly?
Hello @Rachit_Saxena
Unfortunately no. When you “apply” an incremental to a full, you are effectively copying and merging 16K pages from one .ibd file into the original .ibd file. Imagine if you had a single 200GB table trying to apply 10GB of incremental changes. That would be a lot of memory required. Also, it’s not just the delta contents of the .ibd, there’s also the redo log that must be merged with the entire datadir since it contains changes across multiple tables.
Once you have applied incr1 to the full, you can erase incr1 from the local disk since the full is now modified with the changes of incr1.
1 Like
Hi @matthewb
We had built a tool that works on LVM Snapshot based backups for mysql but it has the issues as addressed in Backup MySQL using LVM snapshot.
Given Xtrabackup restore increases the overall disk requirements. Just curious, if there are other ways of backup, where we can apply the incrementals in-memory ?
You can’t apply incrementals in memory. When you apply the incremental, you are quite literally merging physical block pages between .ibd files. Disk is required.