Incremental backup support for Percona MongoDB operator for kubernetes

Hi,

At present, it looks like the PSMDB operator for Kubernetes offers only full backup support.
It takes several Megabytes of S3 storage to maintain multiple backups for a longer period of time.
Is there any plan to provide incremental backup support in any of the upcoming releases?

2 Likes

Hello @siva ,

the problem you are trying to solve is the storage consumption, right?
Our Operator also supports point-in-time recovery - uploading oplogs to S3. Do you think it helps?

1 Like

Hello again @Sergey_Pronin! At risk of unearthing an old conversation, I thought it would be better to comment here rather than create a duplicate topic.

I could not find anything in the operator docs referring to incremental backups, so I’m assuming only full and PITR backups are supported currently.

Our use-case involves a big cluster (let’s say 1TB for simplicity’s sake) which grows a few GB every day. Doing one full backup and then enabling PITR is not ideal as the recovery time would be way too long, and creating full backups every N days would be expensive (mostly in terms of bandwidth, storage cost isn’t that much of an issue) so incrementals seem like a good way to go about it.

From my understanding, incremental backups are considered a Technical preview feature, but as they’ve been around for a couple years now, I was wondering if there’s anything blocking their implementation into the Mongo Operator.

Hello @Semantic ,

no problem with unearthing :slight_smile:
Operator does not have incremental backups yet.

We use Percona Backup for MongoDB (PBM) in the Operator and it added incremental backups only just now for physical: Incremental physical backups - Percona Backup for MongoDB

We don’t have logical incremental backups, just PITR (oplogs).

We will add physical backups in the next release of the operator. Will see if we can add incremental in the same go.

@Sergey_Pronin Ah, my apologies. I assumed that the OP mentioning incremental backups meant they were already out there when he posted about it, but 2.0.3 was indeed created 2 weeks ago.

Let me know if there’s anything I can help with to have it make it to the operator release! I’ll definitely be happy to try it out and report my findings.

I think K8SPSMDB-713 might actually be related since it adds support for physical backups to the operator, although does not seem to touch on incrementals, as it was opened last July.

@Semantic we have this huge pull request for physical backups and restores: https://github.com/percona/percona-server-mongodb-operator/pull/1112

You can check if it would be easy to add incremental backups. What do you think?

Not sure if this is sarcasm, but yes I know, that PR is related to K8SPSMDB-713 I linked above.

Given Ege already added the backupType option, the thing missing is supporting passing the --base param required when creating the first incremental backup which acts exactly the same as a full physical backup. Since pbm should handle creating the incremental based on what changed, I’m assuming adding a backup task entry to run the incremental on a schedule should work as expected.

I’m sure there might be some other considerations like writing the unit tests for it, but from it doesn’t seem that extreme of an add.

No sarcasm at all, I was not sure if you saw the PR :slight_smile:

I will ask Ege about it and see. Thanks for pushing it forward!

1 Like