At present, it looks like the PSMDB operator for Kubernetes offers only full backup support.
It takes several Megabytes of S3 storage to maintain multiple backups for a longer period of time.
Is there any plan to provide incremental backup support in any of the upcoming releases?
the problem you are trying to solve is the storage consumption, right?
Our Operator also supports point-in-time recovery - uploading oplogs to S3. Do you think it helps?
Hello again @Sergey_Pronin! At risk of unearthing an old conversation, I thought it would be better to comment here rather than create a duplicate topic.
I could not find anything in the operator docs referring to incremental backups, so I’m assuming only full and PITR backups are supported currently.
Our use-case involves a big cluster (let’s say 1TB for simplicity’s sake) which grows a few GB every day. Doing one full backup and then enabling PITR is not ideal as the recovery time would be way too long, and creating full backups every N days would be expensive (mostly in terms of bandwidth, storage cost isn’t that much of an issue) so incrementals seem like a good way to go about it.
From my understanding, incremental backups are considered a Technical preview feature, but as they’ve been around for a couple years now, I was wondering if there’s anything blocking their implementation into the Mongo Operator.
@Sergey_Pronin Ah, my apologies. I assumed that the OP mentioning incremental backups meant they were already out there when he posted about it, but 2.0.3 was indeed created 2 weeks ago.
Let me know if there’s anything I can help with to have it make it to the operator release! I’ll definitely be happy to try it out and report my findings.
I think K8SPSMDB-713 might actually be related since it adds support for physical backups to the operator, although does not seem to touch on incrementals, as it was opened last July.
Not sure if this is sarcasm, but yes I know, that PR is related to K8SPSMDB-713 I linked above.
Given Ege already added the backupType option, the thing missing is supporting passing the --base param required when creating the first incremental backup which acts exactly the same as a full physical backup. Since pbm should handle creating the incremental based on what changed, I’m assuming adding a backup task entry to run the incremental on a schedule should work as expected.
I’m sure there might be some other considerations like writing the unit tests for it, but from it doesn’t seem that extreme of an add.