PBM sharded cluster 100 TB

MongoDB Percona Backup for MongoDB
1

Hi team, we are exploring PBM for a 100TB database that will be sharded in PSMDB. The backup would go to S3.

Our POC on a 10TB database with different uploadPartSize and maxUploadParts settings on these EC2 instance types took:
m7i.16xlarge = 3 hours
m7i.2xlarge = 14 hours

With the above numbers, backup duration for 100TB would not be ideal. We are looking for options to make the PBM backup faster.

  1. Would disabling PBM compression make backup faster? Or any other compression method faster than default s2?
  2. Are there any other tuning parameters for PBM to make the backups faster?
    Also, we have our backup script that splits largest files into multiple parts and copies those parts to S3 parallely.
  3. What will be the consistent way to backup oplog in shards for point-in-time recovery with the above file copy script?
  4. Is Hot Backup faster than PBM? Can we use with PBM’s pitr.oplogOnly option to achieve PITR?

Thanks,
Ezee

1 Like
2

Hi, for this scenario physical backups are pretty much mandatory. Did you try those or logical backup with PBM?
To answer your questions:

  1. Probably marginally faster. Typically the default s2 is the fastest
  2. For logical backups, Backup options - Percona Backup for MongoDB
  3. PBM takes care of that for you, no need to script it
  4. PBM physical backup actually uses Hot Backup for physical backup type, you should try that.

Hope it helps

1 Like
3

Thanks for the response. Yes, we did physical backup.

Do we have any tuning parameters for physical backup in PBM? I am thinking: split large files, parallel copy multiple files, etc.

4

For physical backups, PBM already handles the splitting and parallel copy of each shard. You can tune the upload part size, but we use numCPU/2 for multipart upload (feel free to vote here Jira if you think is worth having)

Also see

Btw we typically suggest having each shard not bigger than 1 or 2 TB as best practice.

1 Like