Not the answer you need?
Register and ask your own question!

Unable to take logical backup on s3 using percona backup for mongodb

sbotlaguntasbotlagunta Current User Role Contributor

Hi,

I got the below error while taking the backup on s3 using "percona backup for mongodb".

Error:

2020/09/03 08:40:08 [INFO] backup/2020-09-03T08:19:07Z: mark backup as error `mongodump: write data: upload to S3: MultipartUpload: upload multipart failed

upload id: XXXX

caused by: TotalPartsExceeded: exceeded total allowed configured MaxUploadParts (10000). Adjust PartSize to fit in this limit.`: <nil>

My total mongoDB size is 1.5TB

Please assist on the same ASAP.

Thanks,

Surendra

Best Answers

Answers

  • sbotlaguntasbotlagunta Current User Role Contributor

    Anyone please answer this question.

    Thanks,

    Surendra Babu

  • sbotlaguntasbotlagunta Current User Role Contributor


  • sbotlaguntasbotlagunta Current User Role Contributor
    edited September 8
    Thank you for the reply Akira, I am hoping/waiting for the minor release 1.3.2 with the fix soon.
  • Akira KuroganeAkira Kurogane Percona Percona Staff Role

    Hi. Just to give a heads-up v1.3.2 is expected to be released some day next week.

  • sbotlaguntasbotlagunta Current User Role Contributor

    Thank you Akira

  • sbotlaguntasbotlagunta Current User Role Contributor

    Hi Akira,

    Initiated the percona backup for mongodb to s3,the db size is 1.6TB. We got the below issue after completing the backup to s3 (it took 7 hrs) .

    Config command: pbm config --file /etc/mongodb/pbm_s3_store.yaml --mongodb-uri mongodb://xxxx:[email protected]:27017 --set pitr.enabled=true

    Backup Command: pbm backup --mongodb-uri mongodb://xxxx:[email protected]:27017

    List: pbm list --mongodb-uri mongodb://xxxx:[email protected]:27017

    Backup snapshots:

     2020-10-16T08:33:33Z Failed with "oplog: read data: oplog has insufficient range, some records since the last saved ts {1602837232 4340} are missing. Run `pbm backup` to create a valid starting point for the PITR."

    Error: check PITR state for shard 'zion': PITR backup failed

    PITR <on>:

    !Failed to run PITR backup. Agent logs:

     zion: 2020-10-16T15:26:14.000+0000 [ERROR] pitr: defining starting point for the backup: no backup found, a new backup is required to start PITR

    Please assist on the same.

    Thanks,

    Surendra Babu

  • sbotlaguntasbotlagunta Current User Role Contributor

    Thank you Akira,It worked after increasing the oplog size.

    Thanks,

    Surendra Babu

  • Akira KuroganeAkira Kurogane Percona Percona Staff Role

    Cool, good to hear.

  • sbotlaguntasbotlagunta Current User Role Contributor

    Hi Akira,

    I am trying to restore(standalone node with replica set) the back up data but got the below error.

    error:

    2020-10-22T11:41:15.110+0000 demux finishing when there are still outs (14)

    2020-10-22T11:41:15.000+0000 [ERROR] pitr/2020-10-20T13:32:59Z: restore: restore mongo dump (successes: 2091088089 / fails: 0): <db.collectionname>: error restoring from archive on stdin: reading bson input: error demultiplexing archive; archive io error

    Out of 1.6TB data, one of the collection size is 900GB. My node RAM size is 120GB.

    Is it possible to restore the same with pbm?

    Thanks,

    Surendra Babu

  • Akira KuroganeAkira Kurogane Percona Percona Staff Role

    Hi Surendra.

    When the topic changes please make a new forum discussion. But in this case I'd say please look at this ticket instead: https://jira.percona.com/browse/PBM-555.

Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.