PerconaServerMongoDBRestore: ERROR oplog no such file

MongoDB Percona Operator for MongoDB
,
1

Description:

I am trying to configure MongoDB backup/restore using Percona Operator. Backups are being successfully uploaded on GCS:

Backups:
========
S3 us s3://https://storage.googleapis.com/mongodb-macpaw-staging-backup/test
  Snapshots:
    2024-08-01T10:23:21Z 26.75KB <logical> [restore_to_time: 2024-08-01T10:23:32Z]

When I try to create PerconaServerMongoDBRestore resource to restore DB from previously created backup I get an error:

Backups:
========
S3 us s3://https://storage.googleapis.com/mongodb-macpaw-staging-backup/test
  Snapshots:
    2024-08-01T10:23:21Z 26.75KB <logical> [ERROR: file "2024-08-01T10:23:21Z/test-rs1/oplog": no such file] [2024-08-01T10:23:36Z]

Steps to Reproduce:

Cluster backup config:

backup:
    enabled: true
    image: perconalab/percona-server-mongodb-operator:main-backup
    serviceAccountName: <serviceAccountName>
    storages:
      gcs:
        type: s3
        s3:
          region: us
          credentialsSecret: <credentialsSecret>
          bucket: <bucket>
          endpointUrl: https://storage.googleapis.com
          prefix: <prefix>
    pitr:
      enabled: true
    tasks:
      - name: daily
        enabled: true
        schedule: "0 11 * * *"
        keep: 3
        storageName: gcs
        compressionType: gzip
        compressionLevel: 6

PerconaServerMongoDBRestore manifest

apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDBRestore
metadata:
  name: restore1
  namespace: mongodb
spec:
  clusterName: test
  backupName: cron-test-20240802124900-fjk52
  backupSource:
    type: physical
    destination: s3://<bucket>/<prefix>
    s3:
      region: us
      credentialsSecret: <credentialsSecret>
      bucket: <bucket>
      endpointUrl: https://storage.googleapis.com
      prefix: <prefix>

Version:

Percona MongoDB Operator - 1.5.0

Expected Result:

It looks like restore process expects Oplog file to be named oplog and to be placed in the root directory of the bucket.

Actual Result:

Oplog file is stored under oplog directory: oplog/20240802113626-4.20240802113631-3.gz

Additional Information:

When I reupload oplog/20240802113626-4.20240802113631-3.gz file back into the bucket but with a different name - oplog, restore is succesfull.

2

Just quickly looking through the manifests you shared, the problem is here:

  backupSource:
    type: physical

I don’t see any indication of you using physical backup, but you are trying to restore from it.

Can you please try to change it to logical or just remove completely? If it does not help, we may need to look deeper.

3

Thanks for looking into it.
I have tried setting type to logical/removing completely - unfortunately still the same result.

4

@Ihnat_Makarov sorry, lost this thread till I saw it again on github: PerconaServerMongoDBRestore: ERROR oplog no such file · Issue #1618 · percona/percona-server-mongodb-operator · GitHub

So what I did:

  1. created the cluster and configured the backup to go to gcs bucket:
  backup:
    enabled: true
    image: percona/percona-backup-mongodb:2.5.0
    pitr:
      enabled: true
      oplogOnly: false
      oplogSpanMin: 1
      compressionType: gzip
      compressionLevel: 6
    storages:
      sp-test:
        type: s3
        s3:
          bucket: SOMEBUCKET
          credentialsSecret: backup-s3-secret
          endpointUrl: https://storage.googleapis.com/
          prefix: mypsmdb
  1. Created another cluster in another namespace. The only changed that I changed in the new cluser was the backup section. I disabled some of the items and removed storage config. I did it to ensure that I’m not going to the bucket with this cluster:
  backup:
    enabled: true
    image: percona/percona-backup-mongodb:2.5.0
    pitr:
      enabled: false
    storages:
  1. I restored the cluster with the following restore.yaml:
apiVersion: psmdb.percona.com/v1
kind: PerconaServerMongoDBRestore
metadata:
  name: restore1
spec:
  clusterName: demo-cluster1
  backupSource:
    destination: s3://SOMEBUCKET/mypsmdb/2024-10-07T11:55:55Z
    s3:
      credentialsSecret: backup-s3-secret
      endpointUrl: https://storage.googleapis.com/
      prefix: mypsmdb
      bucket: SOMEBUCKET

My cluster restored succesfully. So as a result I restored a cluster from an S3 bucket.

If you have not solved it yet, we can jump into a quick call: Zoom Scheduler