Hello,
I would like some help with creating a backup to an S3 Bucket for my Percona Operator for MongoDB environment.
I have my configuration file and I didn’t have a backup, but after doing some research, I managed to identify that there is this possibility.
With this, I performed the configuration and deployed it to the environment, however, I received the following error when it was time to perform the backup at the time specified in the “tasks”:
So, I would like some help, to know what I configured wrong or if an extra configuration is needed. Could you please help me with this part?
I am attaching my configuration file used so you can check.
Thank you in advance for your help and availability!
backup-mongodb-operator.txt (2.8 KB)
Hi @waly_ferreira !
If you didn’t try to do a backup, could you please try to do it. Because I think operator sets up storage for the backup agent when the first backup is issued and after that you should stop seeing these messages.
Hello,
@Tomislav_Plavcic, just a doubt, from the moment I perform to apply in Kubernetes, shouldn’t the “tasks” configuration occur?? Or will a manual backup be necessary and then perform the automated one?
I believe that some configuration in my file is wrong and for this reason it is giving error.
Thanks in advance
Hi @waly_ferreira !
Sorry I was a bit in a rush so didn’t read your question fully first time so will expand my answer.
It is normal to see such messages until the first backup of any kind is done (either manual or scheduled). I agree these messages are silly and I will open a ticket for that.
What happens on the first backup you get a psmdb-backup object so you can check it out like this:
$ kubectl get psmdb-backup
NAME CLUSTER STORAGE DESTINATION TYPE STATUS COMPLETED AGE
cron-my-cluster-name-20230227184500-l2w4r my-cluster-name aws-s3 psmdb-demand-backup/2023-02-27T18:45:21Z logical ready 5m25s 5m55s
cron-my-cluster-name-20230227185000-2lxb6 my-cluster-name aws-s3 psmdb-demand-backup/2023-02-27T18:50:21Z logical ready 26s 55s
and if you wish to check metadata/errors for particular backup you can do it like kubectl get psmdb-backup <backup-name> -oyaml.
So in your case this is the first thing I would check (just to see if any backup fired or not).
My backup agent logs look like this:
2023-02-27T18:44:04.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:09.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:14.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:19.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:24.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:29.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:34.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:39.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:44.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:49.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:54.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:44:59.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:45:04.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:45:09.000+0000 E [agentCheckup] check storage connection: unable to get storage: get config: get: mongo: no documents in result
2023-02-27T18:45:22.000+0000 I got command backup [name: 2023-02-27T18:45:21Z, compression: gzip (level: 6)] <ts: 1677523521>
2023-02-27T18:45:22.000+0000 I got epoch {1677523510 2}
2023-02-27T18:45:22.000+0000 D [backup/2023-02-27T18:45:21Z] init backup meta
2023-02-27T18:45:22.000+0000 D [backup/2023-02-27T18:45:21Z] nomination list for rs0: [[my-cluster-name-rs0-1.my-cluster-name-rs0.test.svc.cluster.local:27017 my-cluster-name-rs0-2.my-cluster-name-rs0.test.svc.cluster.local:27017] [my-cluster-name-rs0-0.my-cluster-name-rs0.test.svc.cluster.local:27017]]
2023-02-27T18:45:22.000+0000 D [backup/2023-02-27T18:45:21Z] nomination rs0, set candidates [my-cluster-name-rs0-1.my-cluster-name-rs0.test.svc.cluster.local:27017 my-cluster-name-rs0-2.my-cluster-name-rs0.test.svc.cluster.local:27017]
2023-02-27T18:45:22.000+0000 D [backup/2023-02-27T18:45:21Z] skip after nomination, probably started by another node
2023-02-27T18:45:27.000+0000 D [backup/2023-02-27T18:45:21Z] bcp nomination: rs0 won by my-cluster-name-rs0-1.my-cluster-name-rs0.test.svc.cluster.local:27017
but as you can see in my psmdb-backup output the backups started firing normally (I didn’t do any manual backup, just let the scheduled ones run).
1 Like
@Tomislav_Plavcic, we have same error here.
In our case, we can do backups from PMM console, and if try to test the backup location, returns the error “400 Bad Request: 400 Bad Request.”.
If we try to do a on-demand backup returns error without any log “No logs”.
This is still a problem. the pbmConfig collection is not initialised properly so no pbm commands work until the first scheduled backup happens. Manually running pbm backup does not work either with Error: backup-precheck: no available agent(s) on replsets: MY-REPLICASET
Specifically, this makes it impossible to restore a backup on a pristine install, which is a pretty big deal. This is on pbm 2.4.1 and operator 1.15.0, but it’s been an issue since at least pbm 2.0 and operator 1.14
