Backup stuck in starting starte

PostgreSQL
1

I have a percona pgdb cluster running in kubernetes cluster. Earlier I had all backups enabled (incr, diff, full), i updated and switched to only full backups. Now i see that any backup that starts (full backup) they are stuck in starting state and no job or pod for backup is created. I see a log in operator as below that it is reconciling to a incremental backup and when i delete that backup, it starts to reconcile to another backup that is just older than that. Logs below

2025-09-09T09:18:50.940Z INFO WALWatcher Triggering PGBackup reconcile {“controller”: “perconapgcluster”, “controllerGroup”: “pgv2.percona.com”, “controllerKind”: “PerconaPGCluster”, “PerconaPGCluster”: {“name”:“pgdb”,“namespace”:“prod”}, “namespace”: “prod”, “name”: “pgdb”, “reconcileID”: “932d9e58-73a4-44b1-ad49-22074ef98fbe”, “latestBackup”: “pgdb-repo1-incr-fqgrc”, “latestRestorableTime”: “2025-09-09 09:18:19.034435 +0000 UTC”, “latestCommitTimestamp”: “2025-09-09 09:18:42.506988 +0000 UTC”}
2025-09-09T09:18:51.641Z INFO Got latest restorable timestamp {“controller”: “perconapgbackup”, “controllerGroup”: “pgv2.percona.com”, “controllerKind”: “PerconaPGBackup”, “PerconaPGBackup”: {“name”:“pgdb-repo1-incr-fqgrc”,“namespace”:“prod”}, “namespace”: “prod”, “name”: “pgdb-repo1-incr-fqgrc”, “reconcileID”: “d6bfb6f6-3257-409d-b989-fb148e09f8d6”, “request”: {“name”:“pgdb-repo1-incr-fqgrc”,“namespace”:“prod”}, “timestamp”: “2025-09-09 09:18:42.506988 +0000 UTC”}
2025-09-09T09:18:54.271Z INFO Waiting for backup to start {“controller”: “perconapgbackup”, “controllerGroup”: “pgv2.percona.com”, “controllerKind”: “PerconaPGBackup”, “PerconaPGBackup”: {“name”:“manual-full”,“namespace”:“prod”}, “namespace”: “prod”, “name”: “manual-full”, “reconcileID”: “e15e97a5-1c28-4a7a-a0dc-2bd667655c9e”, “request”: {“name”:“manual-full”,“namespace”:“prod”}}
2025-09-09T09:18:55.488Z INFO Superusers are exposed through PGBouncer {“controller”: “postgrescluster”, “controllerGroup”: “postgres-operator.crunchydata.com”, “controllerKind”: “PostgresCluster”, “PostgresCluster”: {“name”:“pgdb”,“namespace”:“prod”}, “namespace”: “prod”, “name”: “pgdb”, “reconcileID”: “aa2f918f-d36e-40f6-a60f-800a1599f8d6”}

2

@Kushal_Sultania

It could be the previous unsuccessful backups creating some issue.

2025-09-09T09:18:54.271Z INFO Waiting for backup to start {“controller”: “perconapgbackup”, “controllerGroup”: “``pgv2.percona.com``”, “controllerKind”: “PerconaPGBackup”, “PerconaPGBackup”: {“name”:“manual-full”,“namespace”:“prod”}, “namespace”: “prod”, “name”: “manual-full”, “reconcileID”: “e15e97a5-1c28-4a7a-a0dc-2bd667655c9e”, “request”: {“name”:“manual-full”,“namespace”:“prod”}}

You can try removing old annotations from PerconaPGCluster and then re-execute the backup again.

E.g,

kubectl edit PerconaPGCluster -n postgres-operator

Annotations: 
 pgv2.percona.com/backup-in-progress: backup2   ### remove them
              postgres-operator.crunchydata.com/pgbackrest-backup: backup2 ### remove them

Before trying any changes can you please confirm the below details.

kubectl describe PerconaPGCluster -n postgres-operator > PerconaPGCluster.txt

kubectl get pg-backup -n postgres-operator  > backup_list.txt

kubectl describe pg-backup <BACKUP NAME> -n postgres-operator > backup_describe.txt 

kubectl get pods -n postgres-operator > pods.txt

kubectl logs <BACKUP POD> -n postgres-operator > backup_pod_logs.txt

Can you also try accessing the pgbackrest repo pod and share the information ?

kubectl exec -it cluster1-repo-host-0 -n postgres-operator  – bash

pgbackrest info

pgbackrest --log-level-console=info --stanza=db check