Description:
Using Operator v2.4.0, the backup job pods aren’t being cleaned up. PerconaPGBackup are being deleted according to the retention configuration, but the backup job pods are not.
Steps to Reproduce:
...
spec:
backups:
pgbackrest:
configuration:
- secret:
name: pac-registry-db-pgbackrest-backup-s3
global:
repo1-path: /pgbackrest/harbor/harbor-db/repo1
repo1-retention-diff: '5'
repo1-retention-full: '3'
repo1-retention-full-type: count
repo1-s3-uri-style: path
image: percona/percona-postgresql-operator:2.4.0-ppg13-pgbackrest
...
repos:
- name: repo1
...
schedules:
differential: 24 * * * *
full: 48 1 * * *
crVersion: 2.4.0
...
Version:
Percona Postgresql Operator v2.4.0
Expected Result:
The backup job pods be deleted following the retention policy.
Actual Result:
The backup job pods are being kept when list all the pods.
Additional Information
opened 10:32AM - 26 Feb 24 UTC
bug
### Report
Sheduled kubernetes pg-backup objects are not automatically cleaned … up
### More about the problem
Since the 2.3.0 k8s Percona Postgres operator upgrade, pg-backup resources are created for each scheduled backup ([K8SPG-410](https://perconadev.atlassian.net/browse/K8SPG-410))
The pgbackrest backups and related items saved in the storage are still clean-up up properly following the retention rules defined in the spec.backups.pgbackrest.global.<repo>-retention-full* attributes of the pg cluster definition.
BUT the pg-backup related k8s resources are well created but never cleaned up even with the retention attributes properly defined.
This problem lead to a quick accumulation of pg-backups and jobs k8s resources in the namespaces using a pg cluster with scheduled backups.
### Steps to reproduce
1. Create a pg cluster with a backup storage and schedule section defined and a retention period
2. On the scheduled time, pg-backup resource is created with the related job, pod, pgbackrest backup and items in the storage
3. At the end of the retention period, pgbackrest backup and items are removed from the storage but none of the k8s resources (pg-backup and job) are not deleted
### Versions
1. Kubernetes - v1.27.6
2. Operator - Percona for PostgreSQL 2.3.1
3. Database - PostgreSQL 15.5
### Anything else?
_No response_
[K8SPG-410]: https://perconadev.atlassian.net/browse/K8SPG-410?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
I solved it by creating my own pgbackrest image which calls quitquitquit istio-proxy endpoint to terminate the container
Hi @mboncalo I have created the task [K8SPG-629] - Percona JIRA We will try to include the fix into v2.5.0