# pgBackRest Path Issue with Percona PostgreSQL Operator 2.7.0
## Issue Summary
The Percona PostgreSQL Operator 2.7.0 has a critical incompatibility with the pgBackRest image `percona/percona-pgbackrest:2.55.0` that prevents backups from working. The operator generates backup jobs with an incorrect binary path.
## Environment
- **Operator Version**: 2.7.0
- **PostgreSQL Version**: 17
- **pgBackRest Image**: percona/percona-pgbackrest:2.55.0
- **Kubernetes Version**: 1.29.x
- **Installation Method**: Helm chart
## Problem Description
### 1. Binary Path Mismatch
The operator creates backup pods with the command path `/opt/crunchy/bin/pgbackrest`, but in the Percona pgBackRest image, the binary is located at `/usr/bin/pgbackrest`.
**Error message:**
```
Error: container create failed: executable file `/opt/crunchy/bin/pgbackrest` not found in $PATH: No such file or directory
```
**Verification of actual path:**
```bash
$ kubectl exec postgresql-ha-repo-host-0 -n data -c pgbackrest – which pgbackrest
/usr/bin/pgbackrest
```
### 2. Root Cause Analysis
The Percona PostgreSQL Operator is based on the Crunchy Data PostgreSQL Operator. The path `/opt/crunchy/bin/pgbackrest` is from the original Crunchy implementation, while Percona’s pgBackRest images install the binary in the standard Linux location `/usr/bin/pgbackrest`.
## Steps to Reproduce
1. Install Percona PostgreSQL Operator 2.7.0:
```bash
helm repo add percona Percona Helm Charts | percona-helm-charts
helm install percona-postgresql-operator percona/pg-operator \
–namespace data \
--version 2.7.0
```
2. Deploy a PostgreSQL cluster with backup enabled:
```yaml
apiVersion: pgv2.percona.com/v2
kind: PerconaPGCluster
metadata:
name: postgresql-ha
spec:
postgresVersion: 17
backups:
pgbackrest:
image: percona/percona-pgbackrest:2.55.0
repos:
- name: repo1
s3:
bucket: my-backup-bucket
endpoint: s3.amazonaws.com
region: us-east-1
```
3. Observe backup pod failures:
```bash
$ kubectl get pods -n data | grep backup
postgresql-ha-backup-xxxxx 0/1 CreateContainerError 0 30s
$ kubectl describe pod postgresql-ha-backup-xxxxx -n data
Error: container create failed: executable file `/opt/crunchy/bin/pgbackrest` not found in $PATH
```
## Attempted Workarounds
### 1. Using older pgBackRest image
Tried `percona/percona-postgresql-operator:2.5.0-ppg16.4-pgbackrest2.53-1` which has the correct path structure, but this version doesn’t support PostgreSQL 17:
```
ERROR: [VersionNotSupportedError] unexpected control version = 1700 and catalog version = 202406281
HINT: is this version of PostgreSQL supported?
```
### 2. Version Compatibility Matrix Issue
- pgBackRest 2.53 (with correct path) → Doesn’t support PostgreSQL 17
- pgBackRest 2.55 (supports PG17) → Has wrong binary path for operator
## Impact
- **Severity**: High
- **Affected Features**: All backup and restore operations fail
- **Workaround Available**: No viable workaround without custom images or downgrading PostgreSQL
## Expected Behavior
The operator should generate backup job commands using the correct path `/usr/bin/pgbackrest` when using Percona pgBackRest images, or the images should maintain compatibility with the expected `/opt/crunchy/bin/pgbackrest` path.
## Suggested Fixes
1. **Short-term**: Add a symbolic link in the pgBackRest image:
```dockerfile
RUN mkdir -p /opt/crunchy/bin && \
ln -s /usr/bin/pgbackrest /opt/crunchy/bin/pgbackrest
```
2. **Long-term**: Update the operator to detect and use the correct pgBackRest path based on the image being used, or make the path configurable.
3. **Alternative**: Provide a compatibility layer or environment variable to specify the pgBackRest binary path.
## Additional Information
### CRD Version Check
```bash
$ kubectl get crd perconapgclusters.pgv2.percona.com -o yaml | grep “pgv2.percona.com/version”
pgv2.percona.com/version: v2.7.0
```
### Related Components
- The issue appears to stem from the fork relationship between Percona’s operator and Crunchy Data’s operator
- This is likely affecting all users attempting to use PostgreSQL 17 with the Percona Operator 2.7.0
## Questions for the Community
1. Is there an official Percona pgBackRest image that’s compatible with both the 2.7.0 operator AND PostgreSQL 17?
2. Are there plans to fix this path discrepancy in upcoming releases?
3. Is there a recommended configuration or image combination for PostgreSQL 17 with working backups?
## System Information
```yaml
Operator Deployment:
Image: docker.io/percona/percona-postgresql-operator:2.7.0
PostgreSQL Cluster:
Version: 17
Image: docker.io/percona/percona-postgresql-operator:2.7.0-ppg17-postgres
pgBackRest:
Image: percona/percona-pgbackrest:2.55.0
Version in Container: pgBackRest 2.55.0
```
-–
*This issue is preventing production deployment of PostgreSQL 17 with the Percona Operator. Any guidance or timeline for a fix would be greatly appreciated.*