# How to configure S3 backup for PostgreSQL using values.yaml

**URL:** <https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659>\
**Category:** Percona Operator for PostgreSQL\
**Created:** [March 8, 2022, 2:01pm UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659 "2022-03-08T14:01:56Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Michael\_Chudinov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/michael_chudinov/32/4732_2.png) [@Michael\_Chudinov](https://forums.percona.com/u/Michael_Chudinov)\
**Post date:** [March 8, 2022, 2:01pm UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/1 "2022-03-08T14:01:56Z")

</div>

How to configure S3 backup storage using values.yaml file or helm chart options?  
I do

```auto
backup:
  storages:
    s3-storage:
      type: s3
      bucket: "my-backup"
      region: default
      endpointUrl: "https://s3.mystorage.local"
  schedule:
    - name: "daily-s3"
      schedule: "0 3 * * *"
      keep: 5
      storage: s3-storage
      type: full

```

Get an error:  
Error: INSTALLATION FAILED: YAML parse error on pg-db/templates/cluster.yaml: error converting YAML to JSON: yaml: line 70: did not find expected key

How to fix this?

---

<div class="post-metadata">

**Author:** ![Slavisa\_Milojkovic](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/slavisa_milojkovic/32/7097_2.png) [@Slavisa\_Milojkovic](https://forums.percona.com/u/Slavisa_Milojkovic)\
**Post date:** [July 29, 2022, 3:28pm UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/2 "2022-07-29T15:28:16Z")

</div>

Hello everyone, I managed to set up that part in values, I even set up s3 parameters in values of pg operator, but to no avail.  
This seems to work:

```auto
  storages:
    - name: repo1
      s3:
        type: "s3"
        bucket: "percona-backup"
.....

```

A backup pod is always running this:

```auto
msg="command to execute is [pgbackrest backup --type=full --db-host=10.42.0.184 --db-path=/pgdata/pg-db]"

```

I managed to run arguments using pgtask:

```auto
apiVersion: pg.percona.com/v1
kind: Pgtask
metadata:
  labels:
    pg-cluster: pg-db
    pgouser: admin
  name: pg-db-backrest-full-backup
spec:
  name: pg-db-backrest-full-backup
  parameters:
    backrest-command: backup
# backup type can be:
# Differential: Create a backup of everything since the last full backup was taken
# --type=diff
# Full: Back up the entire database
# --type=full
# Incremental: Back up everything since the last backup was taken, whether it was full, differential, or incremental
# --type=incr
# backup retention can be:
# --repo1-retention-full=2 how many full backups to retain
# --repo1-retention-diff=4 how many differential backups to retain
# --repo1-retention-archive=6 how many sets of WAL archives to retain alongside
# the full and differential backups that are retained
    backrest-opts: --type=full --repo1-retention-full=5 --repo1-type=s3 --repo1-s3-bucket=percona-backup --repo1-s3-endpoint=.......
    backrest-s3-verify-tls: "false"
    backrest-storage-type: "s3"
    job-name: pg-db-backrest-full-backup
    pg-cluster: pg-db
  tasktype: backrest

```

but in this case, it’s not allowing me to use repo-s3-key in cmd line 😕

I suppose it should be using environment variables from pg-db-backrest-repo-config (which I configured too) but it just isn’t. I don’t know what else to try. If anyone can help me it would be great. Thanks

---

<div class="post-metadata">

**Author:** ![Ivan\_Pylypenko](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_pylypenko/32/2452_2.png) [@Ivan\_Pylypenko](https://forums.percona.com/u/Ivan_Pylypenko)\
**Post date:** [August 2, 2022, 12:08pm UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/3 "2022-08-02T12:08:51Z")

</div>

Hi

Could you please share the helm CLI call on that?

---

<div class="post-metadata">

**Author:** ![Slavisa\_Milojkovic](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/slavisa_milojkovic/32/7097_2.png) [@Slavisa\_Milojkovic](https://forums.percona.com/u/Slavisa_Milojkovic)\
**Post date:** [August 3, 2022, 9:20am UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/4 "2022-08-03T09:20:48Z")

</div>

Hello, helm command is the ususal

```auto
helm upgrade --install pg-db percona/pg-db --version 1.2.0 -n percona --create-namespace -f values-pg-db.yaml

```

And the values.yaml is:

```auto
backup:
  image: perconalab/percona-postgresql-operator:main-ppg14-pgbackrest
# imagePullPolicy: Always
  backrestRepoImage: perconalab/percona-postgresql-operator:main-ppg14-pgbackrest-repo
  resources:
    requests:
      cpu: "200m"
      memory: "48Mi"
# limits:
# cpu: "1"
# memory: "64Mi"
# affinity:
# antiAffinityType: preferred
  volumeSpec:
    size: 1G
    accessmode: ReadWriteOnce
    storagetype: dynamic
    storageclass: "local-path"
# matchLabels: ""
  storages:
    - name: repo1
      s3:
        type: "s3"
        bucket: "percona-backup"
        region: "east-us"
        uriStyle: "host"
        verifyTLS: "false"
        endpointUrl: "objectstorage.XXX.host"
  schedule:
    - name: "5min-backup"
      schedule: "*/5 * * * *"
      keep: 3
      type: full
      storage: local,s3

```

And I used this file to add pgbackrest params, to try and change the command in the backup pod

```auto
https://github.com/percona/percona-postgresql-operator/blob/main/deploy/backup/backup.yaml

```

I followed and tried everything from this doc:

> **[Backup and restore - Percona Operator for PostgreSQL](https://docs.percona.com/percona-operator-for-postgresql/backups.html)**
>
> Documentation

It creates a local backup, and everything else is working fine, but I can’t get s3 upload to work.  
Thank you for your reply

---

<div class="post-metadata">

**Author:** ![Ivan\_Pylypenko](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_pylypenko/32/2452_2.png) [@Ivan\_Pylypenko](https://forums.percona.com/u/Ivan_Pylypenko)\
**Post date:** [August 3, 2022, 3:16pm UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/5 "2022-08-03T15:16:28Z")

</div>

Thank you for the detailed output. We appreciate it.

Helm chart for database cluster object has one small difference inside storage configuration - it’s not an array in terms of yaml but map.  
Please check this link [percona-helm-charts/README.md at pg-operator-1.2.0 · percona/percona-helm-charts · GitHub](https://github.com/percona/percona-helm-charts/blob/pg-operator-1.2.0/charts/pg-db/README.md) up. It has a small hint how to set backups storage as operator expects.

---

<div class="post-metadata">

**Author:** ![Slavisa\_Milojkovic](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/slavisa_milojkovic/32/7097_2.png) [@Slavisa\_Milojkovic](https://forums.percona.com/u/Slavisa_Milojkovic)\
**Post date:** [August 5, 2022, 8:25am UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/6 "2022-08-05T08:25:52Z")

</div>

Hello, and thank you Ivan for your response.  
I used this to deploy (might be useful for others)

```auto
helm upgrade --install pg-db percona/pg-db --version 1.2.0 -n percona --create-namespace -f values-pg-db.yaml \
  --set backup.storages.my-s3.bucket=percona-backup \
  --set backup.storages.my-s3.type=s3 \
  --set backup.storages.my-s3.name=my-s3 \
  --set backup.storages.my-s3.endpointUrl='objectstorage.XXXXXXXX.host' \
  --set backup.storages.my-s3.region='west-us' \
  --set backup.storages.storage0.verifyTLS=false \
  --set backup.storages.storage0.uriStyle='path' \
  --set bucket.key=IK6Oa8DM9HUgP3uC \
  --set bucket.secret=Ec8xH9WU6CCDpXtMC2CNy1ko2b9hQJU9 \
  --set backup.storages.my-local.type=local \
  --set "backup.schedule[0].name=sat-night-backup" \
  --set "backup.schedule[0].schedule=*/5 * * * *" \
  --set "backup.schedule[0].keep=5" \
  --set "backup.schedule[0].type=full" \
  --set "backup.schedule[0].storage=s3" \
  --set pgPrimary.volumeSpec.storageclass=local-path \
  --set backup.volumeSpec.storageclass=local-path \
  --set replicas.volumeSpec.storageclass=local-path

```

So now it does try to use S3, but stanza won’t create. Looks like it is joining the bucket name to minio host domain. I think I set it up correctly. So not sure what is going on

```auto
time="2022-08-05T08:18:51Z" level=info msg="stderr=[ERROR: [039]: HTTP request failed with 404 (Not Found):\n ***Path/Query*** :\n GET /?delimiter=%2F&list-type=2&prefix=backrestrepo%2Fpg-db-backrest-shared-repo%2Farchive%2Fdb%2F\n ***Request Headers*** :\n authorization: <redacted>\n content-length: 0\n host: percona-backup.objectstorage.XXXXX.host\n

```

---

<div class="post-metadata">

**Author:** ![Ivan\_Pylypenko](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_pylypenko/32/2452_2.png) [@Ivan\_Pylypenko](https://forums.percona.com/u/Ivan_Pylypenko)\
**Post date:** [August 6, 2022, 4:03pm UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/7 "2022-08-06T16:03:31Z")

</div>

> [@Slavisa\_Milojkovic](#):
>
> `uriStyle`

HI

Try to use uriStyle: host instead. It should not append bucket name to the overall S3 URI.

---

<div class="post-metadata">

**Author:** ![Slavisa\_Milojkovic](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/slavisa_milojkovic/32/7097_2.png) [@Slavisa\_Milojkovic](https://forums.percona.com/u/Slavisa_Milojkovic)\
**Post date:** [August 15, 2022, 7:43am UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/8 "2022-08-15T07:43:34Z")

</div>

Thank you very much Ivan, I made it work. The problem was that I used the wrong storage name in the code below.

> [@Slavisa\_Milojkovic](#):
>
> ```auto
> --set backup.storages.my-s3.region='west-us' \
> --set backup.storages.storage0.verifyTLS=false \
> --set backup.storages.storage0.uriStyle='path' \
> --set bucket.key=IK6Oa8DM9HUgP3uC \
> 
> ```

---

<div class="post-metadata">

**Author:** ![mboncalo](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mboncalo/32/16954_2.png) [@mboncalo](https://forums.percona.com/u/mboncalo)\
**Post date:** [June 21, 2024, 6:00am UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/9 "2024-06-21T06:00:04Z")

</div>

Hi,

I am having a similar issue with setting up s3 backup on minio with values file.  
My values file looks like this:

```auto

finalizers:

crVersion: 2.3.1
repository: percona/percona-postgresql-operator
image: ""
imagePullPolicy: Always
postgresVersion: 15

pause: false
unmanaged: false
standby:
  enabled: false

customTLSSecret:
  name: ""
customReplicationTLSSecret:
  name: ""

instances:
- name: service-1
  replicas: 3

  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        podAffinityTerm:
          labelSelector:
            matchLabels:
              postgres-operator.crunchydata.com/data: postgres
          topologyKey: kubernetes.io/hostname

  walVolumeClaimSpec:
    storageClassName: "default"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 2Gi

  dataVolumeClaimSpec:
    storageClassName: "default"
    accessModes:
    - ReadWriteOnce
    resources:
      requests:
        storage: 2Gi

proxy:
  pgBouncer:
    replicas: 3
    image: ""

    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 1
          podAffinityTerm:
            labelSelector:
              matchLabels:
                postgres-operator.crunchydata.com/role: pgbouncer
            topologyKey: kubernetes.io/hostname

backups:
  pgbackrest:

    image: ""

    global:
      repo1-retention-full: "14"
      repo1-retention-full-type: time

      repo2-retention-full: "14"
      repo2-retention-full-type: time
      repo2-s3-uri-style: path
      repo2-s3-verify-tls: false
      repo2-s3-key: "..."
      repo2-s3-key-secret: "..."
      repo2-retention-full: "14"
      repo2-retention-full-type: time
      repo2-path: /repo2

    repoHost:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 1
            podAffinityTerm:
              labelSelector:
                matchLabels:
                  postgres-operator.crunchydata.com/data: pgbackrest
              topologyKey: kubernetes.io/hostname

     priorityClassName: high-priority

     topologySpreadConstraints:
     - maxSkew: 1
       topologyKey: my-node-label
       whenUnsatisfiable: ScheduleAnyway
       labelSelector:
         matchLabels:
           postgres-operator.crunchydata.com/pgbackrest: ""

    manual:
      repoName: repo1
      options:
      - --type=full
    repos:
    - name: repo1
      schedules:
        full: "* * * 7 *"

        incremental: "* * * 5 *"
      volume:
        volumeClaimSpec:
          storageClassName: "default"
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: 1Gi
    - name: repo2
      s3:
        bucket: "..."
        endpoint: "..."

        region: "default"
      schedules:
        full: "*/5 * * * *"

pmm:
  enabled: false
  image:
    repository: percona/pmm-client
    tag: 2.41.0

  secret: cluster1-pmm-secret
  serverHost: monitoring-service

secrets:

  primaryuser: test123

  postgres: test123

  pgbouncer:

  pguser: test123

```

I have been researching for a week and couldn’t make it work  
The pods aren’t even starting because pgbackrest-config volume is not mounting. There are no logs

---

<div class="post-metadata">

**Author:** ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)\
**Post date:** [June 24, 2024, 7:50am UTC](https://forums.percona.com/t/how-to-configure-s3-backup-for-postgresql-using-values-yaml/14659/10 "2024-06-24T07:50:01Z")

</div>

Hello @mboncalo.

I do not see any secrets. Have you created any for your S3?