# Documenting my tests in using HTTP with pgbackrest and Minio

**URL:** <https://forums.percona.com/t/documenting-my-tests-in-using-http-with-pgbackrest-and-minio/35556>\
**Category:** Percona Operator for PostgreSQL\
**Created:** [December 9, 2024, 1:40pm UTC](https://forums.percona.com/t/documenting-my-tests-in-using-http-with-pgbackrest-and-minio/35556 "2024-12-09T13:40:26Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![SuperHammer](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@SuperHammer](https://forums.percona.com/u/SuperHammer)\
**Post date:** [December 9, 2024, 1:40pm UTC](https://forums.percona.com/t/documenting-my-tests-in-using-http-with-pgbackrest-and-minio/35556/1 "2024-12-09T13:40:27Z")

</div>

## Description:

Running Percona Operator for PostgreSQL version 2.4.0 on OKD, I configured the backups to write on a Minio instance inside the cluster, but it seems pgbackrest doesn’t support HTTP connections.

## Steps to Reproduce:

Here is the configuration for the PerconaPGCluster.spec.backups.pgbackrest.global CR and the configured repos:

```auto
...
      global:
        log-level-console: info
        log-level-file: debug
        repo1-path: /pgbackrest/postgres-operator/prod/repo1
        repo1-retention-full: "3"
        repo1-retention-full-type: count
        repo1-s3-uri-style: path
        repo1-storage-port: "9000"
        repo1-storage-verify-tls: "n"
...
      repos:
      - name: repo1
        s3:
          bucket: pgdb-prod-backup
          endpoint: myMinioHostname
          region: us-east-1
        schedules:
          full: 0 0 * * *
...

```

Here is the resulting pgbackrest configuration inside the _-instance-_ pod:

```auto
bash-4.4$ cat /etc/pgbackrest/conf.d/pgbackrest_instance.conf
# Generated by postgres-operator. DO NOT EDIT.
# Your changes will not be saved.

[global]
log-level-console= info
log-level-file = debug
log-path = /pgdata/pgbackrest/log
repo1-path = /pgbackrest/postgres-operator/prod/repo1
repo1-retention-full = 3
repo1-retention-full-type = count
repo1-s3-bucket = mybucket
repo1-s3-endpoint = myMinioHostname
repo1-s3-region = us-east-1
repo1-s3-uri-style = path
repo1-storage-port = 443
repo1-storage-verify-tls = n
repo1-type = s3

[db]
pg1-path = /pgdata/pg16
pg1-port = 5432
pg1-socket-path = /tmp/postgres

```

## Version:

Percona Operator for PostgreSQL version 2.4.0

## Logs:

Running a pgbackrest info inside the _-instance-_ pod fails with TLS error:

```auto
bash-4.4$ pgbackrest info

stanza: [invalid]
    status: error (other)
      [ServiceError] TLS error [1:336130315] wrong version number
                [ServiceError] on 2 retries from 120105-180207ms: TLS error [1:336130315] wrong version number
    cipher: none

```

## Expected Result:

The flag “repo1-storage-verify-tls” should either accept self-signed certificates or use an HTTP connection.  
Here is a correct pgbackrest info output:

```auto
bash-4.4$ pgbackrest info
stanza: db
    status: ok
    cipher: none

    db (current)
        wal archive min/max (16): 000000030000000000000005/000000030000000000000007

        full backup: 20241209-104928F
            timestamp start/stop: 2024-12-09 10:49:28+00 / 2024-12-09 10:49:35+00
            wal start/stop: 000000030000000000000006 / 000000030000000000000006
            database size: 30.7MB, database backup size: 30.7MB
            repo1: backup set size: 4.1MB, backup size: 4.1MB

```

## Actual Result:

See above for the TLS error

## Additional Information:

The original operator from Crunchydata supported this configuration and it seems it worked for some people [S3 backups without TLS · Issue #3155 · CrunchyData/postgres-operator · GitHub](https://github.com/CrunchyData/postgres-operator/issues/3155), but thus far I couldn’t reproduce it, most likely they meant self-signed certificates and not HTTP connections.

I also found these issues, but I couldn’t find many details:

> <https://github.com/pgbackrest/pgbackrest/issues/1702>
>
> Please provide the following information when submitting an issue (feature reque…sts or general comments can skip this):
> 
> 1. pgBackRest version: 2.37
> 
> 2. PostgreSQL version: 22
> 
> 3. Operating system/version - if you have more than one server (for example, a database server, a repository host server, one or more standbys), please specify each: Ubuntu 18.04
> 
> 4. Did you install pgBackRest from source or from a package? Package
> 
> 7. Describe the issue:
> I am trying to recover from this error when trying to archive wal:
> repo1: \[AccessError\] raised from remote-0 tls protocol on 'repo1.ourdomain.net': access denied
> repo2: \[AccessError\] raised from remote-0 tls protocol on 'repo2.ourdomain.net': access denied
> 
> I had this working yesterday. Then I changed out the certificates. I am now using public certs with wildcards, like '\*.ourdomain.net'.
> 
> Would the wildcard be a problem?
> 
> I'm not sure if "AccessError" means that it can't authenticate using tls, or if the problem is that the repo server pgbackrest won't give the db server access. If it's the latter, I am undable to understand how to. I have tried all varieties of tls-server-auth=pg1=stanza, including using the fqdn of the pg host, as well as tls-server-auth=\\\*=\\\*, to no avail. And like I said, I had it working yesterday, so I'm not really thinking this is the problem. Or is it?
> 
> Any thoughts?

> <https://github.com/pgbackrest/pgbackrest/issues/2340>
>
> Please provide the following information when submitting an issue (feature reque…sts or general comments can skip this):
> 
> 1. pgBackRest version:2.47
> 
> 2. PostgreSQL version:12
> 
> 7. Describe the issue:
> Hello, I see that pgbackrest currently only supports https protocol, is there any relevant option for users to choose, or can users release the restrictions by changing the underlying code? If yes, could you please inform us of the location for modification? Thank you very much.

In the source code for pgbackrest, inside storage.c at line 1249 [pgbackrest/src/storage/s3/storage.c at main · pgbackrest/pgbackrest · GitHub](https://github.com/pgbackrest/pgbackrest/blob/main/src/storage/s3/storage.c#L1249) there doesn’t seem to be any options for not using TLS/using a normal TCP connection.

The documentation at https:// [pgBackRest - Configuration Reference](http://pgbackrest.org/configuration.html#section-repository/option-repo-storage-verify-tls) says:

> This option provides the ability to enable/disable verification of the storage server TLS certificate

This configuration isn’t a problem per-se, it is actually correct to use TLS by default.

So far, my solution has been to use an OKD/OpenShift route with tls termination on Minio:

```auto
kind: Route
apiVersion: route.openshift.io/v1
metadata:
  name: api
  namespace: minio
  labels:
    component: minio
  annotations:
    openshift.io/host.generated: 'true'
spec:
  host: api-minio.myhostname
  to:
    kind: Service
    name: minio-api
    weight: 100
  port:
    targetPort: 9000
  tls:
    termination: edge
  wildcardPolicy: None

```

Then, I used port 443 to connect:

```auto
      global:
        ...
        repo1-storage-port: "443"
        ...

```

If you are using vanilla Kubernetes or other solutions, see https:// [Enable Network Encryption | AIStor Object Store Documentation](http://min.io/docs/minio/linux/operations/network-encryption.html) on how to create self-signed certificates for your Minio.

I hope this helps someone else working on this configuration, it took me some troubleshooting to understand this was not the correct way to do things.

---

<div class="post-metadata">

**Author:** ![SuperHammer](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@SuperHammer](https://forums.percona.com/u/SuperHammer)\
**Post date:** [December 9, 2024, 1:56pm UTC](https://forums.percona.com/t/documenting-my-tests-in-using-http-with-pgbackrest-and-minio/35556/2 "2024-12-09T13:56:24Z")

</div>

See also [Issues with configuring backup against S3 storage](https://forums.percona.com/t/issues-with-configuring-backup-against-s3-storage/34970) for more information

---

<div class="post-metadata">

**Author:** ![Sergey\_Pronin](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/sergey_pronin/32/14887_2.png) [@Sergey\_Pronin](https://forums.percona.com/u/Sergey_Pronin)\
**Post date:** [December 23, 2024, 8:45am UTC](https://forums.percona.com/t/documenting-my-tests-in-using-http-with-pgbackrest-and-minio/35556/3 "2024-12-23T08:45:27Z")

</div>

This is awesome, @SuperHammer ! Thank you for documenting it.

Let me talk to our PG team to verify pgbackrest capability to use HTTP for S3.