Basebackup failing after configuring pg_tde || PostgreSQL 16

PostgreSQL
1

I have followed below documentation to configure pg_tde in RHEL 8 with PG version 16. pg_tde documentation

After enabling and configuring pg_tde key provider and master key . pg_basebackup is getting failed with below error

[postgres@hostname postgres]$ pg_basebackup -D bkp_test -R -X stream -v
pg_basebackup: initiating base backup, waiting for checkpoint to complete
pg_basebackup: checkpoint completed
pg_basebackup: write-ahead log start point: 0/14000028 on timeline 1
pg_basebackup: starting background WAL receiver
pg_basebackup: created temporary replication slot "pg_basebackup_1391250"
WARNING:  aborting backup due to backend exiting before pg_backup_stop was called
pg_basebackup: error: COPY stream ended before last file was finished
pg_basebackup: removing contents of data directory "bkp_test"

In the logs I can see below error.

2024-10-18 07:55:21.132 EDT [1241049] LOG:  checkpoint starting: force wait
2024-10-18 07:55:21.136 EDT [1241049] LOG:  checkpoint complete: wrote 0 buffers (0.0%); 0 WAL file(s) added, 0 removed, 1 recycled; write=0.001 s, sync=0.001 s, total=0.004 s; sync files=0, longest=0.000 s, average=0.000 s; distance=16384 kB, estimate=27000 kB; lsn=0/14000060, redo lsn=0/14000028
2024-10-18 07:55:21.205 EDT [1391249] WARNING:  aborting backup due to backend exiting before pg_backup_stop was called
2024-10-18 07:55:21.205 EDT [1391249] ERROR:  invalid segment number 0 in file "pg_tde.map"
2024-10-18 07:55:21.205 EDT [1391249] STATEMENT:  BASE_BACKUP ( LABEL 'pg_basebackup base backup',  PROGRESS,  WAIT 0,  MANIFEST 'yes',  TARGET 'client')
2024-10-18 07:55:21.221 EDT [1391250] LOG:  unexpected EOF on standby connection
2024-10-18 07:55:21.221 EDT [1391250] STATEMENT:  START_REPLICATION SLOT "pg_basebackup_1391250" 0/14000000 TIMELINE 1

After loading shared library pg_tde on server level and configuring master key for specific DB level, I can see below files in base directory that is creating issue. but unable to understand .

./16505/pg_tde.map
./16505/pg_tde.dat

below are the databases with OID.

test=# select oid,datname from pg_database;
  oid  |  datname
-------+-----------
     5 | postgres
     1 | template1
     4 | template0
 16505 | test
(4 rows)

Thanks for your support team. I had to post here as I am not getting any article online for this. there might be some issue with my setup I am not sure.

2

Hi there.
pg_tde.map and pg_tde.dat are used by pg_tde to store principal keys metadata and encrypted internal keys. Unfortunately, postgres treats these as files that need checksum validation and tries to do that but fails. We have fixed this in our Percona Server for PostgreSQL but it’s only available in v 17. So if you want to use pg_tde with the community version of PostgreSQL, you have to disable checksums for pg_basebackup (--no-verify-checksums)

3

@Andrew_Pogrebnoi Thank you so much for you help on this.

Current document mentioned principal key but in setup we have master key naming conventions.

SELECT pg_tde_set_principal_key(‘name-of-the-principal-key’, ‘provider-name’); this is in documentation but if you check functions after configuring we need to set pg_tde_set_master_key … then only working. could you please let me know when this is going to fix ? currently function syntax written in documentation are diff than actual downloaded setup .

thanks for you help

4

How do you install pg_tde? When did you installed/updated it last time?
We merged a PR with the renaming back in June and everything should work as the documentation states.

5

Hi @Andrew_Pogrebnoi Thanks for your help on this. we installed in sept only. I will recheck and try . --We used this setup → sudo yum -y install https://repo.percona.com/yum/percona-release-latest.noarch.rpm

6

@amulla sept build should have a fix for this issue. Another thing, just reinstalling the extension won’t help in this case as it is currently doesn’t update .sql manifests (which contains declaration of the functions). You have to also explicitly drop and create the extension:

> DROP EXTENSION pg_tde;
> CREATE EXTENSION pg_tde;

We’ll make proper upgrade facilities later on but for now this is the only way.