# Pg\_wal is filling up disk **URL:** \ **Category:** Percona Operator for PostgreSQL\ **Created:** [September 4, 2023, 5:27am UTC](https://forums.percona.com/t/pg-wal-is-filling-up-disk/24908 "2023-09-04T05:27:15Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![martin.schack](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@martin.schack](https://forums.percona.com/u/martin.schack)\ **Post date:** [September 4, 2023, 5:27am UTC](https://forums.percona.com/t/pg-wal-is-filling-up-disk/24908/1 "2023-09-04T05:27:15Z")
## Description: Our /pgdata directory is getting 100% full when data is loaded to the database. The disks are ~200G in size, which is fine for the database, but with the initial load pg\_wal fills up the disk to 100%. pg\_wal directory occupies more than150G whereas base is below 50G still. We’re using two replicas, and during the load they have replication lag of 0 MB, but still pg\_wal is not cleaned up in time. What kind of parameters can we use to enforce faster pg\_wal backup/cleanup. ## Steps to Reproduce: insert a lot of data into a new and empty database ## Version: Operator 1.4 Postgres version: 13.10 ## Logs: ## Expected Result: pg\_wal that is no longer needed for replication should be cleaned up, and disks should be empty. ## Actual Result: primary database crashes and database is down ## Additional Information: ---
**Author:** ![Jobin\_Augustine](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/jobin_augustine/32/1275_2.png) [@Jobin\_Augustine](https://forums.percona.com/u/Jobin_Augustine)\ **Post date:** [September 4, 2023, 8:05am UTC](https://forums.percona.com/t/pg-wal-is-filling-up-disk/24908/2 "2023-09-04T08:05:43Z")
The most common reason for pg\_wal directory to fill up is : WAL archiving not working. Please inspect the PostgreSQL logs. ---
**Author:** ![Robert\_Bernier](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/robert_bernier/32/1367_2.png) [@Robert\_Bernier](https://forums.percona.com/u/Robert_Bernier)\ **Post date:** [September 4, 2023, 1:29pm UTC](https://forums.percona.com/t/pg-wal-is-filling-up-disk/24908/3 "2023-09-04T13:29:29Z")
As well, depending upon whether it’s a “push” or “pull” you might want to query pg\_stat\_activity and check for connections meant to pull the WALS. Although a “push” usually means you are using “archive\_command” ---
**Author:** ![martin.schack](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@martin.schack](https://forums.percona.com/u/martin.schack)\ **Post date:** [September 7, 2023, 5:33am UTC](https://forums.percona.com/t/pg-wal-is-filling-up-disk/24908/4 "2023-09-07T05:33:33Z")
Hi, Thank you for your answers. Unfortunately I cannot see any errors in postgres logs. We’re using percona operator and all the standards that the operator uses for the backup. So we’re using: archive\_command: source /opt/crunchy/bin/postgres-ha/pgbackrest/pgbackrest-archive-push-local-s3.sh %p How’s the cleanup configured? Is there any configuration possible to influence backup and cleanup when using Percona Postgres Operator? ---
**Author:** ![martin.schack](https://avatars.discourse-cdn.com/v4/letter/m/a5b964/32.png) [@martin.schack](https://forums.percona.com/u/martin.schack)\ **Post date:** [September 20, 2023, 9:21am UTC](https://forums.percona.com/t/pg-wal-is-filling-up-disk/24908/5 "2023-09-20T09:21:41Z")
Hi, Finally I found the root cause. When I tried to execute the archive command manually I received: ERROR: [045]: WAL file ‘0000008D00000410000000CE’ already exists in the repo1 archive with a different checksum command terminated with exit code 45 With that information I removed the affected backup from pgbackrest repository, and the archiving continued as expected. I’m no PostgreSQL expert today. However I wonder how this can happen. Looks like WALs from the same timeline and different checksum have been backed up before. Maybe that the archiving was interrupted by a system-downtime. But I found that 3 archive backups were affected, and I had to delete all the three of them. I still don’t understand the root cause. At least I’m able to perform a manual cleanup and start another full backup. best regards, Martin ---
**Author:** ![Robert\_Bernier](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/robert_bernier/32/1367_2.png) [@Robert\_Bernier](https://forums.percona.com/u/Robert_Bernier)\ **Post date:** [September 22, 2023, 1:43pm UTC](https://forums.percona.com/t/pg-wal-is-filling-up-disk/24908/6 "2023-09-22T13:43:38Z")
Hello, > I still don’t understand the root cause. I suggest you look at the WAL original timestamps, maybe using stat, and compare it to the events of the various changes of state.