Logical replication on large scale - 133 Databases - 140 Large tables - 2 replication slots per database

Hi,

Is 266 replication slots too much and I am being optimistic to believe that PostgreSQL 14.8 can handles it?

I’ll be honest, I’ve never tested the limits of maximum replication slots possible. Offhand there’s two limitations you need to consider:

  • hardcoded limitations: in that case get onto the hackers maillist and ask there
  • network IO: your network admin can help you determine bottlenecks

The simple way of course is to build out a demo environment and try it out. If you choose that option I’d suggest looking into leveraging pgbench using a custom script.

… we ran backups (parallel pg_dump on all 133 databases flag “–jobs 8”), the CPU used by the logical replication user skyrocketed,…

Sounds like the Long Running Transaction bugaboo. You can also have a look at this blog too for further guidance/

Hope this helps.

1 Like