# Pbm-agent uses all memory and crashes during a resync **URL:** \ **Category:** MongoDB\ **Created:** [April 13, 2023, 8:33pm UTC](https://forums.percona.com/t/pbm-agent-uses-all-memory-and-crashes-during-a-resync/21394 "2023-04-13T20:33:11Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![gregl](https://avatars.discourse-cdn.com/v4/letter/g/c57346/32.png) [@gregl](https://forums.percona.com/u/gregl)\ **Post date:** [April 13, 2023, 8:33pm UTC](https://forums.percona.com/t/pbm-agent-uses-all-memory-and-crashes-during-a-resync/21394/1 "2023-04-13T20:33:11Z")
I run the following command and pbm-agent uses all memory on my server then crashes because of OOM pbm config --force-resync --mongodb-uri=$PBM\_MONGODB\_URI It shows it reading the json file, uses all memory (over 64GB) then gets stopped This is the last entry in journal before crash Apr 13 15:27:31 CARSB61MON001 pbm-agent[14261]: 2023-04-13T15:27:31.000-0500 D [resync] bcp: 2023-03-16T03:01:01Z.pbm.json Has anyone seen this and know of a fix? I have added memory and gone to 8 cpu and 128gb ram same result I have matched both mongodb versions and pbm versions also tried with newer versions pbm ver 2.0.2 mongodb 4.4.12-1 ---
**Author:** ![Pep\_Pla](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/pep_pla/32/9735_2.png) [@Pep\_Pla](https://forums.percona.com/u/Pep_Pla)\ **Post date:** [April 14, 2023, 9:30am UTC](https://forums.percona.com/t/pbm-agent-uses-all-memory-and-crashes-during-a-resync/21394/2 "2023-04-14T09:30:33Z")
Hi @gregl Several factors can trigger suboptimal memory usage. If memory pressure grows, OOM Killer tends to kill the process demanding more memory. Things to check: - Swappiness and NUMA: [MongoDB Best Practices 2020 Edition](https://www.percona.com/blog/mongodb-best-practices-2020-edition/) - Transparent Huge Pages: [https://www.mongodb.com/docs/v6.0/tutorial/transparent-huge-pages/](https://www.mongodb.com/docs/v6.0/tutorial/transparent-huge-pages/) - Disable OOM for a certain process or disable it completely: [How to Adjust Linux Out-Of-Memory Killer Settings for PostgreSQL](https://www.percona.com/blog/out-of-memory-killer-or-savior/) - Memory management subsystem: [SLES 15 SP4 | System Analysis and Tuning Guide | Tuning the memory management subsystem](https://documentation.suse.com/sles/15-SP4/html/SLES-all/cha-tuning-memory.html) I guess that a backup/restore operation uses a lot of filesystem page caching, and the OOM Killer just kills the process that is more memory demanding. I would try: - Reduce swappiness. - Increase vfs\_cache\_pressure - Disable OOM Killer - Disable THP I hope this helps. Pep ---
**Author:** ![gregl](https://avatars.discourse-cdn.com/v4/letter/g/c57346/32.png) [@gregl](https://forums.percona.com/u/gregl)\ **Post date:** [April 14, 2023, 7:06pm UTC](https://forums.percona.com/t/pbm-agent-uses-all-memory-and-crashes-during-a-resync/21394/3 "2023-04-14T19:06:11Z")
This is not MongoDB doing this but pbm-agent that was consuming memory until killed by OOM. I watched was was using memory and it was pbm-agent growing to consume all memory in the span of 2 minutes. I added over 128GB of RAM so I don’t think this is a memory optimization issue. I even tried limiting memory used by pbm-agent per documentation in the config. It didn’t help. After doing some research I saw that I used version 1.6.1 in my repo originally. This problem stopped happening once I got the older version (1.6.1) installed on the system I was restoring to. The really odd part is if I copied a backup that was done with a newer version (2.0.2) I still had to restore it with the older version of pbm-agent. Apparently there is some serious differences between file formats versions that the newer version does not like. ---
**Author:** ![gregl](https://avatars.discourse-cdn.com/v4/letter/g/c57346/32.png) [@gregl](https://forums.percona.com/u/gregl)\ **Post date:** [April 15, 2023, 7:48pm UTC](https://forums.percona.com/t/pbm-agent-uses-all-memory-and-crashes-during-a-resync/21394/4 "2023-04-15T19:48:15Z")
My comments aside, Thank for the well thought response.