Hi all, recently my mongodb instance has seen massive memory growth during the day as my application is used by several dozen users. From what I have seen so far, the memory grows throughout the day, and is not related to total user connections which remain steady from the beginning of the day (when people open the app).
It also does not appear to be wired tiger which is at approximately 50% as expected.
I have noticed “total cursors” much higher than pinned and feel this is an avenue to look into however there is little in the tooling or stats to be able to see these seemingly internal cursors.
For reference total cursors could be 2-4 thousand growing during the day with active sticking around 200.
I have also noticed a failover does not kill the cursors many stay around for several hours.
I can confirm this is seen on MongoDb 5 and 6 as well as persona vs community.
The application does a variety of operations, from inserts, basic finds, getmore and change streams.
Looking for any suggested avenues for investigation or common application pitfalls for memory consumption.
About 9 out of 10 MongoDB “memory leak” issues are caused by that freelist. Some workloads cause that kind of bloat, and adjusting the tcmallocReleaseRate parameter will fix it.
So, I would go by checking db.serverStatus().tcmalloc to see if there is any freelist, and maybe you can consider setting tcmallocReleaseRate:5 (default is 1)
Yunus’s suggestion works well in most cases to release the unused memory. Added to that, MongoDB v8.0 has many new features added and it handles tcmalloc which inturn uses per-CPU caches instead of per-thread caches. See here. So you can explore psmdb v8.0 for better performance.
Also, regarding the cursors, you can check how the application is handling those cursors. whether they are closed after finishing the work or keep it idle etc. This logic change will help with the cursor most of the time.
Thank you both for the suggestions. The tcmalloc change sounds promising, I will test this. If it helps memory usage that is the main issue.
It would still be unclear why the change in usage happened as there are several applications and users so it is hard to pinpoint behaviour or change that caused the recent increase. Cursors seems like a logical culprit based on a corresponding increase in the “total” not pinned number but it seems generally there are no good ways to identify them other than at application level.
I’ll leave this thread open and reply after testing the tcmalloc change for reference to anyone with similar issues.
While the setting did reduce the free page space, the overall usage remains higher. The free page space only accounted for a small percentage of overall ram.
It seems this is down to application overuse or sanitisation with the cursor rather than something that mongo can adjust.