# Memory usage of InMemory Storage Engine

**URL:** <https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054>\
**Category:** Percona Server for MongoDB\
**Created:** [August 25, 2016, 9:36pm UTC](https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054 "2016-08-25T21:36:22Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Seunguck\_Lee](https://avatars.discourse-cdn.com/v4/letter/s/e47c2d/32.png) [@Seunguck\_Lee](https://forums.percona.com/u/Seunguck_Lee)\
**Post date:** [August 25, 2016, 9:36pm UTC](https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054/1 "2016-08-25T21:36:22Z")

</div>

I have tested MongoDB InMemory storage engine.

Each document is about 50~100 bytes and there’s only 50000 document in the collection (Just one collection in mongodb instance).  
And there’s 3 indexes( PK, TTL index, and composite 2dsphere index).

Made 4K connections and load generator ran 45K/second update (there’s only update which change TTL and 2dsphere index)

$ ./bin/mongod --storageEngine=inMemory --inMemorySizeGB=3

After 3 day’s running, MongoDB instance consume about 18.5GB memory. But db.serverStatus().inMemory.cache.“bytes currently in the cache” metric is only below 3GB.  
I don’t know this is normal that MongoDB instance use 15GB only for client connection handler threads.  
But after stop load generator and close all connection, mongodb instance memory usage doesn’t show any changes.

So I tried google perf tools with rebuilt mongodb(–allocator=system)

$ env HEAPPROFILE=/data/profile/hprof HEAP\_PROFILE\_ALLOCATION\_INTERVAL=10737418240 HEAP\_PROFILE\_INUSE\_INTERVAL=1073741824 HEAP\_PROFILE\_TIME\_INTERVAL=600 ./bin/mongod -f ./mongod.conf

After 12 hours running, MongoDB instance is using 5.0GB and wired tiger cache is using only 650MB(678575506 bytes).  
And generate memory usage graph with pprof. But more weird things is that pprof graph shows only 777MB memory usage.

Does mongodb allocate memory out side of tcmalloc api ?  
Are there any missing point in my profiling procedure ?

Regards,  
Matt.

[pprof\_0177.pdf](https://forums.percona.com/uploads/short-url/tVlr1hN714bmniHfkJcxXyaxmYF.pdf) (14.4 KB)

[pprof\_0177.txt](https://forums.percona.com/uploads/short-url/37hIWK1WjnMMJ81bULpi2zRWrxh.txt) (10.3 KB)

---

<div class="post-metadata">

**Author:** ![vadimtk](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vadimtk/32/9528_2.png) [@vadimtk](https://forums.percona.com/u/vadimtk)\
**Post date:** [August 26, 2016, 9:10am UTC](https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054/2 "2016-08-26T09:10:23Z")

</div>

There should not be places that allocates memory outside of tcmalloc.

How do you measure 18GB memory usage?

---

<div class="post-metadata">

**Author:** ![Seunguck\_Lee](https://avatars.discourse-cdn.com/v4/letter/s/e47c2d/32.png) [@Seunguck\_Lee](https://forums.percona.com/u/Seunguck_Lee)\
**Post date:** [August 28, 2016, 8:45am UTC](https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054/3 "2016-08-28T08:45:32Z")

</div>

Hi Vadim.

I just check that with “top” and “ps” and “free” command in linux.  
Of course there could be some gap with real memory usage, but not supposed to be this big.

Have you profiled mongodb memory usage with google-perf tools ?

---

<div class="post-metadata">

**Author:** ![vadimtk](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vadimtk/32/9528_2.png) [@vadimtk](https://forums.percona.com/u/vadimtk)\
**Post date:** [August 30, 2016, 10:16am UTC](https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054/4 "2016-08-30T10:16:51Z")

</div>

Matt,

What column do you check in “top” and “ps” ?

---

<div class="post-metadata">

**Author:** ![Seunguck\_Lee](https://avatars.discourse-cdn.com/v4/letter/s/e47c2d/32.png) [@Seunguck\_Lee](https://forums.percona.com/u/Seunguck_Lee)\
**Post date:** [August 30, 2016, 9:55pm UTC](https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054/5 "2016-08-30T21:55:25Z")

</div>

Resident size of top command. (Virtual size is much greater than RSS in top result)

---

<div class="post-metadata">

**Author:** ![mipa](https://avatars.discourse-cdn.com/v4/letter/m/e0b2c6/32.png) [@mipa](https://forums.percona.com/u/mipa)\
**Post date:** [November 24, 2016, 12:42pm UTC](https://forums.percona.com/t/memory-usage-of-inmemory-storage-engine/5054/6 "2016-11-24T12:42:26Z")

</div>

running into the same “issue”. any news on that?

mongodb says (using only one collection):

“count” : 283180000,  
“size” : 27978383355,  
“storageSize” : 27978383355,  
“totalIndexSize” : 23503940000,  
“indexSizes” : {  
“_id_” : 5380420000,  
“\_id\_hashed” : 4530880000,  
“date\_1” : 3114980000,  
“fileid\_1\_date\_1” : 4530880000,  
“ip\_1\_date\_1\_fileid\_1” : 5946780000  
},  
“avgObjSize” : 98.80070398686348,

but top says 140GB RES memory allocated.