There have been different approaches to host a lot of databases in a single MongoDB server. As many tickets in MongoDB Jira already show, there are still issues.
In our project the goal is to host at least 10M databases and now it looks like we have found a solution:
Lets say you name the databases with numbers from 1-n. And naively one would create x=3 MongoDB clusters. Then these databases can be distributed to the clusters by using n mod x. Obviously if n is getting high, there will be too many databases/collection per MongoDB cluster.
We have found that 50’000 dbs each with approx 10 coll (=1M coll+index files) can be hosted on a single cluster. A restart (after a crash or pod rescheduling) would take approx 15min.
So in order to host 10M db we would need 200 clusters - which would make Percona happy, if you have a support contract. Nevertheless it would be feasible to manage them - but it takes a lot of resources.
To avoid that, we are swapping out the collections (=backup) if they are not needed and since our collections are small, this can be done on the fly (restore) and with a CronJob that runs every couple of minutes for backup.
So it looks like with 16-32 clusters you are able to host 10M dbs and this system is incredible fast (especially for writes) - because of the additional parallelization.
NAME READY STATUS RESTARTS AGE pod/mongodb00-cluster-rs-0 1/1 Running 0 4m33s pod/mongodb00-cluster-rs-1 1/1 Running 0 3m51s pod/mongodb00-cluster-rs-2 1/1 Running 0 3m12s pod/mongodb01-cluster-rs-0 1/1 Running 0 4m29s pod/mongodb01-cluster-rs-1 1/1 Running 0 3m44s pod/mongodb01-cluster-rs-2 1/1 Running 0 2m59s pod/mongodb02-cluster-rs-0 1/1 Running 0 4m25s pod/mongodb02-cluster-rs-1 1/1 Running 0 3m40s pod/mongodb02-cluster-rs-2 1/1 Running 0 3m1s pod/mongodb03-cluster-rs-0 1/1 Running 0 4m22s pod/mongodb03-cluster-rs-1 1/1 Running 0 3m34s pod/mongodb03-cluster-rs-2 1/1 Running 0 2m52s pod/mongodb04-cluster-rs-0 1/1 Running 0 4m19s pod/mongodb04-cluster-rs-1 1/1 Running 0 3m34s pod/mongodb04-cluster-rs-2 1/1 Running 0 2m48s pod/mongodb05-cluster-rs-0 1/1 Running 0 4m14s pod/mongodb05-cluster-rs-1 1/1 Running 0 3m26s pod/mongodb05-cluster-rs-2 1/1 Running 0 2m35s pod/mongodb06-cluster-rs-0 1/1 Running 0 4m11s pod/mongodb06-cluster-rs-1 1/1 Running 0 3m22s pod/mongodb06-cluster-rs-2 1/1 Running 0 2m37s pod/mongodb07-cluster-rs-0 1/1 Running 0 4m8s pod/mongodb07-cluster-rs-1 1/1 Running 0 3m20s pod/mongodb07-cluster-rs-2 1/1 Running 0 2m28s ... pod/percona-server-mongodb-operator-f674f67b7-q9w6g 1/1 Running 0 91m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/mongodb00-cluster-rs ClusterIP None <none> 27017/TCP 4m34s service/mongodb01-cluster-rs ClusterIP None <none> 27017/TCP 4m30s service/mongodb02-cluster-rs ClusterIP None <none> 27017/TCP 4m26s service/mongodb03-cluster-rs ClusterIP None <none> 27017/TCP 4m23s service/mongodb04-cluster-rs ClusterIP None <none> 27017/TCP 4m20s service/mongodb05-cluster-rs ClusterIP None <none> 27017/TCP 4m15s service/mongodb06-cluster-rs ClusterIP None <none> 27017/TCP 4m12s service/mongodb07-cluster-rs ClusterIP None <none> 27017/TCP 4m9s ... NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/percona-server-mongodb-operator 1/1 1 1 91m NAME DESIRED CURRENT READY AGE replicaset.apps/percona-server-mongodb-operator-f674f67b7 1 1 1 91m NAME READY AGE statefulset.apps/mongodb00-cluster-rs 3/3 4m34s statefulset.apps/mongodb01-cluster-rs 3/3 4m30s statefulset.apps/mongodb02-cluster-rs 3/3 4m26s statefulset.apps/mongodb03-cluster-rs 3/3 4m23s statefulset.apps/mongodb04-cluster-rs 3/3 4m20s statefulset.apps/mongodb05-cluster-rs 3/3 4m15s statefulset.apps/mongodb06-cluster-rs 3/3 4m12s statefulset.apps/mongodb07-cluster-rs 3/3 4m9s ...
Happy New Year !