Hi,
I’ve collection with 15gb of data, could I partition it now?
I’m interested in 2 options:
- I’ve timestamp field in collection, so could I partition current data?
- Could I just add new partitions daily and remove old one(30 days rotation)?
Hi,
I’ve collection with 15gb of data, could I partition it now?
I’m interested in 2 options:
Yes, you can absolutely partition your 15GB collection using both of the methods you’ve described.
This approach in MongoDB is called sharding. Sharding horizontally scales your database by distributing a single collection’s data across multiple servers (called shards). You can definitely shard your existing 15GB collection. However, choosing the right shard key is critical.
This is an excellent design pattern for managing time-series data and is often a better fit than sharding for managing data lifecycle. It doesn’t use the built-in sharding feature but rather a logical separation at the application level. Your application logic should determine the current date and writes new documents to the corresponding daily collection (e.g., my_collection_2025_09_10
). When querying for a specific time range, your application may need to query multiple collections. For example, to get data for the last 3 days, you would query my_collection_2025_09_10
, my_collection_2025_09_09
, and my_collection_2025_09_08
. MongoDB’s aggregation framework with $unionWith
can make this easier. To remove data older than 30 days, you don’t run a slow, resource-intensive deleteMany()
command. Instead, you simply run a db.collection.drop()
command on the old collection (e.g., db.my_collection_2025_08_11.drop()
). Dropping a collection is an extremely fast, metadata-only operation
Unanswered | Unsolved | Solved
MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright © 2006 - 2024 Percona LLC. All rights reserved.