We have a large MongoDB database (about 400GB data file each day), MongoDB 3.2.6/3.2.7, engine rocksdb, operating system redhat 6.3.
After dropping the collection or database, the disk space used on the machine was not reclaimed.
After rm some sst files which i think should be dropped by rocksdb,mongodb can still work fine.
By the way,show dbs is not corresponding to du in linux.
Best Regards
Hi Ysanne,
It seems like the size of delete messages is small and you probably have a low priority and second, nodes will not have messages flushed unless it is in cache. Therefore, cleanup is not always guaranteed and an explicit reIndex() should be done. A reIndex() operation may not always result in a smaller file due to several factors including when checkpoints have been made and where the end of file marker is. it’s worth reindexing to improve performance in most cases.
One strategy would be to perform a reIndex() on the secondary(ies), then stepDown the primary and reindex on that host once it becomes a secondary. This will work if all the workload is directed to the primary, if you direct reads to the secondaries, you’ll want to account for the workload and likely reindex in the background.
Another point is, that you need to repeat the procedure of reIndex() because that command is not replicated to secondaries.
Another workaround for the situation is to execute repair but you need space disk available. The following steps will help you to reclaim disk space on Mongodb. This is also assuming there is not enough disk space locally and uses some temporary space such as a NFS mount. The repair requires as much space as is currently being used to run.
Here are an example of the steps to follow:
Steps
Create Temp Directory on some available mounted space.
sudo mkdir -p /mnt/nfs_share/repair_tmp
Create Temp Symlink (Have to do this if journeling enabled)
sudo ln -s /mnt/nfs_share/repair_tmp /var/lib/mongo/repair_tmp
Set ownership of temp directory to mongod user and group
sudo chown -R mongod.mongod /mnt/nfs_share/repair_tmp
Stop mongod daemon
sudo service mongod stop
Switch to mongod user
sudo su mongod
Run repair which will reclaim the disk space.
mongod --dbpath /var/lib/mongo --repair --repairpath /var/lib/mongo/repair_tmp
A third option would be : (Warning: This could take long if the database is big)
- On one secondary stop mongodb
- Wipe data directory
- Start mongodb instance again
- Let the server to complete the full resync
Repeat these steps for the other secondaries
Then on primary, run rs.stepDown() , this will provoke primary becomes secondary, and then repeat the steps once again.
regards,
hi,luis.contreras
Our mongodb is standalone.
I have post this question on github:
[url]https://github.com/mongodb-partners/mongo-rocks/issues/32[/url]