Hi,
I’ve been able to enable support for facebooks (amazing) Zstandard compression in Percona MongoDB, by simply modifying the wiredtiger SConscript file (using apt-get source percona-server-mongodb-34).
I did various benchmarks (with that patch) that have proven that ZSTD offers a way better compression/performance ratio, than snappy or zlib (see [URL]https://github.com/facebook/zstd[/URL] for more details)
Looking forward to see it default supported in Percona Server for MongoDB
--- percona-server-mongodb-34-3.4.4.orig/src/third_party/wiredtiger/SConstruct
+++ percona-server-mongodb-34-3.4.4/src/third_party/wiredtiger/SConstruct
@@ -114,6 +114,7 @@ env['STATIC_AND_SHARED_OBJECTS_ARE_THE_S
useZlib = GetOption("zlib")
useSnappy = GetOption("snappy")
+useZstd = true
useLz4 = GetOption("lz4")
useBdb = GetOption("bdb")
useTcmalloc = GetOption("tcmalloc")
@@ -144,6 +145,15 @@ if useSnappy:
print 'snappy-c.h must be installed!'
Exit(1)
+if useZstd:
+ if conf.CheckCHeader('zstd.h'):
+ env.Append(CPPDEFINES=['HAVE_BUILTIN_EXTENSION_ZSTD'])
+ wtlibs.append("zstd")
+ else:
+ print 'zstd.h must be installed!'
+ Exit(1)
+
+
if useLz4:
conf.env.Append(CPPPATH=[useLz4 + "/include"])
conf.env.Append(LIBPATH=[useLz4 + "/lib"])
@@ -270,6 +280,9 @@ if useZlib:
if useSnappy:
wtsources.append("ext/compressors/snappy/snappy_compress.c")
+if useZstd:
+ wtsources.append("ext/compressors/zstd/zstd_compress.c")
+
if useLz4:
wtsources.append("ext/compressors/lz4/lz4_compress.c")
To compile (tested on Ubuntu) you need to install: libzstd-dev libzstd1
Since zstd is not native supported in MongoDB, only directly via wiredtiger you need to use:
db.createCollection( "myNewCol", { storageEngine: { wiredTiger: { configString: "block_compressor=zstd" }}})
or
wiredTiger:
engineConfig:
cacheSizeGB: 1
checkpointSizeMB: 1000
statisticsLogDelaySecs: 0
journalCompressor: none
directoryForIndexes: false
collectionConfig:
blockCompressor: zlib
configString: "block_compressor=zstd"
indexConfig:
prefixCompression: true
configString: "block_compressor=zstd"
optional you can also set “compression_level” which is defautl set to “3” (I’ve not yet tested)