Hi,
I am newly in DB
I just trying to install percona mongodb and try to encrypt my data.
Tried all things but can’t able to do it
can someone help me with mongodb percona with encryption.
Hi Vijendra.
Setting up Hashicorp Vault is another matter beyond what this particular forum will help with. If that’s the part you’re having trouble with may I suggest first practicing using Data-at-rest encryption using only local keys.
Data at Rest Encryption # Local key management using a keyfile
But using a local key file is like leaving a key in an easy-to-find place, in the same room as the safe it locks. So you will have to master using Hashicorp Vault too to achieve a good Data-at-rest solution for your business.
I got error while restarting MongoDB service after enabling vault in mongodb.conf
{"t":{"$date":"2021-03-23T18:32:06.239+05:30"},"s":"I", "c":"CONTROL", "id":20698, "ctx":"main","msg":"***** SERVER RESTARTED *****"}
{"t":{"$date":"2021-03-23T18:32:06.249+05:30"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2021-03-23T18:32:06.253+05:30"},"s":"W", "c":"ASIO", "id":22601, "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2021-03-23T18:32:06.254+05:30"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2021-03-23T18:32:06.255+05:30"},"s":"I", "c":"STORAGE", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":4648,"port":27017,"dbPath":"/var/lib/mongodb","architecture":"64-bit","host":"percona-VirtualBox"}}
{"t":{"$date":"2021-03-23T18:32:06.255+05:30"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.4-6","gitVersion":"f3dd4bc7c7500705a537de40bb4d6127ba498bd3","openSSLVersion":"OpenSSL 1.1.1 11 Sep 2018","modules":[],"allocator":"tcmalloc","environment":{"distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2021-03-23T18:32:06.255+05:30"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"18.04"}}}
{"t":{"$date":"2021-03-23T18:32:06.255+05:30"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/etc/mongod.conf","net":{"bindIp":"127.0.0.1","port":27017},"processManagement":{"fork":true,"pidFilePath":"/var/run/mongod.pid"},"security":{"enableEncryption":true,"vault":{"port":8200,"secret":"secret/data/dc/psmongodb1","serverCAFile":"/etc/mongodb/vault.crt","serverName":"192.168.159.238","tokenFile":"/etc/mongodb/token"}},"storage":{"dbPath":"/var/lib/mongodb","journal":{"enabled":true}},"systemLog":{"destination":"file","logAppend":true,"path":"/var/log/mongodb/mongod.log"}}}}
{"t":{"$date":"2021-03-23T18:32:06.257+05:30"},"s":"I", "c":"STORAGE", "id":22270, "ctx":"initandlisten","msg":"Storage engine to use detected by data files","attr":{"dbpath":"/var/lib/mongodb","storageEngine":"wiredTiger"}}
{"t":{"$date":"2021-03-23T18:32:06.257+05:30"},"s":"I", "c":"STORAGE", "id":22297, "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
{"t":{"$date":"2021-03-23T18:32:06.267+05:30"},"s":"I", "c":"STORAGE", "id":29037, "ctx":"initandlisten","msg":"Initializing KeyDB with wiredtiger_open config: {cfg}","attr":{"cfg":"create,config_base=false,extensions=[local=(entry=percona_encryption_extension_init,early_load=true,config=(cipher=AES256-CBC,rotation=false))],encryption=(name=percona,keyid=\"\"),log=(enabled,file_max=5MB),transaction_sync=(enabled=true,method=fsync),"}}
{"t":{"$date":"2021-03-23T18:32:07.070+05:30"},"s":"I", "c":"STORAGE", "id":29039, "ctx":"initandlisten","msg":"Encryption keys DB is initialized successfully"}
{"t":{"$date":"2021-03-23T18:32:07.070+05:30"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=481M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress,compact_progress],encryption=(name=percona,keyid=\"/default\"),extensions=[local=(entry=percona_encryption_extension_init,early_load=true,config=(cipher=AES256-CBC)),],"}}
{"t":{"$date":"2021-03-23T18:32:07.134+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:134668][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 606: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.136+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:136083][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 612: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.137+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:136977][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 615: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.137+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:137593][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 618: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.156+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:156562][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 606: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.157+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:157444][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 612: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.158+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:158097][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 615: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.159+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:159056][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 618: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.181+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:181090][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 606: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.182+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:182249][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 612: WiredTiger has failed to open its metadata: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.182+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:182908][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 615: This may be due to the database files being encrypted, being from an older version or due to corruption on disk: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.183+05:30"},"s":"E", "c":"STORAGE", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error","attr":{"error":-31802,"message":"[1616504527:183526][4648:0x7f79a21cc140], file:WiredTiger.wt, connection: __wt_btree_tree_open, 618: You should confirm that you have opened the database with the correct options including all encryption and compression options: WT_ERROR: non-specific WiredTiger error"}}
{"t":{"$date":"2021-03-23T18:32:07.187+05:30"},"s":"W", "c":"STORAGE", "id":22347, "ctx":"initandlisten","msg":"Failed to start up WiredTiger under any compatibility version. This may be due to an unsupported upgrade or downgrade."}
{"t":{"$date":"2021-03-23T18:32:07.187+05:30"},"s":"W", "c":"STORAGE", "id":22348, "ctx":"initandlisten","msg":"WiredTiger metadata corruption detected"}
{"t":{"$date":"2021-03-23T18:32:07.187+05:30"},"s":"F", "c":"STORAGE", "id":50944, "ctx":"initandlisten","msg":"Please read the documentation for starting MongoDB with --repair here: http://dochub.mongodb.org/core/repair"}
{"t":{"$date":"2021-03-23T18:32:07.188+05:30"},"s":"F", "c":"-", "id":23091, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":50944,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp","line":1293}}
{"t":{"$date":"2021-03-23T18:32:07.188+05:30"},"s":"F", "c":"-", "id":23092, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}
Did you manage to solve the issue?
I have managed to fix the issue. Leaving the solution here in case any one needs it.
The issue is due to the existing data in the dbPath. You need to delete all the files within that directory.
You can check the directory in the /etc/mongod.conf file. By default it’s either /data/mongo or /var/lib/mongodb.
For my machine, it’s /var/lib/mongodb. So i removed all the files under that directory.
Then start mongodb using - “sudo mongod --config /etc/mongod.conf”.
Yes, you cannot turn on/off encryption on existing data.
In other words if you plan to use encryption you need to start from empty data directory with encryption enabled.