Token rotation with Hashicorp Vault integration

MongoDB Percona Server for MongoDB
1

I want to have at rest encryption for MongoDB with Hashicorp Vault.
The problem is that my vault only issues me an approle.
I can use it to get a token using the approle login, however that token has a ttl of 1h and a max ttl of 1h as well.
I’m wondering how much that can cause an issue with Percona MongoDB, does it just need to get the key at boot time and then keeps it in memory ? If it’s the case, I could just regenerate a new token before each restart and that would be sufficient for my use case.

If not, I’m wondering if Percona Server has a way to rotate tokens ? Otherwise, if I can rotate my secret with my approle in an automated way, is it possible to reload the server so I can avoid restarts ? If I have to restart my server every hour, that is not manageable…

Final question, but that would be a feature request probably, do you think it would be possible in the future for Percona MongoDB to have builtin approle support ?

2

Hi @Gu1nn3zz , and welcome back to our forum! It has been 10 months since your last visit. Looking forward to hearing more from you :slight_smile: . Feedback is gold, as they say.

Here are answers to your question:

Does PSMDB keep the key in memory?

Yes. Percona Server for MongoDB fetches the Master Key from Vault during the initial startup sequence. This Master Key is then used to decrypt the internal “key database” (which contains the actual encryption keys for your data files). See also our docs: Data at rest encryption - Percona Server for MongoDB 8.0

Once the server has successfully started and decrypted the internal keys, it keeps the Master Key in memory. It does not reach out to Vault for every read or write operation.

Can you just regenerate the token before each restart?

Technically, yes (but watch out, rather not recommended!). If you only care about the server being able to start, you could manually (or via script) generate a new token, update it via the command vaultTokenFile (config: security.vault.tokenFile) , and then start/restart the server. As long as the token is valid at the moment mongod it boots, the server will function until the next time it needs to access Vault.

Here’s a risk: The problem occurs if the server needs to access Vault while running. This happens:

  • When a Master Key is rotated. For example, If you trigger a manual rotation of the master key.

  • In case of a process crash: If the process restarts unexpectedly after the 1h TTL has passed, it will fail to boot because the token in the file is no longer valid.

Since your Vault only issues AppRoles, the industry-standard way to handle this without manual intervention or frequent restarts is to use a Vault Agent as a sidecar or a local daemon.

  • The Vault Agent stays active on the server. It handles the AppRole authentication and handles the background task of renewing the token or fetching a new one before it expires.

  • You configure the Vault Agent to write the token to a “sink” file. You then point your PSMDB security.vault.tokenFile configuration to that exact file. Care about file permissions! Details here: https://developer.hashicorp.com/vault/docs/agent-and-proxy/autoauth/sinks/file

PSMDB won’t “hot-reload” the token during a rotation command, so the file on disk will always be valid. If the server ever crashes or needs a maintenance restart, it will always find a fresh, valid token waiting for it.

Is there a way to rotate tokens or reload them without restarting?

Currently, PSMDB does not have a “reload” mechanism for the Vault token file while the server is running. Thanks for the feedback! I’ll write this product extension idea.

3

Hi !

Thanks for the detailed answer !

I love the approach using the vault agent! That really looks like what I need and I will go towards that solution :slight_smile:

As always, thanks for the quick answer and the help!

4

Cool, let us know how it worked :slight_smile: