Cluster with encryption at rest deleted - how to recover data


we just had a severe incident where our GKE cluster got deleted. Therefore any resources on it got deleted. Is there a way to recover the MongoDB data after all ? The disks are there but nothing else (even the encryption key is lost) …

  1. How can I recover the encryption key ?
  2. How can I recover mongodb-keyfile / mongodb-key ?
  3. How can I bundle the disks so that they get accepted again as PVCs by the MongoDB cluster ?
  4. Anything else to consider ?


PS: why are the critical keys generated ? If they would be part of the deployment, this would be preferable - so that they do not get forgotten to be backuped.

1 Like

We discussed it in private a bit.
Summarizing it here:

  • It is not possible to recover the cluster if the encryption key is lost. If it is possible, than we have a security flaw.
  • Encryption key is generated by the Operator by default and stored in the Secret object: my-cluster-name-mongodb-encryption-key. User can always generate it manually and store it in another secret.
1 Like