serverSideEncryption.sseCustomerKey ignored when authenticating with IRSA

MongoDB Percona Operator for MongoDB
1

Description:

When using an s3 storage for backups with sse-c (customer-provided keys) and authenticating with IRSA, the serverSideEncryption.sseCustomerKey is not pushed by the operator to PLM config, which make the backups unencrypted.

Steps to Reproduce:

Deploying a cluster with the following configuration :

secrets:
  encryptionKey: percona-mongodb-mongodb-encryption-key
  keyFile: percona-mongodb-mongodb-keyfile
  sse: percona-mongodb-s3-backup-sse-key
  users: percona-mongodb-users
replsets:
  rs0:
    name: rs0
    size: 3
    serviceAccountName: percona-mongodb
    ...
backup:
  enabled: true
  storages:
    s3-daily:
      main: true
      type: s3
      s3:
        bucket: percona-mongodb-backups
        prefix: daily
        region: us-west-2
        storageClass: STANDARD
        serverSideEncryption:
          sseCustomerAlgorithm: AES256

Authentication on S3 is done with IRSA.

After deployment, the resulting pbm config is :

$ pbm config
storage:
  type: s3
  s3:
    region: us-west-2
    forcePathStyle: true
    bucket: percona-mongodb-backups
    prefix: daily
    credentials: {}
    maxUploadParts: 10000
    storageClass: STANDARD
    insecureSkipTLSVerify: false
pitr:
  enabled: false
  compression: s2
backup:
  oplogSpanMin: 0
  compression: s2
restore: {}

Version:

percona-server-mongodb-operator 1.20.1
percona-backup-mongodb:2.9.1
percona-server-mongodb:7.0.18-11

Logs:

No error found in logs.

Expected Result:

Backups encrypted with the provided sse-c key.

Actual Result:

Backups are pushed unencrypted to the bucket, which can be a little tricky for someone not verifying that the backups are effectively encrypted.

Additional Information:

The issue is coming from this line : percona-server-mongodb-operator/pkg/psmdb/backup/pbm.go at v1.20.1 · percona/percona-server-mongodb-operator · GitHub

I was able to workaround it by creating an empty secret and referring it in my backup config :

backup:
  enabled: true
  storages:
    s3-daily:
      main: true
      type: s3
      s3:
        bucket: percona-mongodb-backups
        prefix: daily
        region: us-west-2
        storageClass: STANDARD
        credentialsSecret: percona-mongodb-s3-dummy
        serverSideEncryption:
          sseCustomerAlgorithm: AES256

Authentication still works by using IRSA and the plm config is generated as expected :

storage:
  type: s3
  s3:
    region: us-west-2
    forcePathStyle: true
    bucket: myscript-us-west-2-preprod-license-percona-mongodb-backups
    prefix: daily
    credentials: {}
    serverSideEncryption:
      sseAlgorithm: ""
      kmsKeyID: ""
      sseCustomerAlgorithm: AES256
      sseCustomerKey: '***'
    maxUploadParts: 10000
    storageClass: STANDARD
    insecureSkipTLSVerify: false
pitr:
  enabled: false
  compression: s2
backup:
  oplogSpanMin: 0
  compression: s2
restore: {}

Backups files are then encrypted with the aes key provided in sse secret.

I have a concern, however, that the workaround I used may stop working in a future version. Do you think it would be possible to make the operator accept sse-c keys with oidc authentication natively ?

2

@gnobiron Thank you for your report and the detailed description.
I’ve checked the code and confirmed that it’s a bug. We’ll fix it in one of the upcoming releases.
You can track the task here: Jira

1 Like