Logrotate helm configuration for psmdb-operator

MongoDB
1

Hi all,

I am using the psmdb-operator for a MongoDB deployement. It works great and I love it. However, the logfiles the database generates can grow quite a bit and unfortunately, they are uncompressed.

I stumbled over this article, that describes how to overwrite the logrotate configuration: Log rotation - Percona Operator for MongoDB

But apparently there is no support for it in the helm chart, there is no logrotate section in the logcollector. Will support for this be added in the future or what is the recommended way to enable compression of the rotated logs when using a helm chart to deploy the psmdb-db? I am using the latest version of the chart 1.22.2 with operator 1.22.0.

2

Hi, even though there is no helm parameter you should be able to customize by providing a yaml to helm. For example something like:

tee values.yaml <<'EOF'
logcollector:
  enabled: true
  logrotate:
    schedule: "0 15 2 * * 0"
    configuration: |
      /data/db/logs/mongod.full.log {
         daily
         minsize 100K
         maxsize 200M
         rotate 14
         compress
         missingok
         notifempty
         copytruncate
         sharedscripts
         prerotate
             mongosh "mongodb://${MONGODB_USER}:${MONGODB_PASSWORD}@${MONGODB_HOST}:${MONGODB_PORT}/admin" \
                     --eval 'db.adminCommand({ logRotate: 1 })'
             find /data/db/logs/ -type f -name 'mongod.log.*' -mtime +1 -delete
         endscript
      }
EOF

then install with:

helm upgrade --install cluster1 percona/psmdb-db \
  -n mongodb \
  -f values.yaml

keep in mind this functionality appeared only in operator 1.22 or newer.

3

Hi Ivan,

I am using ArgoCD to deploy the database, running helm upgrade locally succeeds but does not update my deployement. I created a values-log.yaml and have included that in my namespace configuration . I see the parameters applied in ArgoCD, however, the operator does not seem to update the pods.

I deleted the pods and let the operator recreate them. I also issued a reconcile on the operator but nothing changes. Am I right to assume that this should update the config /opt/percona/logcollector/logrotate/logrotate.conf in the pod, do I need to do something else for the operator to pick up the config?

And yes I have operator version 1.22.0 - or do I need to deploy that config on the operator itself?

4

Hi, I think there is a bug on the helm chart, so it’s not taking your custom config… we are looking into this with eng team.

5

What would be a dream scenario is when we could configure logrotate and fluentbit filters directly in helm without additional yamls, especially when the default is nocompress. My testdb has only about 50mb of data but produces several hundred of mb logfiles each day, mainly ACCESS and NETWORK from liveliness probes and logins from psmdb-pbm-operator. When I gzip them myself they go down by a factor of 10-15. And, because I have already audit configured logging directly to splunk, I want to filter ACCESS completely from the logfiles which again, apparently is not possible directly in helm.

[EDIT| just a quick update: I deployed a new database with kustomize and plane CRDs and it is working, I have compression enabled now and fluentbit filters work as well:

sh-5.1$ pwd
/opt/percona/logcollector/logrotate/conf.d
sh-5.1$ more mongodb.conf
/data/db/logs/mongod.full.log {
daily
minsize 100K
maxsize 200M
rotate 7
missingok
compress

6

Hi @Raphi79 Thank you for sharing the issue you faced with the helm chart. As @Ivan_Groenewold mentioned the chart needs to be fixed, ticket https://perconadev.atlassian.net/browse/K8SPSMDB-1668 is created and this will be resolved soon.

With this fix, you can change your configuration from the helm chart and values file. The config change doesn’t need any pod restart for the changes to reflect.