Nginx logs filling up volume

It appears that the Nginx logs are not rotating like the rest of the log files, and are therefor growing out of control.

sudo ls -lah /var/lib/docker/volumes/37d310a60891d69b99d7f8ab76daa059c7211fdbd5e29374836d833a451178e2/_data/logs/
total 1.7G
drwxrwxr-x  2 1000 1000 4.0K Aug 11 14:08 .
drwxr-xr-x 12 root root  189 Feb 16 19:07 ..
-rw-r--r--  1 root root  14K Aug  2 17:52 alertmanager.log
-rw-r-----  1 root root    0 Feb 16 19:07 clickhouse-server.err.log
-rw-r-----  1 root root 704M Aug 11 14:52 clickhouse-server.log
-rw-r-----  1 root root  50M Jun 26 22:28 clickhouse-server.log.0.gz
-rw-r-----  1 root root  50M Apr 22 06:06 clickhouse-server.log.1.gz
-rw-r--r--  1 root root 5.1K Aug  2 17:52 clickhouse-server.startup.log
-rw-r--r--  1 root root    0 Feb 16 19:07 cron.log
-rw-r--r--  1 root root 357K Aug  2 17:52 dashboard-upgrade.log
-rw-r--r--  1 root root 1.4M Aug 11 10:56 grafana.log
-rw-r--r--  1 root root 1.2G Aug 11 14:52 nginx.access.log
-rw-r--r--  1 root root  800M Aug 11 10:56 nginx.error.log
-rw-r--r--  1 root root  112 Aug  2 17:52 nginx.startup.log
-rw-r--r--  1 root root 7.3M Aug 11 14:52 pmm-agent.log
-rw-r--r--  1 root root  11M Jul 20 14:09 pmm-agent.log.1
-rw-r--r--  1 root root  11M Jun 19 20:18 pmm-agent.log.2
-rw-r--r--  1 root root  11M May 20 03:35 pmm-agent.log.3
-rw-r--r--  1 root root 5.2M Aug 11 14:52 pmm-managed.log
-rw-r--r--  1 root root  11M Aug 11 14:08 pmm-managed.log.1
-rw-r--r--  1 root root  11M Aug 11 12:42 pmm-managed.log.2
-rw-r--r--  1 root root  11M Aug 11 11:17 pmm-managed.log.3
-rw-r--r--  1 root root  19K Jul 20 12:59 pmm-update-perform.log
-rw-------  1   26   26  11K Aug  2 17:52 postgresql.log
-rw-r--r--  1 root root  12K Aug  2 17:52 postgresql.startup.log
-rw-r--r--  1 root root 5.5M Aug 11 14:52 qan-api2.log
-rw-r--r--  1 root root  11M Aug 10 05:51 qan-api2.log.1
-rw-r--r--  1 root root  11M Aug  7 16:43 qan-api2.log.2
-rw-r--r--  1 root root  11M Aug  5 03:37 qan-api2.log.3
-rw-r--r--  1 root root 131K Aug 11 03:14 supervisord.log
-rw-r--r--  1 root root 6.2M Aug 11 14:52 victoriametrics.log
-rw-r--r--  1 root root  11M Aug  8 15:59 victoriametrics.log.1
-rw-r--r--  1 root root  11M Aug  3 21:03 victoriametrics.log.2
-rw-r--r--  1 root root  11M Jul 30 09:15 victoriametrics.log.3
-rw-r--r--  1 root root 216K Aug  2 17:52 vmalert.lo

I’ve already read over Configure logging drivers | Docker Documentation and checked my /etc/docker/daemon.json, but since the other log files are rotating just fine, this seems to be more of a pmm issue, specific to Nginx.

Thanks!

Hi @dbray_sd thanks for posting to the Percona forums!
In fact if you upgrade to PMM 2.20 you’ll be able to take advantage of proper log rotation of all files in /srv/logs. Let us know your experiences!
https://jira.percona.com/browse/PMM-7627

1 Like

@Michael_Coburn wow, can’t believe I missed that there was an upgrade available. Usually I always apply updates before posting about an issue. Either way, I’ve upgraded to the latest 2.20, and waited for 24hrs, but I still see the large nginx.access.log and nginx.error.log files. Is there some additional steps I should perform after the update?

1 Like

Hi @dbray_sd
Have the logs been rotated? if not I’ll want to file a bug

1 Like

Just checked again, and no, the logs are not rotating. I restarted everything to be sure, waited another 24hrs, but the logs are still growing. Thank you.

1 Like

Hi @dbray_sd
I’m checking with the Engineers to see what’s going on. https://jira.percona.com/browse/PMM-7627 should have fixed your use case

1 Like

Hi @dbray_sd,

We use a new file name for nginx logs in PMM 2.20 version:
/srv/logs/nginx.log
Instead:

nginx.startup.log
nginx.error.log
nginx.access.log

The only nginx.log will use and you can remove old log files. nginx.log also will rotate :slight_smile:

1 Like

@nikita.beletskii

Interesting. I’m running PMM 2.20, and I do see the nginx.log file, but it was last written to about a week ago, whereas the other log files are all recent:

$ sudo ls -lah /var/lib/docker/volumes/37d310a60891d69b99d7f8ab76daa059c7211fdbd5e29374836d833a451178e2/_data/logs/ | grep nginx
-rw-r--r--  1 root root 724M Aug 17 16:21 nginx.access.log
-rw-r--r--  1 root root 146K Aug 17 11:02 nginx.error.log
-rw-r--r--  1 root root  336 Aug 11 19:01 nginx.log
-rw-r--r--  1 root root  112 Aug  2 17:52 nginx.startup.log

The above is all in UTC time.

1 Like

Hmm…looks strange. Can you run following commands in container, pls:

cat /etc/supervisord.d/pmm.ini
ps -aux | grep nginx
cat /etc/nginx/nginx.conf | grep access_log

Thank you!

1 Like

Sure thing.

cat /etc/supervisord.d/pmm.ini

[unix_http_server]
chmod = 0700
username = dummy
password = dummy

[supervisord]
logfile = /srv/logs/supervisord.log
pidfile = /tmp/supervisord.pid
nodaemon = true
nocleanup = false
; TODO childlogdir = /srv/logs/supervisor
user = root
strip_ansi = false

[supervisorctl]
username = dummy
password = dummy

[program:postgresql]
priority = 1
command =
    /usr/pgsql-11/bin/postgres
        -D /srv/postgres
        -c shared_preload_libraries=pg_stat_statements
        -c pg_stat_statements.max=10000
        -c pg_stat_statements.track=all
        -c pg_stat_statements.save=off
user = postgres
autorestart = true
autostart = true
startretries = 10
startsecs = 1
stopsignal = INT  ; Fast Shutdown mode
stopwaitsecs = 300
; postgresql.conf contains settings to log to /srv/logs/postgresql.log,
; so log startup messages only (if any) with supervisord
stdout_logfile = /srv/logs/postgresql.startup.log
stdout_logfile_maxbytes = 1MB
stdout_logfile_backups = 3
redirect_stderr = true

[program:clickhouse]
priority = 2
command = /usr/bin/clickhouse-server --config-file=/etc/clickhouse-server/config.xml
autorestart = true
autostart = true
startretries = 10
startsecs = 1
stopsignal = TERM
stopwaitsecs = 300
; config.xml contains settings to log to /srv/logs/clickhouse-server.log and .err.log,
; so log startup messages only (if any) with supervisord
stdout_logfile = /srv/logs/clickhouse-server.startup.log
stdout_logfile_maxbytes = 1MB
stdout_logfile_backups = 3
redirect_stderr = true

[program:grafana]
priority = 3
command =
    /usr/sbin/grafana-server
        --homepath=/usr/share/grafana
        --config=/etc/grafana/grafana.ini
        cfg:default.paths.data=/srv/grafana
        cfg:default.paths.plugins=/var/lib/grafana/plugins
        cfg:default.paths.logs=/srv/logs
        cfg:default.log.mode=console
        cfg:default.log.console.format=console
        cfg:default.server.root_url="%%(protocol)s://%%(domain)s:%%(http_port)s/graph"
user = grafana
directory = /usr/share/grafana
environment = HOME=/usr/share/grafana
autorestart = true
autostart = true
startretries = 10
startsecs = 1
stopsignal = TERM
stopwaitsecs = 300
stdout_logfile = /srv/logs/grafana.log
stdout_logfile_maxbytes = 10MB
stdout_logfile_backups = 3
redirect_stderr = true

[program:nginx]
priority = 4
command = nginx
autorestart = true
autostart = true
startretries = 10
startsecs = 1
stopsignal = TERM
stopwaitsecs = 10
; nginx.conf contains settings to log to /var/log/nginx/error.log and access.log,
; so log startup messages only (if any) with supervisord
stdout_logfile = /srv/logs/nginx.startup.log
stdout_logfile_maxbytes = 1MB
stdout_logfile_backups = 3
redirect_stderr = true

[program:cron]
priority = 5
command = /usr/sbin/crond -n
autorestart = true
autostart = true
startretries = 10
startsecs = 1
stopsignal = TERM
stopwaitsecs = 10
stdout_logfile = /srv/logs/cron.log
stdout_logfile_maxbytes = 10MB
stdout_logfile_backups = 3
redirect_stderr = true

[program:dashboard-upgrade]
priority = 10
command = /usr/share/percona-dashboards/import-dashboards.py
autorestart = unexpected
exitcodes = 0
autostart = true
startretries = 10
startsecs = 0  ; it can exit quickly if there is nothing to do
stopsignal = TERM
stopwaitsecs = 10
stdout_logfile = /srv/logs/dashboard-upgrade.log
stdout_logfile_maxbytes = 10MB
stdout_logfile_backups = 3
redirect_stderr = true

; [program:clickhouse_exporter]
; priority = 11
; command =
;     /usr/local/percona/pmm-client/clickhouse_exporter
;         -telemetry.address=localhost:9116
; stdout_logfile = /srv/logs/clickhouse_exporter.log
; stderr_logfile = /srv/logs/clickhouse_exporter.log
; autorestart = true

[program:pmm-managed]
priority = 14
command =
    /usr/sbin/pmm-managed
        --victoriametrics-config=/etc/victoriametrics-promscrape.yml
        --victoriametrics-url=http://127.0.0.1:9090/prometheus
        --postgres-name=pmm-managed
        --postgres-username=pmm-managed
        --postgres-password=pmm-managed
        --supervisord-config-dir=/etc/supervisord.d
autorestart = true
autostart = true
startretries = 1000
startsecs = 1
stopsignal = TERM
stopwaitsecs = 300
stdout_logfile = /srv/logs/pmm-managed.log
stdout_logfile_maxbytes = 10MB
stdout_logfile_backups = 3
redirect_stderr = true

priority = 15
command = /usr/sbin/pmm-agent --config-file=/usr/local/percona/pmm2/config/pmm-agent.yaml
autorestart = true
autostart = true
startretries = 1000
startsecs = 1
stopsignal = TERM
stopwaitsecs = 10
stdout_logfile = /srv/logs/pmm-agent.log
stdout_logfile_maxbytes = 10MB
stdout_logfile_backups = 3
redirect_stderr = true

[program:pmm-update-perform]
command = /usr/sbin/pmm-update -perform -playbook=/usr/share/pmm-update/ansible/playbook/tasks/update.yml
directory = /
autorestart = unexpected
exitcodes = 0
autostart = false
startretries = 10
startsecs = 1
stopsignal = TERM
stopwaitsecs = 300
stdout_logfile = /srv/logs/pmm-update-perform.log
stdout_logfile_maxbytes = 50MB
stdout_logfile_backups = 3
redirect_stderr = true

ps -aux | grep nginx
root 435 0.0 0.0 9108 876 pts/0 S+ 17:08 0:00 grep --color=auto nginx

grep access_log /etc/nginx/nginx.conf
access_log /srv/logs/nginx.access.log main;

1 Like

These files from PMM before 2.20 :face_with_raised_eyebrow:
Can you run another command, pls?

yum list installed | grep pmm
1 Like

That is so odd. I’m looking at the web app, and I see:
“Current version: 2.20.0 (July 26, 2021)”

Here is the yum output:

[root@8b97d47777f1 opt]# yum list installed | grep pmm
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
pmm-managed.x86_64                     2.14.0-17.2101281011.787c431.el7 @local
pmm-server.noarch                      2.14.0-21.2101281241.cb737e6.el7 @local
pmm-update.noarch                      2.14.0-51.2101281242.f607c2c.el7 @local
pmm2-client.x86_64                     2.14.0-6.el7                     @local

1 Like

OK…We found why you have old files. Now we need to understand why you have an old version of the packages :grin:
Can you run commands, pls:

docker cp pmm-server:/srv/logs/pmm-update-perform.log .

and attach pmm-update-perform.log file to the ticket.

Also, I need output from two commands:

yum repolist
cat /etc/yum.repos.d/percona-original-release.repo

Thank you!

1 Like

@nikita.beletskii

Here you go:

[root@8b97d47777f1 opt]# yum list installed | grep pmm
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
pmm-managed.x86_64                     2.14.0-17.2101281011.787c431.el7 @local
pmm-server.noarch                      2.14.0-21.2101281241.cb737e6.el7 @local
pmm-update.noarch                      2.14.0-51.2101281242.f607c2c.el7 @local
pmm2-client.x86_64                     2.14.0-6.el7                     @local

[root@8b97d47777f1 opt]# yum repolist
Loaded plugins: changelog, fastestmirror, ovl
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
Determining fastest mirrors
 * base: mirror.mojohost.com
 * epel: mirror.es.its.nyu.edu
 * extras: mirror.ilumno.com
 * updates: ftp.usf.edu
repo id                                                                   repo name                                                                                       status
!base/7/x86_64                                                            CentOS-7 - Base                                                                                 10072
!epel/x86_64                                                              Extra Packages for Enterprise Linux 7 - x86_64                                                  13514
!extras/7/x86_64                                                          CentOS-7 - Extras                                                                                 448
!nginx/x86_64                                                             nginx repo                                                                                        761
!percona-ppg-11                                                           PostgreSQL YUM repository - x86_64                                                                 98
!percona-release-noarch/7                                                 Percona Original release/noarch YUM repository                                                     63
!percona-release-x86_64/7                                                 Percona Original release/x86_64 YUM repository                                                   2256
!pmm2-server                                                              PMM Server YUM repository - x86_64                                                                191
!prel-release-noarch/7                                                    Percona Release release/noarch YUM repository                                                       3
!updates/7/x86_64                                                         CentOS-7 - Updates                                                                               1216
repolist: 28622
[root@8b97d47777f1 opt]# cat /etc/yum.repos.d/percona-original-release.repo
#
# This repo is managed by "percona-release" utility, do not edit!
#
[percona-release-x86_64]
name = Percona Original release/x86_64 YUM repository
baseurl = http://repo.percona.com/percona/yum/release/$releasever/RPMS/x86_64
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/PERCONA-PACKAGING-KEY

[percona-release-noarch]
name = Percona Original release/noarch YUM repository
baseurl = http://repo.percona.com/percona/yum/release/$releasever/RPMS/noarch
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/PERCONA-PACKAGING-KEY

[percona-release-sources]
name = Percona Original release/sources YUM repository
baseurl = http://repo.percona.com/percona/yum/release/$releasever/SRPMS
enabled = 0
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/PERCONA-PACKAGING-KEY


Let me know if you need that at the OS level.

I’ve attached the pmm-update-perform.log file.
pmm-update-perform.log (18.9 KB)

1 Like

Sorry, I missed notification from the forum :frowning:

As I see from your update log you should have 2.20 version:

Updated:
  pmm-update.noarch 0:2.20.0-60.2107271101.d72f0db.el7

Beside, according to the logs, you updated from 2.19 to 2.20:

---> Package pmm-update.noarch 0:2.19.0-59.2106301054.58b0b89.el7 will be updated
---> Package pmm-update.noarch 0:2.20.0-60.2107271101.d72f0db.el7 will be an update

This string also looks suspicious because we have yum-cron in our image.
Can you run also these commands, pls:

pmm-admin --version
yum clean all
yum upgrade --assumeno
1 Like

I appreciate your attention to this issue. Here are the results:

pmm-admin --version

ProjectName: pmm-admin
Version: 2.14.0
PMMVersion: 2.14.0
Timestamp: 2021-01-28 12:36:42 (UTC)
FullCommit: cdf593c6774c3c239aaf22fff27e9e7aa1364a60

yum clean all

Loaded plugins: changelog, fastestmirror, ovl
Repodata is over 2 weeks old. Install yum-cron? Or run: yum makecache fast
Cleaning repos: base epel extras nginx percona-ppg-11 percona-release-noarch percona-release-x86_64 pmm2-server
              : prel-release-noarch updates
Cleaning up list of fastest mirrors
Other repos take up 5.3 M of disk space (use --verbose for details)

yum upgrade --assumeno

=====================================================================================================================
 Package                          Arch       Version                                Repository                  Size
=====================================================================================================================
Updating:
 ansible                          noarch     2.9.24-2.el7                           epel                        17 M
 bind-license                     noarch     32:9.11.4-26.P2.el7_9.5                updates                     91 k
 dbaas-controller                 x86_64     2.20.0-1.2106281154.316d3d2.el7        pmm2-server                4.3 M
 device-mapper                    x86_64     7:1.02.170-6.el7_9.5                   updates                    297 k
 device-mapper-libs               x86_64     7:1.02.170-6.el7_9.5                   updates                    325 k
 glib2                            x86_64     2.56.1-9.el7_9                         updates                    2.5 M
 glibc                            x86_64     2.17-324.el7_9                         updates                    3.6 M
 glibc-common                     x86_64     2.17-324.el7_9                         updates                     12 M
 libblkid                         x86_64     2.23.2-65.el7_9.1                      updates                    183 k
 libmount                         x86_64     2.23.2-65.el7_9.1                      updates                    185 k
 libsmartcols                     x86_64     2.23.2-65.el7_9.1                      updates                    143 k
 libuuid                          x86_64     2.23.2-65.el7_9.1                      updates                     84 k
 nginx                            x86_64     1:1.21.1-1.el7.ngx                     nginx                      790 k
 nss                              x86_64     3.53.1-7.el7_9                         updates                    869 k
 nss-sysinit                      x86_64     3.53.1-7.el7_9                         updates                     66 k
 nss-tools                        x86_64     3.53.1-7.el7_9                         updates                    535 k
 openldap                         x86_64     2.4.44-23.el7_9                        updates                    356 k
 percona-alertmanager             x86_64     0.21.0-3.el7                           pmm2-server                 12 M
 percona-dashboards               noarch     2.20.0-15.2107131605.75d67a4.el7       pmm2-server                 32 M
 percona-grafana                  x86_64     7.5.7-93.2107220953.908ff0c.el7        pmm2-server                 41 M
 percona-postgresql11             x86_64     1:11.12-2.el7                          percona-ppg-11             1.7 M
 percona-postgresql11-contrib     x86_64     1:11.12-2.el7                          percona-ppg-11             615 k
 percona-postgresql11-libs        x86_64     1:11.12-2.el7                          percona-ppg-11             368 k
 percona-postgresql11-server      x86_64     1:11.12-2.el7                          percona-ppg-11             4.8 M
 percona-qan-api2                 x86_64     2.20.0-15.2107120744.af14526.el7       pmm2-server                4.4 M
 percona-release                  noarch     1.0-27                                 percona-release-noarch      20 k
 percona-victoriametrics          x86_64     1.60.0-1.el7                           pmm2-server                7.9 M
 perl                             x86_64     4:5.16.3-299.el7_9                     updates                    8.0 M
 perl-Pod-Escapes                 noarch     1:1.04-299.el7_9                       updates                     52 k
 perl-libs                        x86_64     4:5.16.3-299.el7_9                     updates                    690 k
 perl-macros                      x86_64     4:5.16.3-299.el7_9                     updates                     44 k
 pmm-managed                      x86_64     2.20.0-17.2107220951.3c660c3.el7       pmm2-server                 16 M
 pmm-server                       noarch     2.20.0-23.2107271100.e68715f.el7       pmm2-server                2.0 M
 pmm-update                       noarch     2.20.0-60.2107271101.d72f0db.el7       pmm2-server                832 k
 pmm2-client                      x86_64     2.20.0-6.el7                           percona-release-x86_64      43 M
 screen                           x86_64     4.1.0-0.27.20120314git3c2946.el7_9     updates                    553 k
 systemd                          x86_64     219-78.el7_9.3                         updates                    5.1 M
 systemd-libs                     x86_64     219-78.el7_9.3                         updates                    418 k
 systemd-sysv                     x86_64     219-78.el7_9.3                         updates                     97 k
 unzip                            x86_64     6.0-22.el7_9                           updates                    171 k
 util-linux                       x86_64     2.23.2-65.el7_9.1                      updates                    2.0 M
 zlib                             x86_64     1.2.7-19.el7_9                         updates                     90 k

Transaction Summary
=====================================================================================================================
Upgrade  42 Packages

This is a little confusing. It appears that some things thinks it has been upgraded to the latest 2.20, however some do not.

1 Like

OK, let’s try to update PMM.
Could you do a backup before update:

It’ll be a big update and backup can be useful.

yum upgrade -y pmm-update

And after command will be finished:

/usr/sbin/pmm-update -perform -playbook=/usr/share/pmm-update/ansible/playbook/tasks/update.yml
1 Like

The yum upgrade completed without issue:

Running transaction
  Updating   : pmm-update-2.20.0-60.2107271101.d72f0db.el7.noarch                                                                                                     1/2
  Cleanup    : pmm-update-2.14.0-51.2101281242.f607c2c.el7.noarch                                                                                                     2/2
  Verifying  : pmm-update-2.20.0-60.2107271101.d72f0db.el7.noarch                                                                                                     1/2
  Verifying  : pmm-update-2.14.0-51.2101281242.f607c2c.el7.noarch                                                                                                     2/2

Updated:
  pmm-update.noarch 0:2.20.0-60.2107271101.d72f0db.el7

Complete!

However the ansible script failed:

INFO[0001] pmm-update:
before update = {Version:2.20.0 FullVersion:2.20.0-60.2107271101.d72f0db.el7 BuildTime:2021-07-27 11:01:17 +0000 UTC Repo:pmm2-server}
 after update = {Version:2.20.0 FullVersion:2.20.0-60.2107271101.d72f0db.el7 BuildTime:2021-07-27 11:01:17 +0000 UTC Repo:pmm2-server}
INFO[0001] Version did not change.
Starting "ansible-playbook --flush-cache /usr/share/pmm-update/ansible/playbook/tasks/update.yml" ...
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'

PLAY [localhost] ***************************************************************

TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [detect /srv/pmm-distribution] ********************************************
ok: [localhost]

TASK [detect containers] *******************************************************
ok: [localhost]

TASK [set dbaas toggle] ********************************************************
ok: [localhost]

TASK [force container] *********************************************************
skipping: [localhost]

TASK [Configure systemd] *******************************************************
skipping: [localhost]

TASK [Remove old supervisord service confiuration] *****************************
skipping: [localhost]

TASK [Increase number of open files for jobs] **********************************
skipping: [localhost]

TASK [Reread supervisord configuration] ****************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["supervisorctl", "reread"], "delta": "0:00:00.732999", "end": "2021-08-24 18:12:43.944505", "msg": "non-zero retu
rn code", "rc": 2, "start": "2021-08-24 18:12:43.211506", "stderr": "", "stderr_lines": [], "stdout": "error: <class 'socket.error'>, [Errno 2] No such file or directory:
 file: /usr/lib64/python2.7/socket.py line: 224", "stdout_lines": ["error: <class 'socket.error'>, [Errno 2] No such file or directory: file: /usr/lib64/python2.7/socket.
py line: 224"]}

PLAY RECAP *********************************************************************
localhost                  : ok=4    changed=0    unreachable=0    failed=1    skipped=4    rescued=0    ignored=0

FATA[0008] RunPlaybook failed: exit status 2

1 Like

Did you change files in the container before this problem?

Most likely these error means that you don’t have supervisor socket in a container and it’s super weird because a supervisor is PID 1 in a container (it should be run always).

We tested updates between any versions on each release and I didn’t see problems like this…

Anyway, can you restart the container with PMM?

docker restart pmm-server

where pmm-server is container name and run Ansible command again:

/usr/sbin/pmm-update -perform -playbook=/usr/share/pmm-update/ansible/playbook/tasks/update.yml
1 Like

Nope, I’ve never changed any files manually.

The restart and rerun of the pmm-update command fixed the update.

Now I guess I’ll sit back and wait the ## hours to see if the logs rotate? Or, is there anything else you want me to check?

Thanks!

1 Like