Upgrade Percona PMM 2.41.x issues (Appliance)

Dear all,
from 2.41.0 we have issues using the upgrade functionalities (2.41.1 e 2.41.2)

The upgrade script loops, with this error:

Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Wed 20 Mar 2024 06:07:51 PM CET
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

time=“2024-03-20T18:17:42+01:00” level=info msg=“pmm-update:\nbefore update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}\n after update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}”
time=“2024-03-20T18:17:42+01:00” level=info msg=“Version did not change.”
Starting “ansible-playbook --flush-cache /usr/share/pmm-update/ansible/playbook/tasks/update.yml” …
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match ‘all’
ERROR! couldn’t resolve module/action ‘ini_file’. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in ‘/usr/share/pmm-update/ansible/playbook/tasks/update.yml’: line 111, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: Disable pmm-update-perform-init
  ^ here

time=“2024-03-20T18:17:43+01:00” level=fatal msg=“RunPlaybook failed: exit status 4”

How I can fix?

Hello,
It’s super weird behavior.
could you share more details please?
Which version are you upgrading from? Is it OVF, AMI or Docker installment?
Could you share whole file, please?

Thank you.

Dear,
Currently I’m using the OVA appliance (last one provided, Rocky Linux based).

Settings and data has been migrated form the old appliance (Centos7 based) using che official guide.
Services are up and running correctly, agents are in communication.
We have this issue, the only one message is the log already provided.

Can you help me?

Is it 2.41.1?
I’m kindly asking you to provide the whole file, please? the root cause might be hidden somewhere else in the logs.

image

Which files you need?

ProjectName: pmm-update
Version: 2.41.2
PMMVersion: 2.41.2
Timestamp: 2024-03-19 21:09:27 (UTC)
FullCommit: 10aa17ad6fe680a99f253cca8b32d3beb04050f6
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Wed 20 Mar 2024 06:07:51 PM CET
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

Starting “yum update --assumeyes pmm-update” …
Last metadata expiration check: 6:41:35 ago on Fri 22 Mar 2024 06:07:55 AM CET.
Dependencies resolved.
Nothing to do.
Complete!
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Wed 20 Mar 2024 06:07:51 PM CET
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

time=“2024-03-22T12:49:31+01:00” level=info msg=“pmm-update:\nbefore update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}\n after update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}”
time=“2024-03-22T12:49:31+01:00” level=info msg=“Version did not change.”
Starting “ansible-playbook --flush-cache /usr/share/pmm-update/ansible/playbook/tasks/update.yml” …
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match ‘all’
ERROR! couldn’t resolve module/action ‘ini_file’. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in ‘/usr/share/pmm-update/ansible/playbook/tasks/update.yml’: line 111, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: Disable pmm-update-perform-init
  ^ here

time=“2024-03-22T12:49:32+01:00” level=fatal msg=“RunPlaybook failed: exit status 4”

Guys, any help?

Currently I’m unable to perform updates.


The error appears to be in ‘/usr/share/pmm-update/ansible/playbook/tasks/update.yml’: line 111, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: Disable pmm-update-perform-init
  ^ here

ProjectName: pmm-update
Version: 2.41.1
PMMVersion: 2.41.1
Timestamp: 2024-01-29 13:32:29 (UTC)
FullCommit: f4004eac5c03e97327ae235dbf40f91e31ba2208
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.1
Release : 67.2401291332.f4004ea.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.1-67.2401291332.f4004ea.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Mon 29 Jan 2024 02:32:27 PM CET
Install time : Tue 13 Feb 2024 04:49:31 PM CET
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

Starting “yum update --assumeyes pmm-update” …
Last metadata expiration check: 0:02:00 ago on Tue 09 Apr 2024 04:31:44 PM CEST.
Dependencies resolved.

Package Arch Version Repository Size

Upgrading:
pmm-update noarch 2.41.2-67.2403192109.10aa17a.el9 pmm2-server 921 k

Transaction Summary

Upgrade 1 Package

Total download size: 921 k
Downloading Packages:
pmm-update-2.41.2-67.2403192109.10aa17a.el9.noa 2.1 MB/s | 921 kB 00:00

Total 2.1 MB/s | 921 kB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Upgrading : pmm-update-2.41.2-67.2403192109.10aa17a.el9.noarch 1/2
Cleanup : pmm-update-2.41.1-67.2401291332.f4004ea.el9.noarch 2/2
Verifying : pmm-update-2.41.2-67.2403192109.10aa17a.el9.noarch 1/2
Verifying : pmm-update-2.41.1-67.2401291332.f4004ea.el9.noarch 2/2

Upgraded:
pmm-update-2.41.2-67.2403192109.10aa17a.el9.noarch

Complete!
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Tue 09 Apr 2024 04:33:45 PM CEST
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

time=“2024-04-09T16:33:46+02:00” level=info msg=“pmm-update:\nbefore update = {Version:2.41.1 FullVersion:2.41.1-67.2401291332.f4004ea.el9 BuildTime: Repo:pmm2-server}\n after update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}”
time=“2024-04-09T16:33:46+02:00” level=info msg=“Version changed, exiting.”
ProjectName: pmm-update
Version: 2.41.2
PMMVersion: 2.41.2
Timestamp: 2024-03-19 21:09:27 (UTC)
FullCommit: 10aa17ad6fe680a99f253cca8b32d3beb04050f6
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Tue 09 Apr 2024 04:33:45 PM CEST
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

Starting “yum update --assumeyes pmm-update” …
Last metadata expiration check: 0:02:06 ago on Tue 09 Apr 2024 04:31:44 PM CEST.
Dependencies resolved.
Nothing to do.
Complete!
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Tue 09 Apr 2024 04:33:45 PM CEST
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

time=“2024-04-09T16:33:51+02:00” level=info msg=“pmm-update:\nbefore update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}\n after update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}”
time=“2024-04-09T16:33:51+02:00” level=info msg=“Version did not change.”
Starting “ansible-playbook --flush-cache /usr/share/pmm-update/ansible/playbook/tasks/update.yml” …
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match ‘all’
ERROR! couldn’t resolve module/action ‘ini_file’. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in ‘/usr/share/pmm-update/ansible/playbook/tasks/update.yml’: line 111, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: Disable pmm-update-perform-init
  ^ here

time=“2024-04-09T16:33:52+02:00” level=fatal msg=“RunPlaybook failed: exit status 4”
ProjectName: pmm-update
Version: 2.41.2
PMMVersion: 2.41.2
Timestamp: 2024-03-19 21:09:27 (UTC)
FullCommit: 10aa17ad6fe680a99f253cca8b32d3beb04050f6
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Tue 09 Apr 2024 04:33:45 PM CEST
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

Starting “yum update --assumeyes pmm-update” …
Last metadata expiration check: 0:02:12 ago on Tue 09 Apr 2024 04:31:44 PM CEST.
Dependencies resolved.
Nothing to do.
Complete!
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Tue 09 Apr 2024 04:33:45 PM CEST
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

time=“2024-04-09T16:33:57+02:00” level=info msg=“pmm-update:\nbefore update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}\n after update = {Version:2.41.2 FullVersion:2.41.2-67.2403192109.10aa17a.el9 BuildTime: Repo:pmm2-server}”
time=“2024-04-09T16:33:57+02:00” level=info msg=“Version did not change.”
Starting “ansible-playbook --flush-cache /usr/share/pmm-update/ansible/playbook/tasks/update.yml” …
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match ‘all’
ERROR! couldn’t resolve module/action ‘ini_file’. This often indicates a misspelling, missing collection, or incorrect module path.

The error appears to be in ‘/usr/share/pmm-update/ansible/playbook/tasks/update.yml’: line 111, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

- name: Disable pmm-update-perform-init
  ^ here

time=“2024-04-09T16:33:57+02:00” level=fatal msg=“RunPlaybook failed: exit status 4”
ProjectName: pmm-update
Version: 2.41.2
PMMVersion: 2.41.2
Timestamp: 2024-03-19 21:09:27 (UTC)
FullCommit: 10aa17ad6fe680a99f253cca8b32d3beb04050f6
Starting “yum --verbose info installed pmm-update” …
Loaded plugins: builddep, changelog, config-manager, copr, debug, debuginfo-install, download, generate_completion_cache, groups-manager, needs-restarting, playground, repoclosure, repodiff, repograph, repomanage, reposync, system-upgrade, ulninfo
YUM version: 4.14.0
cachedir: /var/cache/dnf
Unknown configuration option: async = 1 in /etc/yum.repos.d/clickhouse.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/nginx.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-11.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/percona-ppg-14.repo
Unknown configuration option: async = 1 in /etc/yum.repos.d/pmm2-server.repo
User-Agent: constructed: ‘libdnf (Oracle Linux Server 9.3; server; Linux.x86_64)’
Installed Packages
Name : pmm-update
Version : 2.41.2
Release : 67.2403192109.10aa17a.el9
Architecture : noarch
Size : 2.4 M
Source : pmm-update-2.41.2-67.2403192109.10aa17a.el9.src.rpm
Repository : @System
From repo : pmm2-server
Packager : None
Buildtime : Tue 19 Mar 2024 10:09:25 PM CET
Install time : Tue 09 Apr 2024 04:33:45 PM CEST
Installed by : System
Summary : Tool for updating packages and OS configuration for PMM Server
URL : GitHub - percona/pmm: Percona Monitoring and Management: an open source database monitoring, observability and management tool
License : AGPLv3
Description : Tool for updating packages and OS configuration for PMM Server

Starting “yum update --assumeyes pmm-update” …