Failed to register pmm-agent on PMM Server: Internal server error

Percona Monitoring and Management (PMM) PMM 2.x
1

Description:

[Detailed description of the issue or question]

os:redhat 7.9
install file : pmm2-client-2.32.0.tar.gz

[postgres@cjDfarmdb pmm2-client-2.32.0]$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/pg_data/.local/bin:/pg_data/bin:/usr/pgsql-14/bin:/pg_dump/pmm/pmm2-client-2.32.0/bin
[postgres@cjDfarmdb pmm2-client-2.32.0]$
[postgres@cjDfarmdb pmm2-client-2.32.0]$
[postgres@cjDfarmdb pmm2-client-2.32.0]$
[postgres@cjDfarmdb pmm2-client-2.32.0]$
[postgres@cjDfarmdb pmm2-client-2.32.0]$ pmm-agent setup --config-file=${PMM_DIR}/config/pmm-agent.yaml --server-address=10.65.10.40 --server-insecure-tls --server-username=admin --server-password=****** --paths-tempdir=${PMM_DIR}/tmp --paths-base=${PMM_DIR}
INFO[2024-02-05T10:52:18.765+09:00] Loading configuration file /pg_dump/pmm/pmm2-client-2.32.0/config/pmm-agent.yaml. component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/node_exporter component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/mysqld_exporter component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/mongodb_exporter component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/postgres_exporter component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/proxysql_exporter component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/rds_exporter component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/azure_exporter component=setup
INFO[2024-02-05T10:52:18.765+09:00] Using /pg_dump/pmm/pmm2-client-2.32.0/exporters/vmagent component=setup
INFO[2024-02-05T10:52:18.765+09:00] Updating PMM Server address from “10.65.10.40” to “10.65.10.40:443”. component=setup
Checking local pmm-agent status…
pmm-agent is not running.
Registering pmm-agent on PMM Server…
Failed to register pmm-agent on PMM Server: Internal server error…

Steps to Reproduce:

[Step-by-step instructions on how to reproduce the issue, including any specific settings or configurations]

Version:

[Insert the version number of the software]

Logs:

[If applicable, include any relevant log files or error messages]

Expected Result:

[What the user expected to see or happen before the issue occurred]

Actual Result:

[What actually happened when the user encountered the issue]

Additional Information:

[Include any additional information that could be helpful to diagnose the issue, such as browser or device information]

2

Hi @anonymous_lee,

could you share PMM Server version? Also could you check logs on PMM server side, more specifically pmm-managed logs. You can collect them by calling PMM_SERVER_URL/logs.zip endpoint

3

pmm-managed.log (184.7 KB)

I tested it with pmm 2.16.0, and I also tested it as a root user.
But the result was the same

4

There are nothing in logs related to this.
Could you collect logs right after the operation?
PMM designed the way that PMM Server(2.16.0) might not support newer version of PMM Client(2.32.0).
PMM 2.16.0 is quite outdated, we recommend to update PMM Server version.

5

pmm-managed.log (185.0 KB)
Currently, the pmm server is 2.32.0 and the client is also testing 2.32.0.
However, there is no 2.32.0 client installation manual, so when I install 2.32.0 according to the 2.41.0 client installation manual, the error below appears.

I tested again and uploaded pmm-managed.log

6

looks like your DB scheme was upgraded to version newer then 2.32.0, have you upgraded and then downgraded your PMM Server? Because of new constraints in PMM Server DB introduced in newer version of PMM it can’t write new record to DB.

There are 2 solutions:

  1. Upgrade your PMM Server, so it can handle new constraints correctly
  2. Downgrade DB scheme using reverse engineering