PMM agent version mismatch

Percona Monitoring and Management (PMM)
1

This is the output of pmm-agent -v

ProjectName: pmm-agent
Version: 2.26.0
PMMVersion: 2.26.0
Timestamp: 2022-02-03 14:03:52 (UTC)
FullCommit: c74f5ae58331dc29af34d29c2b6905b8f47cc3ab

whereas when I add mysql with this command

sudo pmm-admin add mysql --trace --json --username=stats --socket=$(...) --replication-set=repl_set --query-source=perfschema

among other stuff I get this output

{"component":"agentlocal-transport","file":"/tmp/go/pkg/mod/github.com/go-openapi/runtime@v0.19.20/client/runtime.go:446","func":"github.com/go-openapi/runtime/client.(*Runtime).Submit","level":"debug","msg":"HTTP/1.1 200 OK\r\nContent-Length: 1171\r\nContent-Type: application/json\r\nDate: Tue, 31 May 2022 03:53:49 GMT\r\nGrpc-Metadata-Content-Type: application/grpc\r\n\r\n{\n  \"agent_id\": \"/agent_id/8f33ef6f-1f6e-473c-8e6d-282735217f98\",\n  \"runs_on_node_id\": \"/node_id/20fb4f3c-6157-4a16-b276-874c81f424e8\",\n  \"server_info\": {\n    \"url\": \"https://admin:admin@pmm-server-common-tls.service.consul:443/\",\n    \"insecure_tls\": true,\n    \"connected\": true,\n    \"version\": \"2.26.0\",\n    \"latency\": null,\n    \"clock_drift\": null\n  },\n  \"agents_info\": [\n    {\n      \"agent_id\": \"/agent_id/475fe48d-5dc2-4562-b454-ea2e58d6249a\",\n      \"agent_type\": \"VM_AGENT\",\n      \"status\": \"RUNNING\",\n      \"listen_port\": 42000\n    },\n    {\n      \"agent_id\": \"/agent_id/4c649d28-f799-49ac-8b45-c564e4581ec2\",\n      \"agent_type\": \"NODE_EXPORTER\",\n      \"status\": \"RUNNING\",\n      \"listen_port\": 42001\n    },\n    {\n      \"agent_id\": \"/agent_id/51aef961-98f6-4bc3-9d05-6e118e8fa9d3\",\n      \"agent_type\": \"QAN_MYSQL_PERFSCHEMA_AGENT\",\n      \"status\": \"RUNNING\",\n      \"listen_port\": 0\n    },\n    {\n      \"agent_id\": \"/agent_id/e0b540e6-5d4d-4427-8e0d-bc861551f3dd\",\n      \"agent_type\": \"MYSQLD_EXPORTER\",\n      \"status\": \"RUNNING\",\n      \"listen_port\": 42002\n    }\n  ],\n  \"config_filepath\": \"/usr/local/percona/pmm2/config/pmm-agent.yaml\",\n  \"agent_version\": \"2.17.0\"\n}\n","time":"2022-05-31T03:53:49Z"}

on the last line it indicates that the agent version is actually 2.17.0.
Can someone explain where this discrepancy is coming from? I am trying to upgrade the pmm-agent to above 2.20.0 to make the Explain/Table tabs work in QAN, because at the pmm-agent version of 2.17.0 the output of the REST calls to retrieve the query explain information is different than what the dashboard expects due to this change [PMM-8059] Update Query Analytics UI to clarify estimated results on MySQL 'explain' response where we modified original query - Percona JIRA and its related backend change [PMM-6278] Modification of MySQL 'Delete' queries to provide 'Explain' information - Percona JIRA

Thanks!

2

Hello, this seems like a bug with the reported version. How are you installing the pmm2-client? rpm? You should try to always match the same version between client and server.

3

Hi.
Which one is the reported version in this case?
I am installing by downloading manually then using the package manager, Ubuntu 20.04.
I am trying to upgrade my client to match the server, which is 2.26. I thought I did that, but it doesn’t seem so.

4

My understanding is that by upgrading the client past 2.20, this change would be included, so that the JSON response from the Explain API-s would be in the new format, because currently it is using the old format which the frontend code fails to handle.

5

can you post here the exact steps you took to upgrade? at the risk of saying something obvious did you restart pmm-agent service after the upgrade?

6

We are using an internal deployment tool to install PMM, I am not familiar with the details but I know that it is using dpkg at some point.
I did restart the service by running
pmm-admin remove mysql service_name
Then running
pmm-admin config...
then
pmm-admin add mysql ...
So which version is meant to be the correct one? The one coming from pmm-agent -v or the one in the JSON output when I ran pmm-admin add?

7

What I mean is try to restart the pmm agent systemd service:

systemctl pmm-agent restart

it is possible you installed the new binaries but the old ones are still in-memory, so there is a mismatch? try the above and let me know

8

That worked! Thank you!