Galera Replication Latency - No data

Hello! After I upgraded PMM2 to version 2.29.1 stopped collecting information about MySQL/PXC/Galera Cluster Summary/Galera Replication Latency (No data appears) and Maximum Galera Replication Latency. All other metrics appear correctly. Does anyone know what this is caused and how to fix it? Maybe I can manually collect data about it from the server? Upgrading to 2.30 didn’t correct this problem, agents are also updated.

1 Like

Hi @Silver3 welcome back to the Percona forums!

This is certainly unexpected. Can you check the logs of pmm-agent to see whether there are any errors being reported?

In the meantime I’ll spin up some PXC instances to see if I can reproduce

2 Likes

Thanks for the answer! Unfortunately, I don’t see any errors in pmm-agent log on db servers. Example:

Sep 15 16:11:00 mysql pmm-agent[1017]: #033[36mINFO#033[0m[2022-09-15T16:11:00.045+01:00] Sending 31 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:11:00 mysql pmm-agent: #033[36mINFO#033[0m[2022-09-15T16:11:00.045+01:00] Sending 31 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:12:00 mysql pmm-agent[1017]: #033[36mINFO#033[0m[2022-09-15T16:12:00.045+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:12:00 mysql pmm-agent: #033[36mINFO#033[0m[2022-09-15T16:12:00.045+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:13:00 mysql pmm-agent[1017]: #033[36mINFO#033[0m[2022-09-15T16:13:00.045+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:13:00 mysql pmm-agent: #033[36mINFO#033[0m[2022-09-15T16:13:00.045+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:14:00 mysql pmm-agent[1017]: #033[36mINFO#033[0m[2022-09-15T16:14:00.048+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:14:00 mysql pmm-agent: #033[36mINFO#033[0m[2022-09-15T16:14:00.048+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:15:00 mysql pmm-agent[1017]: #033[36mINFO#033[0m[2022-09-15T16:15:00.043+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:15:00 mysql pmm-agent: #033[36mINFO#033[0m[2022-09-15T16:15:00.043+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:16:00 mysql pmm-agent[1017]: #033[36mINFO#033[0m[2022-09-15T16:16:00.047+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:16:00 mysql pmm-agent: #033[36mINFO#033[0m[2022-09-15T16:16:00.047+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:17:00 mysql pmm-agent[1017]: #033[36mINFO#033[0m[2022-09-15T16:17:00.053+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent
Sep 15 16:17:00 mysql pmm-agent: #033[36mINFO#033[0m[2022-09-15T16:17:00.053+01:00] Sending 27 buckets. #033[36magentID#033[0m=/agent_id/9aea332e-1464-409a-9aa7-d199212a5bf9 #033[36mcomponent#033[0m=agent-builtin #033[36mtype#033[0m=qan_mysql_perfschema_agent

1 Like

I see same problem on on the demo page. Installing the client version 2.28 fixes error.
Grafana

1 Like