Not the answer you need?
Register and ask your own question!

"No data" on PXC/Galera Cluster Summary

milosbmilosb ContributorCurrent User Role Contributor
After updating the PMM server docker instance from 2.1.0->2.2.0 I noticed that all graphs in this dashboard no longer show any data.

I have also updated the clients on all of the systems monitored. Interestingly enough, before the update the dashboard "MySQL Command/Handler counters compare"
reported "No Data" on all graphs except for one host. This was auto-resolved after this update.

However, everything else so far seems to be fine.

Any clues to what might be going on here?

Comments

  • roma.novikovroma.novikov Percona Percona Staff Role
    Hi milosb . Thanks for staying on top of our new version and reporting problems!
    Have you used some Labels (cluster, environment) for your installation? if yes - can you provide them?

    Also pls check https://PMM_SERVER/prometheus/targets for Errors
  • milosbmilosb Contributor Current User Role Contributor
    Hi roma.novikov No, not that I'm aware of. Please remind me where I've seen the option to add Labels, just so I know we're on the same page.

    As for the Prometheus targets, everything is fine there - that's the first thing I check. No unhealthy services.

    I've moved this environment from PMMv1 over to PMMv2 back in October and it worked well - except for the "No Data" dashboards,... earlier with the MySQL Handler/command counters, now this.
  • roma.novikovroma.novikov Percona Percona Staff Role
    milosb you can see your labels on https://PMM_SERVER/graph/d/pmm-inventory/pmm-inventory "Services" tab
  • milosbmilosb Contributor Current User Role Contributor
    roma.novikov There's just a table in there with all the nodes monitored,... with columns: ID, Service Type, Service Name, Node ID, Addresses, Port, Other details

    Service name is in the form ${HOSTNAME}-mysql

    Is there something else I should be lookin at?
  • milosbmilosb Contributor Current User Role Contributor
    The awkward thing is: Before the update to v2.2.0 (and even while I was on 2.0.1) there was "No Data" in the MySQL Handlers/Command Counters shown... but after the update, they're all of the sudden there - even for the previous months. So, the data was collected, but for some reason not shown.

    I suspect it's the same situation now, but am unable to figure out where the problem lies. The Grafana log shows only this, related to alert graph images not attached when alerts are sent (this also used to work:

    lvl=eror msg="Failed to upload alert panel image." logger=alerting.notifier error="Timeout error. You can set timeout in seconds with &timeout url parameter
  • milosbmilosb Contributor Current User Role Contributor
    I'll also add that the PXC/Galera Node Summary and PXC/Galera Nodes Compare display data just fine for all cluster nodes.

    The drop-down menu in the PXC/Galera Cluster Summary shows no cluster... Only "None" to choose from.
  • milosbmilosb Contributor Current User Role Contributor
    roma.novikov Here's some development of this issue: If I open the PXC/Galera Node summary dashboard, the graphs display fine both initially and after
    a refresh interval, while the "Cluster" drop-down menu has a cluster name to choose from, as it was set in the 'wsrep_cluster_name".
    Switching from this, to the PXC/Galera Cluster Summary initially loads the graphs fine, but after the very first refresh the graphs are gone, with "No data" filling the place of the
    curves... and there's no more value to choose from in the "Cluster" drop-down menu.
    Hitting the Cluster menu before the refresh, again results in "No data" in the graphs, and no options to choose from.

    Entering the cluster name manually in the drop-down menu brings the graphs back up again, but again *only* until the first dashboard refresh. Looks like a bug...

    Hope this helps.
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.