MySQL User Statistics is enabled but graphs are not displayed in Pmm MySQL User Statistics dashboard

skumar · April 10, 2020, 8:13am

Enabled the userstat in mysql and also updated in the my.cnf

SET GLOBAL userstat=1;

Here are the outputs for the pmm-admin info/list/check-network.

[root@e-mysql-01.lga1 ~]# pmm-admin info
pmm-admin 1.17.3

PMM Server | prod-perconamm-01.lga1.admarketplace.net (password-protected)
Client Name | e-mysql-01.lga1.admarketplace.net
Client Address | 10.2.0.135
Service Manager | linux-upstart

Go Version | 1.10.1
Runtime Info | linux/amd64

[root@e-mysql-01.lga1 ~]# pmm-admin list
pmm-admin 1.17.3

PMM Server | prod-perconamm-01.lga1.admarketplace.net (password-protected)
Client Name | e-mysql-01.lga1.admarketplace.net
Client Address | 10.2.0.135
Service Manager | linux-upstart

-------------- ---------------------------------- ----------- -------- -------------------------------------------- ---------------
SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS
-------------- ---------------------------------- ----------- -------- -------------------------------------------- ---------------
linux:metrics e-mysql-01.lga1.admarketplace.net 42000 NO -
mysql:metrics e-mysql-01.lga1.admarketplace.net 42002 NO pmm_app:***@unix(/var/lib/mysql/mysql.sock) tablestats=OFF
[root@e-mysql-01.lga1 ~]# pmm-admin check-network
PMM Network Status

Server Address | prod-perconamm-01.lga1.admarketplace.net
Client Address | 10.2.0.135

* System Time
NTP Server (0.pool.ntp.org) | 2020-04-09 13:43:42 +0000 UTC
PMM Server | 2020-04-09 13:43:42 +0000 GMT
PMM Client | 2020-04-09 09:43:42 -0400 EDT
PMM Server Time Drift | OK
PMM Client Time Drift | OK
PMM Client to PMM Server Time Drift | OK

* Connection: Client --&gt; Server
-------------------- -------
SERVER SERVICE STATUS
-------------------- -------
Consul API OK
Prometheus API OK
Query Analytics API DOWN

Connection duration | 1.656123ms
Request duration | -1.315941ms
Full round trip | 340.182µs


* Connection: Client &lt;-- Server
-------------- ---------------------------------- ----------------- ------- ---------- ---------
SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD
-------------- ---------------------------------- ----------------- ------- ---------- ---------
linux:metrics e-mysql-01.lga1.admarketplace.net 10.2.0.135:42000 OK - -
mysql:metrics e-mysql-01.lga1.admarketplace.net 10.2.0.135:42002 OK - -

Peter · April 10, 2020, 1:30pm

Hi,
In this case I suggest you to check

If MySQL is reporting some data for USER statistics through SHOW USER_STATISTICS
If this information is shown in MySQL Exporter status output
If this information can be seen by querying Prometheus directly

This can help to point out where specifically things go wrong
For more information see
https://www.percona.com/blog/2018/01/17/troubleshooting-percona-monitoring-and-management-pmm-metrics/

skumar · April 14, 2020, 7:47am

Hi Peter,
> I have checked the SHOW USER_STATISTICS for all the hosts and its reporting data for 5 hosts out of 6.
> Not sure how to check the MySQL exporter status output.
> I have seen the Prometheus directly, and here’s the output.

for e-mysql-01.lga1.admarketplace.net SHOW USER_STATISTICS data is not displayed.

For w-mysql-01.sfo1.admarketplace.net, w-mysql-02.sfo1.admarketplace.net and w-mysql-03.sfo1.admarketplace.net, SHOW USER_STATISTICS data is displayed but graphs are not displayed.

Once in a while we can see few host names in the drop down list and later they disappear.

Please let me know if any more information needed.

Peter · April 14, 2020, 7:00pm

These Graphs show the Prometheus Scrape statistics where you can see it fails on many hosts for “low resolution data” probably due to timeout.

To see what MySQL exporter returns you should go to the URL which you see being scraped such as https://10.4.0.118:42002/metrics-lr
See if you get output and it it has anything matching user_stat out there

skumar · April 17, 2020, 11:47am

I have opened the and searched for user_stat and I found the following

# HELP mysql_info_schema_user_statistics_access_denied_total The number of times this userâ€™s connections issued commands that were denied.

# TYPE mysql_info_schema_user_statistics_access_denied_total counter
mysql_info_schema_user_statistics_access_denied_total{user="adengine_app"} 0

Peter · April 17, 2020, 2:12pm

Is this the only match you found or is it one of many lines ?

skumar · April 17, 2020, 2:17pm

It was around 336 lines.
Similar to below line, but the only difference was the user name.

mysql_info_schema_user_statistics_access_denied_total{user="adengine_app"} 0

Peter · April 17, 2020, 2:21pm

OK. So were any other values reported

mysql_info_schema_user_statistics_access_denied_total

This corresponds to access denied cases while there should be also other data about rows read written etc which are actually plotted on dashboard

skumar · April 17, 2020, 2:25pm

No other values where reported.

Peter · April 17, 2020, 2:46pm

If this is the case you have problem on the exporter side - might be it is timing out or something - check error logs,
You need to have other values reported such as

``` # HELP mysql_info_schema_user_statistics_bytes_sent_total The number of bytes sent to this user’s connections. # TYPE mysql_info_schema_user_statistics_bytes_sent_total counter mysql_info_schema_user_statistics_bytes_sent_total{user="mysql.session"} 0 mysql_info_schema_user_statistics_bytes_sent_total{user="pmm"} 1.5773185981e+10 mysql_info_schema_user_statistics_bytes_sent_total{user="root"} 328 mysql_info_schema_user_statistics_bytes_sent_total{user="sbtest"} 79362 ```

skumar · April 17, 2020, 2:49pm

Heres the over all search list.

# HELP mysql_info_schema_user_statistics_access_denied_total The number of times this userâ€™s connections issued commands that were denied.
# TYPE mysql_info_schema_user_statistics_access_denied_total counter
mysql_info_schema_user_statistics_access_denied_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="cte_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="pgurung"} 0
mysql_info_schema_user_statistics_access_denied_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_access_denied_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_access_denied_total{user="tungsten"} 0
mysql_info_schema_user_statistics_access_denied_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_access_denied_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_binlog_bytes_written_total The number of bytes written to the binary log from this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_binlog_bytes_written_total counter
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="cte_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="pgurung"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="pmm_app"} 334
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="tungsten"} 1480
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_binlog_bytes_written_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_busy_seconds_total The cumulative number of seconds there was activity on connections from this user.
# TYPE mysql_info_schema_user_statistics_busy_seconds_total counter
mysql_info_schema_user_statistics_busy_seconds_total{user="adengine_app"} 39570
mysql_info_schema_user_statistics_busy_seconds_total{user="adland_svc_app"} 1496
mysql_info_schema_user_statistics_busy_seconds_total{user="ampx_indexer_app"} 64543
mysql_info_schema_user_statistics_busy_seconds_total{user="cte_app"} 165
mysql_info_schema_user_statistics_busy_seconds_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_busy_seconds_total{user="pgurung"} 0
mysql_info_schema_user_statistics_busy_seconds_total{user="pmm_app"} 40562
mysql_info_schema_user_statistics_busy_seconds_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_busy_seconds_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_busy_seconds_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_busy_seconds_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_busy_seconds_total{user="tungsten"} 32015
mysql_info_schema_user_statistics_busy_seconds_total{user="ui_dataggdae_app"} 55
mysql_info_schema_user_statistics_busy_seconds_total{user="vporalla_dbo"} 1
# HELP mysql_info_schema_user_statistics_bytes_received_total The number of bytes received from this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_bytes_received_total counter
mysql_info_schema_user_statistics_bytes_received_total{user="adengine_app"} 5.4027519e+07
mysql_info_schema_user_statistics_bytes_received_total{user="adland_svc_app"} 1.66139308e+08
mysql_info_schema_user_statistics_bytes_received_total{user="ampx_indexer_app"} 9.594585e+06
mysql_info_schema_user_statistics_bytes_received_total{user="cte_app"} 1.661243e+06
mysql_info_schema_user_statistics_bytes_received_total{user="log_rotation_app"} 6461
mysql_info_schema_user_statistics_bytes_received_total{user="pgurung"} 1135
mysql_info_schema_user_statistics_bytes_received_total{user="pmm_app"} 2.228222178e+09
mysql_info_schema_user_statistics_bytes_received_total{user="pub_revrpt_app"} 473204
mysql_info_schema_user_statistics_bytes_received_total{user="rmukkama"} 1455
mysql_info_schema_user_statistics_bytes_received_total{user="rpt_gen_app"} 7840
mysql_info_schema_user_statistics_bytes_received_total{user="skumar_dbo"} 80
mysql_info_schema_user_statistics_bytes_received_total{user="tungsten"} 5.5504849201e+10
mysql_info_schema_user_statistics_bytes_received_total{user="ui_dataggdae_app"} 2.7989692e+07
mysql_info_schema_user_statistics_bytes_received_total{user="vporalla_dbo"} 5.8900328e+07
# HELP mysql_info_schema_user_statistics_bytes_sent_total The number of bytes sent to this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_bytes_sent_total counter
mysql_info_schema_user_statistics_bytes_sent_total{user="adengine_app"} 2.54002127728e+11
mysql_info_schema_user_statistics_bytes_sent_total{user="adland_svc_app"} 5.695864832e+09
mysql_info_schema_user_statistics_bytes_sent_total{user="ampx_indexer_app"} 2.00110194688e+11
mysql_info_schema_user_statistics_bytes_sent_total{user="cte_app"} 4.78543872e+09
mysql_info_schema_user_statistics_bytes_sent_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_bytes_sent_total{user="pgurung"} 0
mysql_info_schema_user_statistics_bytes_sent_total{user="pmm_app"} 4.6605893632e+10
mysql_info_schema_user_statistics_bytes_sent_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_bytes_sent_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_bytes_sent_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_bytes_sent_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_bytes_sent_total{user="tungsten"} 4.741496832e+09
mysql_info_schema_user_statistics_bytes_sent_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_bytes_sent_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_commit_transactions_total The number of COMMIT commands issued by this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_commit_transactions_total counter
mysql_info_schema_user_statistics_commit_transactions_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="cte_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="pgurung"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="tungsten"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_commit_transactions_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_concurrent_connections The number of concurrent connections for this user.
# TYPE mysql_info_schema_user_statistics_concurrent_connections gauge
mysql_info_schema_user_statistics_concurrent_connections{user="adengine_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="cte_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="pgurung"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="pmm_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="rmukkama"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="tungsten"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_concurrent_connections{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_connected_time_seconds_total The cumulative number of seconds elapsed while there were connections from this user.
# TYPE mysql_info_schema_user_statistics_connected_time_seconds_total counter
mysql_info_schema_user_statistics_connected_time_seconds_total{user="adengine_app"} 5.369424e+06
mysql_info_schema_user_statistics_connected_time_seconds_total{user="adland_svc_app"} 4.3145767e+07
mysql_info_schema_user_statistics_connected_time_seconds_total{user="ampx_indexer_app"} 67907
mysql_info_schema_user_statistics_connected_time_seconds_total{user="cte_app"} 4.4562e+06
mysql_info_schema_user_statistics_connected_time_seconds_total{user="log_rotation_app"} 2
mysql_info_schema_user_statistics_connected_time_seconds_total{user="pgurung"} 1
mysql_info_schema_user_statistics_connected_time_seconds_total{user="pmm_app"} 3.293349e+06
mysql_info_schema_user_statistics_connected_time_seconds_total{user="pub_revrpt_app"} 2887
mysql_info_schema_user_statistics_connected_time_seconds_total{user="rmukkama"} 1
mysql_info_schema_user_statistics_connected_time_seconds_total{user="rpt_gen_app"} 13
mysql_info_schema_user_statistics_connected_time_seconds_total{user="skumar_dbo"} 2
mysql_info_schema_user_statistics_connected_time_seconds_total{user="tungsten"} 2.418238e+06
mysql_info_schema_user_statistics_connected_time_seconds_total{user="ui_dataggdae_app"} 136
mysql_info_schema_user_statistics_connected_time_seconds_total{user="vporalla_dbo"} 292
# HELP mysql_info_schema_user_statistics_cpu_time_seconds_total The cumulative CPU time elapsed, in seconds, while servicing this user's connections.
# TYPE mysql_info_schema_user_statistics_cpu_time_seconds_total counter
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="adengine_app"} 39066
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="adland_svc_app"} 1485
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="ampx_indexer_app"} 35697
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="cte_app"} 120
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="pgurung"} 0
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="pmm_app"} 39963
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="tungsten"} 9124
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="ui_dataggdae_app"} 54
mysql_info_schema_user_statistics_cpu_time_seconds_total{user="vporalla_dbo"} 1
# HELP mysql_info_schema_user_statistics_denied_connections_total The number of connections denied to this user.
# TYPE mysql_info_schema_user_statistics_denied_connections_total counter
mysql_info_schema_user_statistics_denied_connections_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="cte_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="pgurung"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="tungsten"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_denied_connections_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_empty_queries_total The number of times this userâ€™s connections sent empty queries to the server.
# TYPE mysql_info_schema_user_statistics_empty_queries_total counter
mysql_info_schema_user_statistics_empty_queries_total{user="adengine_app"} 35896
mysql_info_schema_user_statistics_empty_queries_total{user="adland_svc_app"} 383001
mysql_info_schema_user_statistics_empty_queries_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="cte_app"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="pgurung"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="pmm_app"} 596908
mysql_info_schema_user_statistics_empty_queries_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="tungsten"} 224958
mysql_info_schema_user_statistics_empty_queries_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_empty_queries_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_lost_connections_total The number of this userâ€™s connections that were terminated uncleanly.
# TYPE mysql_info_schema_user_statistics_lost_connections_total counter
mysql_info_schema_user_statistics_lost_connections_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="cte_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="pgurung"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="tungsten"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_lost_connections_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_other_commands_total The number of other commands executed from this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_other_commands_total counter
mysql_info_schema_user_statistics_other_commands_total{user="adengine_app"} 169
mysql_info_schema_user_statistics_other_commands_total{user="adland_svc_app"} 400264
mysql_info_schema_user_statistics_other_commands_total{user="ampx_indexer_app"} 48624
mysql_info_schema_user_statistics_other_commands_total{user="cte_app"} 24
mysql_info_schema_user_statistics_other_commands_total{user="log_rotation_app"} 117
mysql_info_schema_user_statistics_other_commands_total{user="pgurung"} 7
mysql_info_schema_user_statistics_other_commands_total{user="pmm_app"} 3.008159e+06
mysql_info_schema_user_statistics_other_commands_total{user="pub_revrpt_app"} 2148
mysql_info_schema_user_statistics_other_commands_total{user="rmukkama"} 10
mysql_info_schema_user_statistics_other_commands_total{user="rpt_gen_app"} 63
mysql_info_schema_user_statistics_other_commands_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_other_commands_total{user="tungsten"} 1.1212195e+07
mysql_info_schema_user_statistics_other_commands_total{user="ui_dataggdae_app"} 137628
mysql_info_schema_user_statistics_other_commands_total{user="vporalla_dbo"} 10
# HELP mysql_info_schema_user_statistics_rollback_transactions_total The number of ROLLBACK commands issued by this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_rollback_transactions_total counter
mysql_info_schema_user_statistics_rollback_transactions_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="cte_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="pgurung"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="tungsten"} 11
mysql_info_schema_user_statistics_rollback_transactions_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_rollback_transactions_total{user="vporalla_dbo"} 0
# HELP mysql_info_schema_user_statistics_rows_fetched_total The number of rows fetched by this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_rows_fetched_total counter
mysql_info_schema_user_statistics_rows_fetched_total{user="adengine_app"} 6.881499121e+09
mysql_info_schema_user_statistics_rows_fetched_total{user="adland_svc_app"} 1.63657489e+08
mysql_info_schema_user_statistics_rows_fetched_total{user="ampx_indexer_app"} 3.069147248e+09
mysql_info_schema_user_statistics_rows_fetched_total{user="cte_app"} 1.30021221e+08
mysql_info_schema_user_statistics_rows_fetched_total{user="log_rotation_app"} 65
mysql_info_schema_user_statistics_rows_fetched_total{user="pgurung"} 1
mysql_info_schema_user_statistics_rows_fetched_total{user="pmm_app"} 9.72696376e+08
mysql_info_schema_user_statistics_rows_fetched_total{user="pub_revrpt_app"} 2143
mysql_info_schema_user_statistics_rows_fetched_total{user="rmukkama"} 59
mysql_info_schema_user_statistics_rows_fetched_total{user="rpt_gen_app"} 14
mysql_info_schema_user_statistics_rows_fetched_total{user="skumar_dbo"} 14
mysql_info_schema_user_statistics_rows_fetched_total{user="tungsten"} 3.5778861e+07
mysql_info_schema_user_statistics_rows_fetched_total{user="ui_dataggdae_app"} 42713
mysql_info_schema_user_statistics_rows_fetched_total{user="vporalla_dbo"} 20
# HELP mysql_info_schema_user_statistics_rows_updated_total The number of rows updated by this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_rows_updated_total counter
mysql_info_schema_user_statistics_rows_updated_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="cte_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="pgurung"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="tungsten"} 4.8366265e+07
mysql_info_schema_user_statistics_rows_updated_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_rows_updated_total{user="vporalla_dbo"} 2338
# HELP mysql_info_schema_user_statistics_select_commands_total The number of SELECT commands executed from this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_select_commands_total counter
mysql_info_schema_user_statistics_select_commands_total{user="adengine_app"} 326147
mysql_info_schema_user_statistics_select_commands_total{user="adland_svc_app"} 468182
mysql_info_schema_user_statistics_select_commands_total{user="ampx_indexer_app"} 12156
mysql_info_schema_user_statistics_select_commands_total{user="cte_app"} 23030
mysql_info_schema_user_statistics_select_commands_total{user="log_rotation_app"} 65
mysql_info_schema_user_statistics_select_commands_total{user="pgurung"} 1
mysql_info_schema_user_statistics_select_commands_total{user="pmm_app"} 9.608409e+06
mysql_info_schema_user_statistics_select_commands_total{user="pub_revrpt_app"} 460
mysql_info_schema_user_statistics_select_commands_total{user="rmukkama"} 3
mysql_info_schema_user_statistics_select_commands_total{user="rpt_gen_app"} 14
mysql_info_schema_user_statistics_select_commands_total{user="skumar_dbo"} 3
mysql_info_schema_user_statistics_select_commands_total{user="tungsten"} 8.177789e+06
mysql_info_schema_user_statistics_select_commands_total{user="ui_dataggdae_app"} 24068
mysql_info_schema_user_statistics_select_commands_total{user="vporalla_dbo"} 13
# HELP mysql_info_schema_user_statistics_table_rows_read_total The number of rows read from tables by this userâ€™s connections. (It may be different from ROWS_FETCHED.)
# TYPE mysql_info_schema_user_statistics_table_rows_read_total counter
mysql_info_schema_user_statistics_table_rows_read_total{user="adengine_app"} 7.5516836638e+10
mysql_info_schema_user_statistics_table_rows_read_total{user="adland_svc_app"} 1.897971993e+09
mysql_info_schema_user_statistics_table_rows_read_total{user="ampx_indexer_app"} 5.0617063911e+10
mysql_info_schema_user_statistics_table_rows_read_total{user="cte_app"} 1.51552321e+08
mysql_info_schema_user_statistics_table_rows_read_total{user="log_rotation_app"} 21372
mysql_info_schema_user_statistics_table_rows_read_total{user="pgurung"} 0
mysql_info_schema_user_statistics_table_rows_read_total{user="pmm_app"} 1.68930314e+09
mysql_info_schema_user_statistics_table_rows_read_total{user="pub_revrpt_app"} 178196
mysql_info_schema_user_statistics_table_rows_read_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_table_rows_read_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_table_rows_read_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_table_rows_read_total{user="tungsten"} 8.11266425e+08
mysql_info_schema_user_statistics_table_rows_read_total{user="ui_dataggdae_app"} 1.2655324e+07
mysql_info_schema_user_statistics_table_rows_read_total{user="vporalla_dbo"} 7783
# HELP mysql_info_schema_user_statistics_total_connections The number of connections created for this user.
# TYPE mysql_info_schema_user_statistics_total_connections counter
mysql_info_schema_user_statistics_total_connections{user="adengine_app"} 23
mysql_info_schema_user_statistics_total_connections{user="adland_svc_app"} 50033
mysql_info_schema_user_statistics_total_connections{user="ampx_indexer_app"} 6078
mysql_info_schema_user_statistics_total_connections{user="cte_app"} 4
mysql_info_schema_user_statistics_total_connections{user="log_rotation_app"} 65
mysql_info_schema_user_statistics_total_connections{user="pgurung"} 1
mysql_info_schema_user_statistics_total_connections{user="pmm_app"} 96003
mysql_info_schema_user_statistics_total_connections{user="pub_revrpt_app"} 307
mysql_info_schema_user_statistics_total_connections{user="rmukkama"} 1
mysql_info_schema_user_statistics_total_connections{user="rpt_gen_app"} 7
mysql_info_schema_user_statistics_total_connections{user="skumar_dbo"} 1
mysql_info_schema_user_statistics_total_connections{user="tungsten"} 747064
mysql_info_schema_user_statistics_total_connections{user="ui_dataggdae_app"} 22373
mysql_info_schema_user_statistics_total_connections{user="vporalla_dbo"} 7
# HELP mysql_info_schema_user_statistics_total_ssl_connections_total The number of times this userâ€™s connections connected using SSL to the server.
# TYPE mysql_info_schema_user_statistics_total_ssl_connections_total counter
mysql_info_schema_user_statistics_total_ssl_connections_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="ampx_indexer_app"} 6078
mysql_info_schema_user_statistics_total_ssl_connections_total{user="cte_app"} 4
mysql_info_schema_user_statistics_total_ssl_connections_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="pgurung"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="rpt_gen_app"} 7
mysql_info_schema_user_statistics_total_ssl_connections_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="tungsten"} 2
mysql_info_schema_user_statistics_total_ssl_connections_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_total_ssl_connections_total{user="vporalla_dbo"} 3
# HELP mysql_info_schema_user_statistics_update_commands_total The number of UPDATE commands executed from this userâ€™s connections.
# TYPE mysql_info_schema_user_statistics_update_commands_total counter
mysql_info_schema_user_statistics_update_commands_total{user="adengine_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="adland_svc_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="ampx_indexer_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="cte_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="log_rotation_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="pgurung"} 0
mysql_info_schema_user_statistics_update_commands_total{user="pmm_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="pub_revrpt_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="rmukkama"} 0
mysql_info_schema_user_statistics_update_commands_total{user="rpt_gen_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="skumar_dbo"} 0
mysql_info_schema_user_statistics_update_commands_total{user="tungsten"} 4.8299743e+07
mysql_info_schema_user_statistics_update_commands_total{user="ui_dataggdae_app"} 0
mysql_info_schema_user_statistics_update_commands_total{user="vporalla_dbo"} 2340

Peter · April 17, 2020, 2:52pm

OK. This means your exporter returns data. Now go to prometheus query interface ( https://PMMSERVERIP/prometheus) and enter “mysql_info_schema_user_statistics_update_commands_total” to see if you’re having data for servers in questions actually making it into Prometheus.

skumar · April 17, 2020, 3:08pm

In the prometheus query interface I have enter the “mysql_info_schema_user_statistics_update_commands_total” and did not found the following servers.
e-mysql-01.lga1.admarketplace.net

w-mysql-01.sfo1.admarketplace.net

w-mysql-02.sfo1.admarketplace.net

w-mysql-03.sfo1.admarketplace.net

Peter · April 18, 2020, 2:26pm

If you see the data in the exporter output but not in the prometheus when most likely you have timeouts getting this data for some reason.

skumar · April 21, 2020, 10:05am

Is there a way to find out the root cause ?
The configuration is similar for all the servers.
FYI, the following hosts names are in the list but graphs are not avaiable
> w-mysql-02.sfo1.admarketplace.net
> w-mysql-03.sfo1.admarketplace.net

Topic		Replies	Views
Pmm graphs not visible PMM 1.x pmm	9	992	April 10, 2020
MySQL Table Statistics dashboard no results PMM 1.x	1	767	November 18, 2016
No charted data in PMM 1.0.2 ? PMM 1.x	2	663	September 14, 2016
PMM - Some Metrics (Checkpoints) are not shown with MySQL PMM 2.x	3	511	February 4, 2021
PMM2 - no data for graphs PMM 2.x pmm	3	839	June 29, 2023

MySQL User Statistics is enabled but graphs are not displayed in Pmm MySQL User Statistics dashboard

Related topics