- All graphs on “MySQL Performance Schema” show “No datapoints” for bad server, for other servers there is data.
- what mysql vendor and version do you use?
3. what mysql vendor and version do you use?
Percona-Server-client-57-5.7.17-11.1.el7.x86_64
Percona-Server-server-57-5.7.17-11.1.el7.x86_64
Percona-Server-shared-57-5.7.17-11.1.el7.x86_64
Percona-Server-shared-compat-57-5.7.17-11.1.el7.x86_64
can you share output of following commands?
grep collect.perf_schema.file_events /etc/systemd/system/pmm-mysql-metrics-42002.service
curl https://bad-server:42002/metrics-mr --insecure | grep file_events_total
grep collect.perf_schema.file_events /etc/systemd/system/pmm-mysql-metrics-42002.service
ExecStart=/bin/sh -c '/usr/local/percona/pmm-client/mysqld_exporter -collect.auto_increment.columns=false -collect.binlog_size=true -collect.global_status=true -collect.global_variables=true -collect.info_schema.innodb_metrics=true -collect.info_schema.processlist=true -collect.info_schema.query_response_time=true -collect.info_schema.tables=false -collect.info_schema.tablestats=false -collect.info_schema.userstats=true -collect.perf_schema.eventswaits=true -collect.perf_schema.file_events=true -collect.perf_schema.indexiowaits=false -collect.perf_schema.tableiowaits=false -collect.perf_schema.tablelocks=false -collect.slave_status=true -web.listen-address=192.168.15.9:42002 -web.auth-file=/usr/local/percona/pmm-client/pmm.yml -web.ssl-cert-file=/usr/local/percona/pmm-client/server.crt -web.ssl-key-file=/usr/local/percona/pmm-client/server.key >> /var/log/pmm-mysql-metrics-42002.log 2>&1'
curl [url]https://mysql.db:42002/metrics-mr[/url] --insecure | grep file_events_total
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 117k 100 117k 0 0 9890 0 0:00:12 0:00:12 --:--:-- 30770
# HELP mysql_perf_schema_file_events_total The total file events by event name/mode.
# TYPE mysql_perf_schema_file_events_total counter
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/FRM",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/FRM",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/FRM",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/data",mode="misc"} 2.270769e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/data",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/data",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/metadata",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/metadata",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/archive/metadata",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/data",mode="misc"} 3.8408893e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/data",mode="read"} 3.840741e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/data",mode="write"} 2.3695092e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/metadata",mode="misc"} 325
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/metadata",mode="read"} 47
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/metadata",mode="write"} 62
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/update",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/update",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/csv/update",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_bmp_file",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_bmp_file",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_bmp_file",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_data_file",mode="misc"} 1.2392295e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_data_file",mode="read"} 1.054533e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_data_file",mode="write"} 1.153722e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_log_file",mode="misc"} 2.4138254e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_log_file",mode="read"} 36
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_log_file",mode="write"} 2.4137377e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_parallel_dblwrite_file",mode="misc"} 1.205863e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_parallel_dblwrite_file",mode="read"} 1
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_parallel_dblwrite_file",mode="write"} 1.204743e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_temp_file",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_temp_file",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/innodb/innodb_temp_file",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/data_tmp",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/data_tmp",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/data_tmp",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/dfile",mode="misc"} 1.2722018e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/dfile",mode="read"} 4.78655e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/dfile",mode="write"} 1.376233e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/kfile",mode="misc"} 9.517399e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/kfile",mode="read"} 2.706582e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/kfile",mode="write"} 1.5881372e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/log",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/log",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisam/log",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisammrg/MRG",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisammrg/MRG",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/myisammrg/MRG",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/charset",mode="misc"} 2
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/charset",mode="read"} 1
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/charset",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/cnf",mode="misc"} 2
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/cnf",mode="read"} 3
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/cnf",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/proc_meminfo",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/proc_meminfo",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/mysys/proc_meminfo",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/partition/ha_partition::parfile",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/partition/ha_partition::parfile",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/partition/ha_partition::parfile",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/ERRMSG",mode="misc"} 2
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/ERRMSG",mode="read"} 3
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/ERRMSG",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/FRM",mode="misc"} 6.1159669e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/FRM",mode="read"} 4.5169241e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/FRM",mode="write"} 5.7591433e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/LOAD_FILE",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/LOAD_FILE",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/LOAD_FILE",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog",mode="misc"} 1.487518e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog",mode="read"} 21703
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog",mode="write"} 1.9176119e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_cache",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_cache",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_cache",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_index",mode="misc"} 4971
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_index",mode="read"} 904
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_index",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_index_cache",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_index_cache",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/binlog_index_cache",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/casetest",mode="misc"} 15
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/casetest",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/casetest",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/dbopt",mode="misc"} 6808
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/dbopt",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/dbopt",mode="write"} 2428
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/des_key_file",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/des_key_file",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/des_key_file",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/file_parser",mode="misc"} 65114
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/file_parser",mode="read"} 18050
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/file_parser",mode="write"} 1670
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/global_ddl_log",mode="misc"} 2
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/global_ddl_log",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/global_ddl_log",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/init",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/init",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/init",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/io_cache",mode="misc"} 208112
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/io_cache",mode="read"} 1.449626e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/io_cache",mode="write"} 1.69409e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/load",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/load",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/load",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/log_event_data",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/log_event_data",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/log_event_data",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/log_event_info",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/log_event_info",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/log_event_info",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/map",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/map",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/map",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/master_info",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/master_info",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/master_info",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/misc",mode="misc"} 2429
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/misc",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/misc",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/partition_ddl_log",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/partition_ddl_log",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/partition_ddl_log",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/pid",mode="misc"} 2
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/pid",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/pid",mode="write"} 1
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/query_log",mode="misc"} 63
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/query_log",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/query_log",mode="write"} 3.94836937e+08
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relay_log_info",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relay_log_info",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relay_log_info",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_cache",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_cache",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_cache",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_index",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_index",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_index",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_index_cache",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_index_cache",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/relaylog_index_cache",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/select_to_file",mode="misc"} 1.0137832e+07
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/select_to_file",mode="read"} 872489
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/select_to_file",mode="write"} 872489
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/send_file",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/send_file",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/send_file",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/slow_log",mode="misc"} 957
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/slow_log",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/slow_log",mode="write"} 6.680671e+06
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/tclog",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/tclog",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/tclog",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/trigger",mode="misc"} 810
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/trigger",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/trigger",mode="write"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/trigger_name",mode="misc"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/trigger_name",mode="read"} 0
mysql_perf_schema_file_events_total{event_name="wait/io/file/sql/trigger_name",mode="write"} 0
looks like data exists in exporter
so lets check prometheus
is “mysql.db” bad host?
Is [url]https://mysql.db:42002/metrics-mr[/url] target “UP” in prometheus targets? [url]http://PMM-SERVER-IP/prometheus/targets[/url]
can you share graph which shown for the following link?
[url]http://PMM-SERVER-IP/prometheus/graph?g0.range_input=1h&g0.expr=mysql_perf_schema_file_events_total{instance%3D"mysql.db"}&g0.tab=0[/url]
this link should show graph for query
Yes, mysql.db is bad host.
PMM and Percona server was updated to latest versions - 1.1.2 and 5.7.17-13
-
Is [URL]https://mysql.db:42002/metrics-mr[/URL] target “UP” in prometheus targets? [URL]http://PMM-SERVER-IP/prometheus/targets[/URL]
-
can you share graph which shown for the following link?
[URL=“http://PMM-SERVER-IP/prometheus/graph?g0.range_input=1h&g0.expr=mysql_perf_schema_file_events_total{instance%3D"mysql.db"}&g0.tab=0”]http://PMM-SERVER-IP/prometheus/grap...22}&g0.tab=0[/URL]
this link should show graph for query
