# PMM MySQL low-res exporter : context deadline exceeded **URL:** \ **Category:** PMM 2.x\ **Created:** [October 7, 2020, 7:46am UTC](https://forums.percona.com/t/pmm-mysql-low-res-exporter-context-deadline-exceeded/8105 "2020-10-07T07:46:23Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![babine](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/babine/32/4_2.png) [@babine](https://forums.percona.com/u/babine)\ **Post date:** [October 7, 2020, 7:46am UTC](https://forums.percona.com/t/pmm-mysql-low-res-exporter-context-deadline-exceeded/8105/1 "2020-10-07T07:46:23Z")
Hello, I have a (mostly) working PMM2 (2.10.0) install. However I noticed that some dashboard seem mostly empty or have very few metrics (lots of gaps). After investigation it seems that all the “low-res” prometheus target are taking too long to be scraped (seen on “Unealthy” on [http://pmmserver/prometheus/targets](http://pmmserver/prometheus/targets) page), for example : ```auto Get "http://devdbserver:42000/metrics?collect%5B%5D=binlog_size&collect%5B%5D=custom_query.lr&collect%5B%5D=engine_tokudb_status&collect%5B%5D=global_variables&collect%5B%5D=heartbeat&collect%5B%5D=info_schema.clientstats&collect%5B%5D=info_schema.innodb_tablespaces&collect%5B%5D=info_schema.userstats&collect%5B%5D=perf_schema.eventsstatements&collect%5B%5D=perf_schema.file_instances": context deadline exceeded ``` This is a dev server on which we have “quite some” tables (which is nowhere near what is on production servers) : ```auto # find /var/lib/mysql -name '*.ibd' | wc -l 98301 ``` Getting the metrics locally (to avoid possible network issues) takes around 17s : ```auto # time curl http://localhost:42000/metrics-lr -u 'pmm:/agent_id/ ********' > /tmp/out.txt <(14:56:13)> % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 145M 0 145M 0 0 8585k 0 --:--:-- 0:00:17 --:--:-- 35.3M curl http://localhost:42000/metrics-lr -u > /tmp/out.txt 0.02s user 0.14s system 0% cpu 17.394 total ``` The metrics are 145M for around 1M lines. Most represented metrics are the following : ```auto $ grep -v '^#' /tmp/out.txt | cut -f1 -d'{' | sort | uniq -c | sort -h | tail 250 mysql_perf_schema_events_statements_sort_rows_total 250 mysql_perf_schema_events_statements_tmp_disk_tables_total 250 mysql_perf_schema_events_statements_tmp_tables_total 250 mysql_perf_schema_events_statements_total 250 mysql_perf_schema_events_statements_warnings_total 98145 mysql_info_schema_innodb_tablespace_allocated_size_bytes 98145 mysql_info_schema_innodb_tablespace_file_size_bytes 98145 mysql_info_schema_innodb_tablespace_space_info 396528 mysql_perf_schema_file_instances_bytes 396528 mysql_perf_schema_file_instances_total ``` Is there a way to increase the timeout or maybe not export some of these metrics ? (For the record HR and MR targets take less than 0.1s and 1s respectively from a remote server) ---
**Author:** ![Agustin\_G](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/agustin_g/32/14111_2.png) [@Agustin\_G](https://forums.percona.com/u/Agustin_G)\ **Post date:** [October 12, 2020, 3:57pm UTC](https://forums.percona.com/t/pmm-mysql-low-res-exporter-context-deadline-exceeded/8105/2 "2020-10-12T15:57:57Z")
Hi babine, We have created the following bug to track this some days ago: [https://jira.percona.com/browse/PMM-6744](https://jira.percona.com/browse/PMM-6744), which is most likely what you are seeing. Can you double-check if collecting all but perf\_schema.file\_instances will make the curl command take less than 10 seconds for you too? ```auto time curl -u 'pmm:/agent_id/ ********' http://devdbserver:42000/metrics?collect%5B%5D=binlog_size&collect%5B%5D=custom_query.lr&collect%5B%5D=engine_tokudb_status&collect%5B%5D=global_variables&collect%5B%5D=heartbeat&collect%5B%5D=info_schema.clientstats&collect%5B%5D=info_schema.innodb_tablespaces&collect%5B%5D=info_schema.userstats&collect%5B%5D=perf_schema.eventsstatements >/dev/null ``` Best, Agustín. ---
**Author:** ![babine](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/babine/32/4_2.png) [@babine](https://forums.percona.com/u/babine)\ **Post date:** [October 13, 2020, 4:08am UTC](https://forums.percona.com/t/pmm-mysql-low-res-exporter-context-deadline-exceeded/8105/3 "2020-10-13T04:08:53Z")
Thanks for the answer, good news ! On the dev server itself : ```auto # time curl 'http://localhost:42000/metrics?collect%5B%5D=binlog_size&collect%5B%5D=custom_query.lr&collect%5B%5D=engine_tokudb_status&collect%5B%5D=global_variables&collect%5B%5D=heartbeat&collect%5B%5D=info_schema.clientstats&collect%5B%5D=info_schema.innodb_tablespaces&collect%5B%5D=info_schema.userstats&collect%5B%5D=perf_schema.eventsstatement' -u 'pmm:/agent_id/ ********' > /tmp/out.txt % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 37.4M 0 37.4M 0 0 11.4M 0 --:--:-- 0:00:03 --:--:-- 11.4M curl -u 'pmm:/agent_id/ ********' > /tmp/out.txt 0.00s user 0.05s system 1% cpu 3.336 total ``` It takes a bit more than 3 seconds for 38MB (locally) I tried modifying the prometheus.yml file in the PMM server docker to change the timeout to 30 seconds but it was somehow reverted to 10 seconds upon restart. ---
**Author:** ![Agustin\_G](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/agustin_g/32/14111_2.png) [@Agustin\_G](https://forums.percona.com/u/Agustin_G)\ **Post date:** [October 13, 2020, 7:14pm UTC](https://forums.percona.com/t/pmm-mysql-low-res-exporter-context-deadline-exceeded/8105/4 "2020-10-13T19:14:05Z")
Great! I suggest you to follow that JIRA ticket, then, to get the latest updates on when it will be resolved. Regarding: > I tried modifying the prometheus.yml file in the PMM server docker to change the timeout to 30 seconds but it was somehow reverted to 10 seconds upon restart. Unfortunately, there is a maximum scrape\_timeout set to 10s globally. Even if you change the scrape\_interval to something greater, this is currently the maximum allowed (and it will be overwritten automatically if you manually change the config file, as you have already noted). You can always create a new feature request, if you think it will be worth it.