# PMM Gravana does not show all data **URL:** \ **Category:** PMM 1.x\ **Created:** [February 17, 2017, 5:03am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417 "2017-02-17T05:03:33Z")\ **Posts on this page:** 20\ **Page:** 3
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 13, 2017, 8:28am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/41 "2017-03-13T08:28:50Z")
And there is still gaps exists on graphics(gaps on pmm-server own statistics): ![9.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/5/58a6d728ff461f4f4ed2f1422c728ce419e437b6.png) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 13, 2017, 9:10am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/42 "2017-03-13T09:10:28Z")
lets revert config from backup and restart ```auto docker exec -it pmm-server cp /opt/prometheus/prometheus.yml_BAK /opt/prometheus/prometheus.yml docker restart pmm-server ``` ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 15, 2017, 1:21am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/43 "2017-03-15T01:21:39Z")
Good: all good servers have all metrix Bad: We have gaps like on pics before, bad server do not have metrics we talk about Question: is it OK that PMM after restart docker container start to show data after 1h? Is start can take so lot of time?.. ![10.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/b/b70ceb903ba79ed5efb8e8cf0f0cb6fd102d2a40.png) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 15, 2017, 1:38am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/44 "2017-03-15T01:38:47Z")
> [@aleksey.filippov;47630](#): > > Question: is it OK that PMM after restart docker container start to show data after 1h? Is start can take so lot of time? no, it is something strange, can you synchronize time on your docker server? ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 15, 2017, 1:59am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/45 "2017-03-15T01:59:03Z")
Hm. Time on pmm-server container delay from db hosts for 3 sec, from my host for 2 sec. Docker server and docker container have the same time. ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 15, 2017, 2:14am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/46 "2017-03-15T02:14:23Z")
Is Grafana only affected or you don’t see graphs also in prometheus? [url][http://PMM-SERVER-IP/prometheus/graph?g0.range\_input=1h&g0.expr=node\_load1&g0.tab=0[/url]](http://PMM-SERVER-IP/prometheus/graph?g0.range_input=1h&g0.expr=node_load1&g0.tab=0%5B/url%5D) ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 15, 2017, 2:45am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/47 "2017-03-15T02:45:31Z")
: Screens from grafana and prometheus ![12.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/4/460c88f3cf1ad808b053d656f523f5cbf788ad3f.png) ![11.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/7/74bb90aef4f5f18fdec3e54fd6343484798b6b08.png) ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 15, 2017, 2:47am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/48 "2017-03-15T02:47:30Z")
But it is Linux metrics, it always exist for bad and good servers. ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 15, 2017, 3:05am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/49 "2017-03-15T03:05:00Z")
Some other news: we have 3 graphs, all different - bad server have no some metrics, other have all metrics, other have gaps in some metrics: ![1.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/3/32e5f256b30d07cdce6570e60a30a19b1e87f2a7.jpeg) ![2.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/c/c63784f5f21ad8db11f2ba7b07f31f0c4da9c972.jpeg) ![3.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/9/9e79eb28d4a3f474af712e9b700daa5afc385b08.jpeg) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 15, 2017, 3:45am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/50 "2017-03-15T03:45:41Z")
> [@](#): > > But it is Linux metrics, it always exist for bad and good servers. oh, can you check prometheus graph for mysql\_global\_variables\_max\_connections ? ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 15, 2017, 9:52am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/51 "2017-03-15T09:52:50Z")
Bad server is missing. And i have feeling like we walk by the circle. ![5.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/c/ce06a7ae3cfd472890af8cca49492e3ba04fc94a.png) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 16, 2017, 5:32am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/52 "2017-03-16T05:32:26Z")
How many tables do you have? I think we have very long or wrong response from [mysqld\_exporter](https://github.com/prometheus/mysqld_exporter). can you measure response time with the following command? ```auto wget https://192.168.200.206:42002/metrics-lr --no-check-certificate ``` wget output should show start time in the first line and end time in the last ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 16, 2017, 6:13am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/53 "2017-03-16T06:13:00Z")
After some internal discussion, my colleague created feature request for mysqld\_exporter - [URL=“[Gather statistics on mysqld\_exporter collection performance · Issue #188 · prometheus/mysqld\_exporter · GitHub](https://github.com/prometheus/mysqld_exporter/issues/188)”][https://github.com/prometheus/mysqld...ter/issues/188[/URL]](https://github.com/prometheus/mysqld...ter/issues/188%5B/URL%5D) feel free to +1 it also we created two internal tickets [URL][[PMM-663] scrape\_duration\_seconds is broken with current prometheus configuration file - Percona JIRA](https://jira.percona.com/browse/PMM-663%5B/URL%5D) [URL][[PMM-664] Gather statistics on mysqld\_exporter collection performance - Percona JIRA](https://jira.percona.com/browse/PMM-664%5B/URL%5D) ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 16, 2017, 7:57am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/54 "2017-03-16T07:57:59Z")
Done 🙂 And ones more. I checked memory usage by mysqld\_exporter with top. Bad server: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18853 root 20 0 2441360 2,054g 4680 S 280,4 13,2 19:02.85 mysqld\_exporter Good servers: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 26945 root 20 0 766688 117948 3560 S 7,0 0,7 2550:20 mysqld\_exporter PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 15257 root 20 0 316880 144584 3168 S 23,6 7,7 2103:36 mysqld\_exporter On bad server RES=2,054g (10 min after pmm-admin restart) On good servers RES=117948(144584) (few days after restart) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 16, 2017, 9:00am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/55 "2017-03-16T09:00:05Z")
How many tables do you have? can you show output of the following command? ```auto wget https://192.168.200.206:42002/metrics-lr --no-check-certificate ``` ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 16, 2017, 9:42am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/56 "2017-03-16T09:42:31Z")
Tables count in bad db = 3639, good db`s = 1736 each About output - command downloaded file 250M size. If you want i can share it somewhere. OUTPUT FROM BAD SERVER: –2017-03-16 17:04:10-- [https://bad\_ip:42002/metrics-lr](https://xxxxxx:42002/metrics-lr) Подключение к bad\_ip:42002… соединение установлено. ПРЕДУПРЕЖДЕНИЕ: невозможно проверить сертификат bad\_ip, выпущенный «O=PMM Client»: Невозможно локально проверить подлинность запрашивающего. HTTP-запрос отправлен. Ожидание ответа… 200 OK Длина: 261433316 (249M) [text/plain] Сохранение в: «metrics-lr» metrics-lr 100%[================================================== ==========================================\>] 249,32M 5,15MB/s in 57s 2017-03-16 17:06:04 (4,34 MB/s) - «metrics-lr» сохранён [261433316/261433316] OUTPUT FROM GOOD SERVER: –2017-03-16 17:48:43-- [https://good\_ip:42002/metrics-lr](https://good_ip:42002/metrics-lr) Подключение к good\_ip:42002… соединение установлено. ПРЕДУПРЕЖДЕНИЕ: невозможно проверить сертификат good\_ip, выпущенный «O=PMM Client»: Невозможно локально проверить подлинность запрашивающего. HTTP-запрос отправлен. Ожидание ответа… 200 OK Длина: 8876739 (8,5M) [text/plain] Сохранение в: «metrics-lr.1» metrics-lr.1 100%[============================================================================================\>] 8,46M 3,30MB/s in 2,6s 2017-03-16 17:48:50 (3,30 MB/s) - «metrics-lr.1» сохранён [8876739/8876739] ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 16, 2017, 11:25am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/57 "2017-03-16T11:25:44Z")
wow! can you share this archive?? ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 16, 2017, 2:21pm UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/58 "2017-03-16T14:21:17Z")
[url][DepositFiles](http://depositfiles.com/files/6x0oiyhuh%5B/url%5D) or [url][http://rg.to/file/79f11a1fb036e734bfe207a525dd46ec/metrics-lr.tar.gz.html[/url]](http://rg.to/file/79f11a1fb036e734bfe207a525dd46ec/metrics-lr.tar.gz.html%5B/url%5D) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 17, 2017, 2:54am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/59 "2017-03-17T02:54:58Z")
I found that table\_statistics is the biggest part of all output (235Mb of 249M). ```auto grep mysql_info_schema_table_statistics_rows_ metrics-lr | wc -c 246603314 ``` can you disable it? ```auto pmm-admin remove mysql:metrics pmm-admin add mysql:metrics --disable-tablestats ``` unfortunately, two graphs on “MySQL Table Statistics” dashboard became empty for bad host - “Top Tables by Rows Read” “Top Tables by Rows Read” finally, after this change everything should work fine 🙂 ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 17, 2017, 5:35am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/60 "2017-03-17T05:35:24Z")
MySQL Table Statistics show no data on all graphics with --disable-tablestats [Previous page](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417.md?page=2) [Next page](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417.md?page=4)