# PMM Gravana does not show all data **URL:** \ **Category:** PMM 1.x\ **Created:** [February 17, 2017, 5:03am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417 "2017-02-17T05:03:33Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 1, 2017, 7:05am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/21 "2017-03-01T07:05:52Z")
I had check Prometheus again. Looks like we watch wrong parameter(mysql\_global\_status\_connections). Look on screen shot(third server were added, and its work fine): ![prometheus1.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/8/89a0a6a851812fb926b65bff3c8741a01d02895b.png) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 2, 2017, 6:42am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/22 "2017-03-02T06:42:08Z")
can you check exporter output? ```auto curl https://login:password@pmm-client:42002/metrics-hr --insecure | grep mysql_global_variables_max_connections ``` ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 2, 2017, 7:06am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/23 "2017-03-02T07:06:41Z")
root@2f744998d466:/opt# curl [https://root](https://root):somepass@good\_server:42002/metrics-hr --insecure | grep mysql\_global\_variables\_max\_connections % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 102k 100 102k 0 0 1436k 0 --:–:-- --:–:-- --:–:-- 1446k root@2f744998d466:/opt# curl [https://root](https://root):somepass@bad\_server:42002/metrics-hr --insecure | grep mysql\_global\_variables\_max\_connections % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 103k 100 103k 0 0 1205k 0 --:–:-- --:–:-- --:–:-- 1215k ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 2, 2017, 10:14am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/24 "2017-03-02T10:14:05Z")
Oh, Aleksey, sorry, right command ```auto curl https://login:password@pmm-client:42002/metrics-lr --insecure | grep mysql_global_variables_max_connections ``` ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 2, 2017, 10:50am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/25 "2017-03-02T10:50:49Z")
root@2f744998d466:/opt# curl [https://root:somepass@good\_server:42002/metrics-lr](https://root) --insecure | grep mysql\_global\_variables\_max\_connections % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:–:-- 0:00:01 --:–:-- 0# HELP mysql\_global\_variables\_max\_connections Generic gauge metric from SHOW GLOBAL VARIABLES. # TYPE mysql\_global\_variables\_max\_connections gauge mysql\_global\_variables\_max\_connections 151 100 5742k 100 5742k 0 0 2845k 0 0:00:02 0:00:02 --:–:-- 2846k root@2f744998d466:/opt# curl [https://root:somepass@bad\_server::42002/metrics-lr](https://root) --insecure | grep mysql\_global\_variables\_max\_connections % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 31.4M 0 1056 0 0 130 0 70:24:32 0:00:08 70:24:24 215# HELP mysql\_global\_variables\_max\_connections Generic gauge metric from SHOW GLOBAL VARIABLES. # TYPE mysql\_global\_variables\_max\_connections gauge mysql\_global\_variables\_max\_connections 800 100 31.4M 100 31.4M 0 0 3487k 0 0:00:09 0:00:09 --:–:-- 8014k ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 2, 2017, 11:28am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/26 "2017-03-02T11:28:36Z")
So, am I understand correctly, values exists in exporter output and in prometheus database? ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 3, 2017, 1:11am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/27 "2017-03-03T01:11:37Z")
May be i confuse you with previous picture, because there is 3 servers but we talk about 2. mysql\_global\_variables\_max\_connections does not showed on picture for bad server, there is only two good servers. I.e. mysql\_global\_variables\_max\_connections values exists in exporter output for bad and good server, values exists in prometheus ONLY for good server(s). I just cant exclude second good server from picture. But if you compare mysql\_global\_status\_connections and ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 3, 2017, 1:35am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/28 "2017-03-03T01:35:27Z")
Hi, can you open targrets page? [url][http://PMM-SERVER-IP/prometheus/targets[/url]](http://PMM-SERVER-IP/prometheus/targets%5B/url%5D) and check status (UP or DOWN) of target. also can you copy-paste “Last Scrape” value, wait a few seconds, refresh page and copy new “Last Scrape” values again (3-5 values needed) ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 3, 2017, 3:15am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/29 "2017-03-03T03:15:10Z")
instance “mysql.db” is a bad server. Others good. ![5.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/f/f59dc1f972659b821955e5bf9062b837003dd796.png) ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 3, 2017, 3:19am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/30 "2017-03-03T03:19:40Z")
Last Scrape: 1m4.203s ago 11.794s ago 19.312s ago 28.762s ago 37.713s ago 44.95s ago 51.95s ago 58.661s ago ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 3, 2017, 3:47am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/31 "2017-03-03T03:47:51Z")
do you see any errors in /var/log/prometheus.log ? can you share it? ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 7, 2017, 1:07am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/32 "2017-03-07T01:07:25Z")
[URL][DepositFiles](http://depositfiles.com/files/7172vjm7t%5B/URL%5D) OR [url][http://rg.to/file/78fa59c303b761b7832ab38facd92001/prometheus.log.html[/url]](http://rg.to/file/78fa59c303b761b7832ab38facd92001/prometheus.log.html%5B/url%5D) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 7, 2017, 3:30am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/33 "2017-03-07T03:30:30Z")
Looks like you have high load. can you change config? try set section ```auto docker exec -it pmm-server vi /opt/prometheus/prometheus.yml docker restart pmm-server ``` after that please monitor prometheus targets page [url][http://pmm-server-ip/prometheus/targets[/url]](http://pmm-server-ip/prometheus/targets%5B/url%5D) /metrics-mr and /metrics-lr targets should became green ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 7, 2017, 10:21am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/34 "2017-03-07T10:21:04Z")
I changed it & restart the pmm-server. After that: It did not work abour 30 mins. In 30 mins: And after that i restart pmm again. It is not work for now: ![1.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/8/88f039871c2064bcc7b748f35bbf71b7e4cebd4e.png) ![2.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/3/34f8c513cc968a247e3d3cc3d6053ed5d96c8bb7.png) ![3.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/c/cf1e7e3993113359d8115ca3e41e94ec198931b5.png) ![4.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/4/442947b3d5ed026a8263d1d9b60d41da12fa5d9c.png) ![6.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/6/60861ead7fa57bf1a0a5e9a43d7fad3f681bc41a.png) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 7, 2017, 10:30am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/35 "2017-03-07T10:30:29Z")
looks like prometheus become broken due to some typo in the config you can just recreate container (without data lose) please don’t remove pmm-data container, delete only pmm-server ```auto docker stop pmm-server docker rm pmm-server docker run -d \ -p 80:80 \ --volumes-from pmm-data \ --name pmm-server \ --restart always \ percona/pmm-server:1.1.1 ``` ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 8, 2017, 2:41am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/36 "2017-03-08T02:41:59Z")
Done. And we have a progress - now all servers have the same problem 🙂 ![7.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/d/d574dcbee8e99f8a5d591e42c708aff7f9f32737.png) ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 8, 2017, 12:27pm UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/37 "2017-03-08T12:27:03Z")
lets try again to modify for mysql-lr section, but now with config backup and without vim :)) please run the following commands, wait for several minutest and check targets again. ```auto docker exec -it pmm-server cp /opt/prometheus/prometheus.yml /opt/prometheus/prometheus.yml_BAK docker exec -it pmm-server sed -i -e '/mysql-lr/{n;;; s/5s/14s/}' /opt/prometheus/prometheus.yml docker restart pmm-server ``` ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 13, 2017, 7:09am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/38 "2017-03-13T07:09:16Z")
Same result ---
**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\ **Post date:** [March 13, 2017, 8:02am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/39 "2017-03-13T08:02:38Z")
/metrics-lr targets still in “unknown” state? am I understand correctly? ---
**Author:** ![aleksey.filippov](https://avatars.discourse-cdn.com/v4/letter/a/71e660/32.png) [@aleksey.filippov](https://forums.percona.com/u/aleksey.filippov)\ **Post date:** [March 13, 2017, 8:25am UTC](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417/40 "2017-03-13T08:25:19Z")
When i wrote #38, result was like in #36, it is about 1h after pmm restart, i.e. all servers in UNKNOWN state. Now: ![8.png](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/4/487337ed9f70248fd7321832c973ab21db944800.png) [Previous page](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417.md?page=1) [Next page](https://forums.percona.com/t/pmm-gravana-does-not-show-all-data/5417.md?page=3)