# PMM QAN stopped to present information for about one hour and then resumed properly **URL:** \ **Category:** PMM 1.x\ **Created:** [May 13, 2019, 3:30am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011 "2019-05-13T03:30:48Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![avi\_vainshtein](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@avi\_vainshtein](https://forums.percona.com/u/avi_vainshtein)\ **Post date:** [May 13, 2019, 3:30am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/1 "2019-05-13T03:30:48Z")
Greetings, We use PMM 1.17.0 to monitor MySQL DB hosts. Currently two MySQL hosts are being monitored, for few months. Today, the QAN presentation for one of the hosts became unavailable.Namely, the QAN information for the time period of about 09:50-10:50 AM is not presented. When trying to present it - a message appears “There is no query data because the MySQL Server is not configured for monitoring. For details about the required configuration, see Configuring MySQL for Percona Monitoring and Management in PMM documentation.”. If i try to view information before/after the time period 09:50-10:50 AM - everything is fine presented. (That happens only for one of the two monitored MySQL hosts. The second host’s QAN information can be presented OK, including the 09:50-10:50 AM period.) The monitored MySQL DB was up and available all the time. Other than QAN information is presented properly for the 09:50-10:50 AM period, for example “MySQL Overview” or “MySQL InnoDB Metrics”. What can be reason of such a behavior, and is there any solution/workaround for it ? Regards, Avi ---
**Author:** ![avi\_vainshtein](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@avi\_vainshtein](https://forums.percona.com/u/avi_vainshtein)\ **Post date:** [May 13, 2019, 3:39am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/2 "2019-05-13T03:39:37Z")
Please find attached few screenshots. ![photoid=54042](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/9/91a2f11d8bb67d7623635df70af4f79e9a93a881.jpeg) ![photoid=54043](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/b/bd4e7d9ae7564a8b72686c59fdcb22d1706529f0.jpeg) ![photoid=54044](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/0/0ada1ab2e57e0a3e0140bf58b0011541d4000ab3.jpeg) ![photoid=54045](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/9/90631dcba3f599cddede4677df571fb10738a6da.jpeg) ---
**Author:** ![avi\_vainshtein](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@avi\_vainshtein](https://forums.percona.com/u/avi_vainshtein)\ **Post date:** [May 13, 2019, 3:51am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/3 "2019-05-13T03:51:20Z")
I’ve noticed , that the /var/log/pmm-mysql-queries-0.log file at the problematic MySQL host includes many error messages, like : 2019/05/13 10:43:03.319750 analyzer.go:411: qan-analyzer-mysql-31762327-worker crashed: ‘4304 2019-05-13 07:42:03 UTC to 2019-05-13 07:43:03 UTC (0-0)’: runtime error: invalid memory address or nil pointer dereference goroutine 3174513 [running]: runtime/debug.Stack(0x4e5c3c, 0xc42009c0f0, 0x2) /usr/lib/golang/src/runtime/debug/stack.go:24 +0xa7 runtime/debug.PrintStack() /usr/lib/golang/src/runtime/debug/stack.go:16 +0x22 [github.com/percona/qan-agent/qan/analyzer/mysql.(\*RealAnalyzer).runWorker.func1](http://github.com/percona/qan-agent/qan/analyzer/mysql.(*RealAnalyzer).runWorker.func1)(0xc42006ac00, 0xc4207d2a80) /tmp/go/src/github.com/percona/qan-agent/qan/analyzer/mysql/analyzer.go:412 +0x2ea panic(0x7ddd60, 0xa883b0) /usr/lib/golang/src/runtime/panic.go:502 +0x229 [github.com/percona/qan-agent/qan/analyzer/mysql/worker/perfschema.(\*Worker).Cleanup](http://github.com/percona/qan-agent/qan/analyzer/mysql/worker/perfschema.(*Worker).Cleanup)(0xc420196a00, 0x0, 0x0) /tmp/go/src/github.com/percona/qan-agent/qan/analyzer/mysql/worker/perfschema/worker.go:308 +0x77 [github.com/percona/qan-agent/qan/analyzer/mysql.(\*RealAnalyzer).runWorker.func2(0xc42006ac00)](http://github.com/percona/qan-agent/qan/analyzer/mysql.(*RealAnalyzer).runWorker.func2(0xc42006ac00)) /tmp/go/src/github.com/percona/qan-agent/qan/analyzer/mysql/analyzer.go:429 +0x3e [github.com/percona/qan-agent/qan/analyzer/mysql.(\*RealAnalyzer).runWorker](http://github.com/percona/qan-agent/qan/analyzer/mysql.(*RealAnalyzer).runWorker)(0xc42006ac00, 0xc4207d2a80) /tmp/go/src/github.com/percona/qan-agent/qan/analyzer/mysql/analyzer.go:448 +0x386 created by [github.com/percona/qan-agent/qan/analyzer/mysql.(\*RealAnalyzer).run](http://github.com/percona/qan-agent/qan/analyzer/mysql.(*RealAnalyzer).run) /tmp/go/src/github.com/percona/qan-agent/qan/analyzer/mysql/analyzer.go:350 +0x7c9 2019/05/13 10:43:03.319841 ERROR qan-analyzer-mysql-31762327 qan-analyzer-mysql-31762327-worker crashed: ‘4304 2019-05-13 07:42:03 UTC to 2019-05-13 07:43:03 UTC (0-0)’: runtime error: invalid memory address or nil pointer dereference However, similar messages appear not only during the 09:50-10:50 AM time period, but also before and after it also… (A similar log file at the second MySQL host (non-problematic) - does not include such messages at all.) Could those messages be somehow related to the described behavior ? How can i get rid of them ? Regards, Avi ---
**Author:** ![Michael\_Coburn](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/michael_coburn/32/18_2.png) [@Michael\_Coburn](https://forums.percona.com/u/Michael_Coburn)\ **Post date:** [May 13, 2019, 12:39pm UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/4 "2019-05-13T12:39:12Z")
hi [avi vainshtein](https://percona.vanillacommunities.com/profile/x/x/31112) Thanks for reporting this - I don’t have any initial conclusions for you. But can you supply the following: [LIST=1] [_]Attach the output of the logs.zip from PMM Server [URL][Percona Monitoring and Management](https://www.percona.com/doc/percona-monitoring-and-management/deploy/index.html#deploy-pmm.diagnostics-for-support%5B/URL%5D) [_]Attach the output of pmm-admin summary from the Client [URL][Percona Monitoring and Management](https://www.percona.com/doc/percona-monitoring-and-management/pmm-admin.html#pmm-admin-diagnostics-for-support%5B/URL%5D) [/LIST] Thanks! ---
**Author:** ![Michael\_Coburn](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/michael_coburn/32/18_2.png) [@Michael\_Coburn](https://forums.percona.com/u/Michael_Coburn)\ **Post date:** [May 13, 2019, 12:49pm UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/5 "2019-05-13T12:49:00Z")
[avi vainshtein](https://percona.vanillacommunities.com/profile/x/x/31112) Could you also share with us a view of the System Overview for the host where the outage occurred? We want to see how loaded it was during this period ---
**Author:** ![avi\_vainshtein](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@avi\_vainshtein](https://forums.percona.com/u/avi_vainshtein)\ **Post date:** [May 14, 2019, 1:21am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/6 "2019-05-14T01:21:22Z")
Hello [Michael Coburn](https://percona.vanillacommunities.com/profile/x/x/22577) Please find attached screenshots of System Overview for the mentioned time period. ![photoid=54056](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/d/d724c30966a2594a063902b5bcaab13188b92d14.jpeg) ![photoid=54057](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/a/a6844451f64b0de222faf68bba0d4e4100fd15dc.jpeg) ![photoid=54058](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/e/e46b5f10826d9989bd5a878e951840f2f2f92f50.jpeg) ---
**Author:** ![avi\_vainshtein](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@avi\_vainshtein](https://forums.percona.com/u/avi_vainshtein)\ **Post date:** [May 14, 2019, 1:25am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/7 "2019-05-14T01:25:30Z")
Please also find attached the logs.zip from PMM Server [pmm-server\_2019-05-14-06-53.zip](https://forums.percona.com/uploads/short-url/uqKDysXFlf3iTIc6obz8ExqPKMO.zip) (68.1 KB) ---
**Author:** ![avi\_vainshtein](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@avi\_vainshtein](https://forums.percona.com/u/avi_vainshtein)\ **Post date:** [May 14, 2019, 3:26am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/8 "2019-05-14T03:26:38Z")
Please find also attached the output of pmm-admin summary from the Client host : [root@vm-tcmmydbr ~]# pmm-admin summary Collecting information for system diagnostic Collect pmm-admin check-network output … Done Collect pmm-admin list output … Done Collect ps output … Done Collect pt-summary output … Done Collect list of open ports … Done Collect service output … Done Collect pt-mysql-summary output … ERROR 1045 (28000): Access denied for user ‘root’@‘localhost’ (using password: NO) 2019\_05\_14\_10\_21\_03 Cannot connect to MySQL. Check that MySQL is running and that the options after – are correct. Data collection complete. Please attach file summary\_vm-tcmmydbr\_2019-05-14T10\_20\_42.tar.gz to the issue as requested by Percona Support. [summary\_vm-tcmmydbr\_2019-05-14T10\_20\_42.tar.zip](https://forums.percona.com/uploads/short-url/iE6ninD0oGVfhhE24awkP2GQBZm.zip) (155 KB) ---
**Author:** ![lorraine.pocklington](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/lorraine.pocklington/32/37_2.png) [@lorraine.pocklington](https://forums.percona.com/u/lorraine.pocklington)\ **Post date:** [May 14, 2019, 8:03am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/9 "2019-05-14T08:03:14Z")
Hello, in this case I think it would be best to raise this in our JIRA system so that our bug analyst can take a proper look at it in detail. Are you OK to do this? I would help with it but if you own the ticket you will be kept up to date with progress. [https://jira.percona.com](https://jira.percona.com) Let me know if you’d like help with setting that up though! ---
**Author:** ![avi\_vainshtein](https://avatars.discourse-cdn.com/v4/letter/a/3ab097/32.png) [@avi\_vainshtein](https://forums.percona.com/u/avi_vainshtein)\ **Post date:** [May 16, 2019, 3:57am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/10 "2019-05-16T03:57:46Z")
Hello @[lorraine.pocklington](https://www.percona.com/forums/member/41223-lorraine-pocklington) I’ve opened an issue in JIRA system, as per your recommendations. Many thanks, Avi ---
**Author:** ![lorraine.pocklington](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/lorraine.pocklington/32/37_2.png) [@lorraine.pocklington](https://forums.percona.com/u/lorraine.pocklington)\ **Post date:** [May 17, 2019, 3:38am UTC](https://forums.percona.com/t/pmm-qan-stopped-to-present-information-for-about-one-hour-and-then-resumed-properly/7011/11 "2019-05-17T03:38:20Z")
Thank you!