Not the answer you need?
Register and ask your own question!

QAN fails to load when RDS in different region/availability zone

mysql:queries data appears to stop loading after a period of time when the pmm server is in a different region or different availability zone within the same region of the RDS server.

It starts logging the following warnings and errors repeatedly when it stops loading (partial logs).

2017/01/05 15:51:00.001149 WARNING qan-analyzer-e4b1835d Skipping interval '14 2017-01-05 15:50:00 UTC to 2017-01-05 15:51:00 UTC (0-0)' because interval '13 2017-01-05 15:49:00 UTC to 2017-01-05 15:50:00 UTC (0-0)' is still being parsed
2017/01/05 15:50:03.073172 ERROR qan-analyzer-e4b1835d-worker Got class twice: .....
2017/01/05 15:54:00.001206 WARNING qan-analyzer-e4b1835d-worker Interval out of sequence: got 17, expected 16

I have set METRICS_RESOLUTION=5s to attempt to mitigate, but it still appears to occur.

The only solution I have found is to deploy the server within the same availability zone as the RDS. Are there any additional methods or future releases that will correct this issue (at least for instances within the same region)?

Thanks

Comments

  • MykolaMykola Percona Percona Staff Role
    It is need to understand that Amazon availability zones in different regions connected via public internet channels (see guide), in result they have regular delays and connectivity issues as in public internet.
    PMM server has very high resolution, so network connectivity can become an issue.

    Can you change collect interval for qan-analyzer from 1 minute to 5 minutes?
    It is needed to login to Percona Query Analytics Web Interface, press "SETTINGS" button and change "Collect interval" value".
  • percona_questionspercona_questions Entrant Inactive User Role Beginner
    Thanks - I'm not sure why I missed this setting. I'm still testing with different values, but this appears to have mitigated the problem.
This discussion has been closed.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.