Not the answer you need?
Register and ask your own question!

Query Analytics hosts list

Hello, I installed PMM Server 1.13.0 in docker and trying monitor MySQL. Everything is ok but on "Query Analitics" tab I have empty hosts list.

When I open any other metric and then go back to "Query Analitics" I see all queryes etc but hosts list still empty.

Can you help with this issue?

Comments

  • Michael CoburnMichael Coburn Principal Architect, Percona Percona Staff Role
    Hi Tarrok

    Can you please provide the output from:
    pmm-admin list
    
    Also can you provide the contents of the error log file from a host, it is stored in:
    /var/log/pmm-mysql-queries*.log
    
    Lastly, you can fetch the logs from PMM Server in one click by navigating to the Prometheus Dashboard, and under Panel PMM Server Logs you will see a link to download logs.zip to your machine. It would be helpful if you could either DM me, or mail me [email protected] and I can review them.

    I'm surprised to see rows in the Query Analytics output yet no hosts in the drop down - this is the first time I've seen this issue! Share with me the details as requested above and I'll dig further into this!
  • TarrokTarrok Entrant Current User Role Beginner
    Hi Michael Coburn and thank you for replay!


    My pmm-admin command output:
    pmm-admin 1.11.0
    
    PMM Server      | pmm.luxnet.ua (SSL, password-protected)
    Client Name     | db.lux.fm
    Client Address  | 192.168.31.34 
    Service Manager | linux-systemd
    
    -------------- ---------- ----------- -------- ------------------------------------------ --------------------------------------------------------------------------------------
    SERVICE TYPE   NAME       LOCAL PORT  RUNNING  DATA SOURCE                                OPTIONS                                                                               
    -------------- ---------- ----------- -------- ------------------------------------------ --------------------------------------------------------------------------------------
    mysql:queries  db.lux.fm  -           YES      root:***@unix(/mnt/mysql/data/mysql.sock)  query_source=slowlog, query_examples=true, slow_log_rotation=true, retain_slow_logs=1 
    linux:metrics  db.lux.fm  42000       YES      -                                                                                                                                
    mysql:metrics  db.lux.fm  42002       YES      root:***@unix(/mnt/mysql/data/mysql.sock)
    


    pmm-mysql-queries-0.log file contains only Warnings when I have problems with Internet connections on client side:
    # Version: percona-qan-agent 1.0.5
    # Basedir: /usr/local/percona/qan-agent
    # PID:     4261
    # API:     pmm.luxnet.ua/qan-api
    # UUID:    ed7e4565d73b478757c61d0e35c5b078
    2018/08/31 16:51:23.025098 main.go:153: Starting agent...
    2018/08/31 16:51:23.025904 main.go:321: Agent is ready
    2018/08/31 16:51:23.684859 main.go:194: API is ready
    {UUID: CollectFrom: Interval:60 ExampleQueries:0xc4204910a0 MaxSlowLogSize:1073741824 SlowLogRotation:0xc4204910a1 RetainSlowLogs:0xc4204910a8 Start:[] Stop:[] ReportLimit:200}
    {UUID:a2f83a990ac44d7755ae637c86ee75a7 CollectFrom:slowlog Interval:60 ExampleQueries:0xc4204910a0 MaxSlowLogSize:1073741824 SlowLogRotation:0xc4204910a1 RetainSlowLogs:0xc4204910a8 Start:[] Stop:[] ReportLimit:200}
    2018/09/03 12:18:18.882978 WARNING agent ws error: EOF
    2018/09/03 12:18:18.883163 WARNING agent Lost connection to API, reconnecting
    2018/09/03 12:18:21.901516 WARNING agent-ws websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/cmd: dial tcp 194.44.175.8:443: getsockopt: connection refused
    2018/09/03 12:18:21.901718 WARNING log Lost connection to API
    2018/09/03 12:18:25.915836 WARNING log-ws websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/log: dial tcp 194.44.175.8:443: getsockopt: connection refused
    2018/09/03 12:18:28.911215 WARNING agent-ws websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/cmd: dial tcp 194.44.175.8:443: getsockopt: connection refused
    2018/09/03 12:18:32.925623 WARNING log-ws websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/log: dial tcp 194.44.175.8:443: getsockopt: connection refused
    2018/09/06 14:37:36.026398 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:37:49.026906 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:38:02.031524 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:38:39.032367 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:38:52.032887 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:39:05.034407 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:39:42.026397 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:39:55.026880 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    2018/09/06 14:40:08.028351 WARNING data-sender Cannot connect to API:  websocket.Dial wss://pmm.luxnet.ua:443/qan-api/agents/ed7e4565d73b478757c61d0e35c5b078/data: dial tcp 194.44.175.8:443: i/o timeout
    



    Server logs I sent you to email.
  • Michael CoburnMichael Coburn Principal Architect, Percona Percona Staff Role
    Hi Tarrok

    It looks to me that you have some level of network problems - take a look at the dashboard Network Overview, for example:
    http://18.223.109.241/graph/d/000000193/network-overview?refresh=1m&orgId=1&var-interval=$__auto_interval_interval&var-host=pmm-server&var-device=All

    This should show you if you have a higher than normal level of errors.

    I'm a​​​​​​Adding relevant log file data here (no site-specific data exists so it is safe).
    qan-api.log:
    [0m2018/09/06 11:57:55 server.go:2923: http: response.WriteHeader on hijacked connection
    2018/09/06 11:57:55 server.go:2923: http: response.Write on hijacked connection
    [0;31mERROR 2018/09/06 11:57:55 results.go:336: Response write failed: http: connection has been hijacked
    [0m2018/09/06 11:57:55.372 127.0.0.1 404 2.68584ms WS /agents/46a88e4f1a4d4e507c16359e19dda957/cmd
    [0m2018/09/06 11:57:58.575 127.0.0.1 0 129.346539ms WS /agents/58daa0cbc0ea47494f7774c30d8d39d7/data
    [0m2018/09/06 11:57:59.056 127.0.0.1 0 73.210028ms WS /agents/ba8a6c9da09c4f737b4113cbb4f65d92/data
    [0m2018/09/06 11:58:00.146 127.0.0.1 0 39.104597ms WS /agents/1b2f8460a6f1440f4eaa1000cdfb8124/data
    [0m2018/09/06 11:58:00.410 127.0.0.1 0 70.530365ms WS /agents/60f08e980138453042cf14284b4fd659/data
    [0m2018/09/06 11:58:02.413 127.0.0.1 0 73.09395ms WS /agents/021ff46a585942597f64d2e7d90cf03e/data
    [0m2018/09/06 11:58:04 server.go:2923: http: response.WriteHeader on hijacked connection
    2018/09/06 11:58:04 server.go:2923: http: response.Write on hijacked connection
    [0;31mERROR 2018/09/06 11:58:04 results.go:336: Response write failed: http: connection has been hijacked
    [0m2018/09/06 11:58:04.129 127.0.0.1 404 2.97105ms WS /agents/10bf777b28474cdb702d22815db10f11/log
    [0m2018/09/06 11:58:06.337 127.0.0.1 0 8.143179ms WS /agents/2aa04aadcaa54c765cd8d059d9f53b1a/data
    [0m2018/09/06 11:58:06 server.go:2923: http: response.WriteHeader on hijacked connection
    2018/09/06 11:58:06 server.go:2923: http: response.Write on hijacked connection
    [0;31mERROR 2018/09/06 11:58:06 results.go:336: Response write failed: http: connection has been hijacked
    [0m2018/09/06 11:58:06.585 127.0.0.1 404 2.853672ms WS /agents/e7ea27cb7a0c48956d386b9a452cd2c4/cmd
    [0m2018/09/06 11:58:06.403 127.0.0.1 0 207.473388ms WS /agents/0a2dbe8fdb94414d7a85afb03935f7ba/data
    

    error.log
    2018/09/06 11:57:06 [error] 167#0: *569318 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 185.41.250.16, server: _, request: "GET /qan-api/agents/f67247e16851468b450e8da017a780cc/log HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/f67247e16851468b450e8da017a780cc/log", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:16 [error] 166#0: *569344 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/46a88e4f1a4d4e507c16359e19dda957/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/46a88e4f1a4d4e507c16359e19dda957/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:20 [error] 164#0: *569356 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.106, server: _, request: "GET /qan-api/agents/1a50ba9e8bba476d76fedb6ac240d6a6/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/1a50ba9e8bba476d76fedb6ac240d6a6/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:23 [error] 166#0: *569360 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.106, server: _, request: "GET /qan-api/agents/1a50ba9e8bba476d76fedb6ac240d6a6/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/1a50ba9e8bba476d76fedb6ac240d6a6/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:23 [error] 162#0: *569362 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.106, server: _, request: "GET /qan-api/agents/1a50ba9e8bba476d76fedb6ac240d6a6/log HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/1a50ba9e8bba476d76fedb6ac240d6a6/log", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:26 [error] 167#0: *569364 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.106, server: _, request: "GET /qan-api/agents/1a50ba9e8bba476d76fedb6ac240d6a6/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/1a50ba9e8bba476d76fedb6ac240d6a6/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:33 [error] 163#0: *569374 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/6614e3adec544af85cfe09e3b1e33aa6/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/6614e3adec544af85cfe09e3b1e33aa6/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:36 [error] 167#0: *569384 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/b73f88e5f9a84f455d748d190886d925/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/b73f88e5f9a84f455d748d190886d925/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:39 [error] 167#0: *569392 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/b73f88e5f9a84f455d748d190886d925/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/b73f88e5f9a84f455d748d190886d925/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:40 [error] 166#0: *569399 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/6614e3adec544af85cfe09e3b1e33aa6/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/6614e3adec544af85cfe09e3b1e33aa6/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:44 [error] 167#0: *569415 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/797f8ff755cd408b55f2f9118c652076/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/797f8ff755cd408b55f2f9118c652076/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:48 [error] 164#0: *569421 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/44ca625cfb1f4c3a4499e7e756b58791/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/44ca625cfb1f4c3a4499e7e756b58791/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:50 [error] 164#0: *569428 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/10bf777b28474cdb702d22815db10f11/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/10bf777b28474cdb702d22815db10f11/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:51 [error] 161#0: *569437 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/44ca625cfb1f4c3a4499e7e756b58791/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/44ca625cfb1f4c3a4499e7e756b58791/data", host: "pmm.luxnet.ua:443"
    2018/09/06 11:57:54 [error] 164#0: *569447 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 194.44.175.2, server: _, request: "GET /qan-api/agents/44ca625cfb1f4c3a4499e7e756b58791/data HTTP/1.1", upstream: "http://127.0.0.1:9001/agents/44ca625cfb1f4c3a4499e7e756b58791/data", host: "pmm.luxnet.ua:443"
    
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.