QAN agent stops working after MySQL server restart

Hi percona!
It seems that after mysql server restart qan-agent struggles with reconnecting back to mysql.

In docker container: WARN 2016/12/28 13:14:49 ws.go:90: [Agent.Cmd] agent_id=8 0.0.9 http://hostname Failed to start: Version: recv timeout
In agent log:
016/12/24 20:51:39.259539 WARNING agent Lost connection to API
2016/12/24 20:51:49.394491 WARNING mrms-monitor Cannot connect to MySQL :@unix(/var/lib/mysql/mysql.sock): connect: no such file or directory: /var/lib/mysql/mysql.sock
2016/12/24 20:52:00.000747 WARNING qan-interval Cannot connect to MySQL :@unix(/var/lib/mysql/mysql.sock): connect: no such file or directory: /var/lib/mysql/mysql.sock
2016/12/24 20:52:00.267222 WARNING agent ws error: EOF
2016/12/24 20:52:00.267296 WARNING agent Lost connection to API
2016/12/24 20:52:23.274767 WARNING agent ws error: EOF
2016/12/24 20:52:23.274827 WARNING agent Lost connection to API

at 20:51 there was a mysql server restart.
The queries still show up in qan web interface after restart, but EXPLAIN and show create table etc don’t work as qan-api can’t connect back to the pmm-client.
1.0.7 both client and server.

Thanks!

Hi Gleb,

Does restarting the client helps ?

pmm-admin restart mysql:queries

If it is repeatable issue for you can you help us by reporting the Bug: https://jira.percona.com/projects/PMM/issues/?filter=allopenissues

Hi Gleb,

can you wait for 5 minutes after server boot and check if service still down?
command for check - “pmm-admin list” or “pmm-admin check-network”

Hello! Yes, restarting pmm-client helps.
Yeah, it is a repeatable issue, I will open the Bug.

Hi Gleb,

can you wait for 5 minutes after server boot and check if service still down?