# Can't add mysql:queries to remote pmm server

**URL:** <https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639>\
**Category:** PMM 1.x\
**Created:** [May 26, 2017, 2:59am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639 "2017-05-26T02:59:07Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bolsvet](https://avatars.discourse-cdn.com/v4/letter/b/7ab992/32.png) [@Bolsvet](https://forums.percona.com/u/Bolsvet)\
**Post date:** [May 26, 2017, 2:59am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/1 "2017-05-26T02:59:07Z")

</div>

Hi !  
I was try to add mysql:queries to my remote pmm server, but with no luck.

[root@hotel ~]# pmm-admin list  
pmm-admin 1.1.3

PMM Server | 46.xx.xx.xx (password-protected)  
Client Name | client-dns  
Client Address | 88.xx.xx.xx  
Service Manager | unix-systemv

* * *

SERVICE TYPE NAME LOCAL PORT RUNNING DATA SOURCE OPTIONS

* * *

linux:metrics xxxxx 42000 YES -  
mysql:metrics xxxxx 42002 YES root:\*\*\*@tcp(127.0.0.1:3306) tablestats=OFF

Both servers - mysql and pmm set with public IP and placed in different DC, linux and mysql metric working fine, but when i try add queries i get next error:

pmm-admin add mysql:queries  
Error adding MySQL queries: timeout 10s waiting on agent to connect to API.

if check log

# Version: percona-qan-agent 1.1.3

# Basedir: /usr/local/percona/qan-agent

# PID: 45814

# API: 46.xx.xx.xx/qan-api

# UUID: xxx

2017/05/26 11:22:48.517407 main.go:165: Starting agent…  
2017/05/26 11:22:48.517892 main.go:346: Agent is ready  
2017/05/26 11:22:58.548048 main.go:201: Cannot connect to API: GET [http://admin:admin@192.168.122.68/qan-api/agents/xxx](http://admin) error: client.Do: Get [http://admin:admin@192.168.122.68/qan-api/agents/xxx:](http://admin) dial tcp 192.168.122.68:80: i/o timeout. Verify that the agent UUID and API hostname printed above are correct and that no network issues prevent this host from accessing the API. Connection attempts to API will continue  
until successful, but additional errors will not be logged, and agent will not send data until connected to API.

looks like qan agent try to use default private address for server connection.

if check qan-agent config they contain right server ip

“ApiHostname”:“46.xx.xx.xx”,

So what’s wrong with my installation and why qan-agent still try connect to private server ip except public?

---

<div class="post-metadata">

**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\
**Post date:** [May 26, 2017, 5:38am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/2 "2017-05-26T05:38:02Z")

</div>

Hi [Bolsvet](https://percona.vanillacommunities.com/profile/x/x/38365),

you can review architecture document if you want - [url][Percona Monitoring and Management](https://www.percona.com/doc/percona-monitoring-and-management/architecture.html%5B/url%5D)  
pay attention that PMM Client connects to PMM Server and PMM Server should have a possibility to connect to client.

try the following steps  
remove all services

```auto

reconfigure pmm-admin with the following command

```

pmm-admin config --server pmm-SERVER-PUBLIC-ip --bind-address pmm-CLIENT-PRIVATE-ip --client-address pmm-CLIENT-PUBLIC-ip

```auto

add services again

```

---

<div class="post-metadata">

**Author:** ![Bolsvet](https://avatars.discourse-cdn.com/v4/letter/b/7ab992/32.png) [@Bolsvet](https://forums.percona.com/u/Bolsvet)\
**Post date:** [May 26, 2017, 5:53am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/3 "2017-05-26T05:53:13Z")

</div>

Ok, i do what you say,

pmm-admin remove --all  
OK, 2 services were removed.

pmm-admin config --server 46.xx.xx.xx --bind-address 88.xx.xx.xx --client-address 88.xx.xx.xx --server-user admin --server-password admin  
OK, PMM server is alive.

PMM Server | 46.xx.xx.xx (password-protected)  
Client Name | client-dns  
Client Address | 88.xx.xx.xx

pmm-admin add mysql  
[linux:metrics] OK, now monitoring this system.  
[mysql:metrics] OK, now monitoring MySQL metrics using DSN root:\*\*\*@tcp(127.0.0.1:3306)  
[mysql:queries] Error adding MySQL queries: timeout 10s waiting on agent to connect to API.

but qan agent still try connect to private ip of server

2017/05/26 14:16:30.183509 main.go:201: Cannot connect to API: GET [http://admin](http://admin):admin@192.168.122.68/qan-api/agents/xxx error: client.Do: Get [http://admin](http://admin):admin@192.168.122.68/qan-api/agents/xxx: dial tcp 192.168.122.68:80: i/o timeout. Verify that the agent UUID and API hostname printed above are correct and that no network issues prevent this host from accessing the API. Connection attempts to API will continue  
until successful, but additional errors will not be logged, and agent will not send data until connected to API.

---

<div class="post-metadata">

**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\
**Post date:** [May 26, 2017, 6:06am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/4 "2017-05-26T06:06:16Z")

</div>

> [@Bolsvet;48556](#):
>
> pmm-admin config --server 46.xx.xx.xx --bind-address 88.xx.xx.xx --client-address 88.xx.xx.xx --server-user admin --server-password admin

is 88.xx.xx.xx address local (and visible in ifconfig output)?  
can you share file content?

---

<div class="post-metadata">

**Author:** ![Bolsvet](https://avatars.discourse-cdn.com/v4/letter/b/7ab992/32.png) [@Bolsvet](https://forums.percona.com/u/Bolsvet)\
**Post date:** [May 26, 2017, 6:12am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/5 "2017-05-26T06:12:03Z")

</div>

Yes it visible, and it local for this server

ifconfig

em1 Link encap:Ethernet inet addr:88.xx.xx.xx  
em2 Link encap:Ethernet inet addr:10.x.x.x

content of pmm.yml file

server\_address: 46.xx.xx.xx  
client\_address: 88.xx.xx.xx  
bind\_address: 88.xx.xx.xx  
client\_name: client-dns  
server\_user: admin  
server\_password: admin

---

<div class="post-metadata">

**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\
**Post date:** [May 26, 2017, 6:37am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/6 "2017-05-26T06:37:07Z")

</div>

can PMM Server reach itself?  
can you run the following command on pmm server side and share output?

```auto
docker exec -it pmm-server curl http://admin:admin&#64;192.168.122.68:80/qan-api/

```

Do you have http proxy between pmm client and pmm server?  
can you run the following command on pmm clietn side and share output?

```auto
curl http://admin:admin&#64;46.xx.xx.xx:80/qan-api/

```

---

<div class="post-metadata">

**Author:** ![Bolsvet](https://avatars.discourse-cdn.com/v4/letter/b/7ab992/32.png) [@Bolsvet](https://forums.percona.com/u/Bolsvet)\
**Post date:** [May 26, 2017, 6:53am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/7 "2017-05-26T06:53:35Z")

</div>

> [@Mykola;48560](#):
>
> Do you have http proxy between pmm client and pmm server?

Ok, part of problem was solved. It was problem in nginx - it not send remote ip header to container.

Now when i try to connect

```auto
[root&#64;hotel log]# pmm-admin repair
OK, removed 1 orphaned services.
[root&#64;hotel log]# pmm-admin add mysql
[linux:metrics] OK, already monitoring this system.
[mysql:metrics] OK, already monitoring MySQL metrics.
[mysql:queries] Error adding MySQL queries: timeout 10s waiting on agent to connect to API.

# Version: percona-qan-agent 1.1.3
# Basedir: /usr/local/percona/qan-agent
# PID: 48867
# API: 46.xx.xx.xx/qan-api
# UUID: 60fb0cbdb96648d446f92ba267c392a7
2017/05/26 15:19:03.082891 main.go:165: Starting agent...
2017/05/26 15:19:03.083386 main.go:346: Agent is ready
2017/05/26 15:19:03.118075 main.go:207: API is ready
2017/05/26 15:19:04.110111 WARNING agent-ws bad status
2017/05/26 15:19:04.111023 WARNING log-ws bad status
2017/05/26 15:19:07.135117 WARNING log-ws bad status
2017/05/26 15:19:07.137429 WARNING agent-ws bad status
2017/05/26 15:19:14.161181 WARNING agent-ws bad status
2017/05/26 15:19:14.161585 WARNING log-ws bad status
2017/05/26 15:19:29.183139 WARNING agent-ws bad status
2017/05/26 15:19:29.187081 WARNING log-ws bad status
2017/05/26 15:20:00.211216 WARNING agent-ws bad status
2017/05/26 15:20:00.216240 WARNING log-ws bad status
2017/05/26 15:20:06.104079 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:20:09.124754 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:20:12.145883 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:21:00.233388 WARNING agent-ws bad status
2017/05/26 15:21:00.238123 WARNING log-ws bad status
2017/05/26 15:21:09.104098 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:21:12.126775 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:21:15.146306 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:22:00.254524 WARNING agent-ws bad status
2017/05/26 15:22:00.260663 WARNING log-ws bad status
2017/05/26 15:22:12.104978 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:22:15.124612 WARNING data-sender Cannot connect to API: bad status
2017/05/26 15:22:18.146449 WARNING data-sender Cannot connect to API: bad status

```

---

<div class="post-metadata">

**Author:** ![Mykola](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mykola/32/30_2.png) [@Mykola](https://forums.percona.com/u/Mykola)\
**Post date:** [May 26, 2017, 7:02am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/8 "2017-05-26T07:02:26Z")

</div>

qan-agent use websockets for some API calls (like /agent-ws /log-ws) so you shouldn’t use a proxy or configure nginx to proxying websockets.  
anyway correct work via proxy is not guaranteed.

---

<div class="post-metadata">

**Author:** ![Bolsvet](https://avatars.discourse-cdn.com/v4/letter/b/7ab992/32.png) [@Bolsvet](https://forums.percona.com/u/Bolsvet)\
**Post date:** [May 31, 2017, 1:05am UTC](https://forums.percona.com/t/cant-add-mysql-queries-to-remote-pmm-server/5639/9 "2017-05-31T01:05:16Z")

</div>

I move monitoring node to same private network with monitored mysql hosts and now all work as expected. Thank you !
