Adding server by hostname instead of IP ok?

Percona Monitoring and Management (PMM) PMM 1.x
#1

Is using a hostname supported instead of an ip? We have a mesos/marathon deploy so the server container does not have a dedicated ip, rather it is routed via haproxy. I can browse the server container fine but when I try and add via the client I get.

Unable to connect to PMM server by address: pmm-server.prdmesos..com

Even though the server is reachable it does not look to be PMM server.
Check if the configured address is correct.

I can also ping that address on pmm-server.prdmesos..com

$ ping pmm-server.prdmesos..com
PING pmm-server.prdmesos..com (10.1.202.165) 56(84) bytes of data.

Any ideas?

#2

Yes, hostname is supported as client address.
To access PMM server, you should be able to connect to pmm-server.prdmesos..com on port 80 (default, if not changed) from the client side.

#3

Any idea why this wouldn’t work then, any tips for what we should check? We can browse the site on port 80 in a browser as well as curl, but when trying to add to the client we get the error

Unable to connect to PMM server by address: pmm-server.prdmesos.<redacted>.com


Even though the server is reachable it does not look to be PMM server.
Check if the configured address is correct.

#4

What command do you run?
pmm-admin config --server pmm-server.prdmesos..com?

Try:
curl http://pmm-server.prdmesos..com/v1/status/leader

You should get “127.0.0.1:8300”.

Note, the server should respond via curl by various paths: /prometheus /v1 /qan-api
For UI, these are: /graph /qan /orchestrator

#5

Hi, I’m working with jasbar on this. We do get responses on /v1/status/leader, /prometheus/, /v1, /qan, but /qan-api, /graph, /orchestrator don’t respond. I’ve taken a look at some of the the logs in the container, and it looks like there are permission issues with the volumes.

I suspect that there is an issue with the fact that I have set up pmm-server to use normal host volumes instead a persistent data container. Do any of you have experience with this?

#6

The volumes should be initialized by the image. So empty ones do not work.
It works for Prometheus, Consul but not for Grafana, QAN API, Orchestrator where mysql datadir should be initialized and loaded.

#7

So we have the server responding now but when we try and add:

sudo pmm-admin add mysql:queries --user root --password 123 --query-source=perfschema

We get:

Error adding MySQL queries: timeout 10s waiting on agent to connect to API.

Checking logs we can see:

2016/12/08 13:38:46.021689 main.go:165: Starting agent…
2016/12/08 13:38:46.022273 main.go:343: Agent is ready
2016/12/08 13:38:46.029994 main.go:207: API is ready
2016/12/08 13:38:47.025344 WARNING log-ws read tcp 192.168.68.16:60127->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:38:47.025466 WARNING agent-ws read tcp 192.168.68.16:60126->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:38:50.027019 WARNING agent-ws read tcp 192.168.68.16:60129->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:38:50.027076 WARNING log-ws read tcp 192.168.68.16:60128->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:38:57.028584 WARNING agent-ws read tcp 192.168.68.16:60130->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:38:57.028670 WARNING log-ws read tcp 192.168.68.16:60131->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:39:12.030263 WARNING agent-ws read tcp 192.168.68.16:60132->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:39:12.030314 WARNING log-ws read tcp 192.168.68.16:60133->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:39:43.031821 WARNING agent-ws read tcp 192.168.68.16:60135->10.1.202.165:80: read: connection reset by peer
2016/12/08 13:39:43.031869 WARNING log-ws read tcp 192.168.68.16:60136->10.1.202.165:80: read: connection reset by peer

In our marathon/mesos setting the connection has to be by hostname and not ip otherwise it won’t route to the correct container. Does this mean it needs a dedicated IP after all?

sudo pmm-admin list
pmm-admin 1.0.6

PMM Server | pmm-server.prdmesos..com
Client Name | prdmesos-db01.
Client Address | 192.168.101.8
Service manager | linux-systemd

SERVICE TYPE NAME CLIENT PORT RUNNING DATA SOURCE OPTIONS

linux:metrics prdmesos-db01. 42000 NO -
mysql:metrics prdmesos-db01. 42002 NO root:***@unix(/var/lib/mysql/mysql.sock) tablestats=OFF

#8

We ended up redeploying the server as a host with a custom port and so clients can now connect.

[sysadmin@prdmesos-db03 ~]$ sudo pmm-admin list
pmm-admin 1.0.6

PMM Server | 192.168.101.8:1007
Client Name | prdmesos-db03
Client Address | 192.168.101.9
Service manager | linux-systemd

SERVICE TYPE NAME CLIENT PORT RUNNING DATA SOURCE OPTIONS

linux:metrics prdmesos-db03 42000 NO -
mysql:queries prdmesos-db03 42001 YES root:@unix(/var/lib/mysql/mysql.sock) query_source=slowlog
mysql:metrics prdmesos-db03 42002 NO root:@unix(/var/lib/mysql/mysql.sock) tablestats=OFF

But none of the dashboards on the server really work, notably orchestrator doesn’t work (really wanted to monitor cluster health).

#9

For mysql:metrics to work, PMM Server should be able to connect back to the client to 192.168.101.9:42002, check out pmm-admin check-network output.
Note, this is a requirement for all metric services, i.e. server > client connection. You may need to configure firewall settings or port forwarding.

For mysql:queries the connection is done from client > server.

For Orchestrator to work, check out https://www.percona.com/doc/percona-…l#orchestrator

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright © 2006 - 2021 Percona LLC. All rights reserved.