Not the answer you need?
Register and ask your own question!

pmm-admin can't add a service

Hi there Percona forums,

I've been trying to set up PMM, I have a server running docker with the PMM-server installed, my client has the pmm-agent installed and the initial connection appeared to succeed using the following command:
sudo pmm-admin config --server-url=https://username:password@server.com --server-insecure-tls

The following configuration was generated:
# Updated by `pmm-agent setup`.
---
id: /agent_id/ID_HERE
listen-port: 7777
server:
  address: server.com:443
  username: username
  password: password
  insecure-tls: true
paths:
  exporters_base: /usr/local/percona/pmm2/exporters
  node_exporter: /usr/local/percona/pmm2/exporters/node_exporter
  mysqld_exporter: /usr/local/percona/pmm2/exporters/mysqld_exporter
  mongodb_exporter: /usr/local/percona/pmm2/exporters/mongodb_exporter
  postgres_exporter: /usr/local/percona/pmm2/exporters/postgres_exporter
  proxysql_exporter: /usr/local/percona/pmm2/exporters/proxysql_exporter
  rds_exporter: /usr/local/percona/pmm2/exporters/rds_exporter
  tempdir: /tmp
ports:
  min: 42000
  max: 51999
debug: false
trace: false

But when I run the pmm-admin command to create a MySQL service, it fails with the following error:
pmm-admin add mysql --query-source=perfschema --username=mysql_username --password=mysql_password name_of_service
Failed to get PMM Server parameters from local pmm-agent: pmm-agent is running, but not set up.
Please run `pmm-admin config` with --server-url flag.

I've noticed that the agent service is running, but when restarting and checking status I see the following:
Jan 31 11:48:19 hostname pmm-agent[19685]: INFO[2020-01-31T11:48:19.219+00:00] Connecting to https://username:***@server.com:443/ ...  component=client
Jan 31 11:48:24 hostname pmm-agent[19685]: ERRO[2020-01-31T11:48:24.219+00:00] Failed to connect to server.com:443: timeout.  component=client

My server is sitting internally behind a firewall and proxy, with ports 443 and 80 open, is there anything else that needs to be opened?

Many thanks,

David

Comments

  • InfluentialDaveInfluentialDave Entrant Current User Role Novice
    Ok, I tried to uninstall and reinstall (using apt), and now when trying to configure the agent I receive the following:
    Checking local pmm-agent status...
    pmm-agent is running.
    Registering pmm-agent on PMM Server...
    Failed to register pmm-agent on PMM Server: Node with name "server.com" already exists..
    

    How do I delete the node from the server? By viewing the following page I can see a list of the added nodes: https://server.com/graph/d/pmm-inventory/pmm-inventory?
    This provides me with 3 tabs, "Services", "Agents" and "Nodes", and I can view the node I want to remove on the "Nodes" tab, but I can't remove it.

    Please help
  • StaterosStateros Contributor Current User Role Patron
    Hello, when you first time added node to monitoring, pmm-server save the name of the node. Till you not removed this node you can't register new one with the same name.
    For now you should execute command for remove service from pmm-server:
     pmm-admin remove mysql <name_of_service>
    
    or you can open url pmm.server.com/swagger/ You can find there how to manage pmm-server without clients.

    For the first issue, check your network, ports and other related staff between your DB machine and pmm-server. Looks like some security groups closed or sth like that
  • InfluentialDaveInfluentialDave Entrant Current User Role Novice
    Hi there,

    Thank you for your response! I wasn't aware of the swagger URL, that helped me to remove the node completely and allowed me to re-register. Now I'm back to the initial problem. Could you please advise what ports are required for communication between the server and client? I only have 443 and 80 open on the server.

    Many thanks,

    David
  • StaterosStateros Contributor Current User Role Patron
    I don't remember exactly but, pmm-server use 42000-42002 ports to contact with different exporters (linux, mysql, etc.) Try to open this range and check. Should help

    jfi: 42003 for MongoDB
  • InfluentialDaveInfluentialDave Entrant Current User Role Novice
    Thank you Stateros,

    Currently ports 40000-50000 are open on my client as I understood the different services utilised a few different ports. My server has ports 80 and 443 open to the client, will this suffice?

    Many thanks,

    David
  • InfluentialDaveInfluentialDave Entrant Current User Role Novice
    I corrected a certificate issue and re-added the client successfully without the --server-insecure-tls flag
    pmm-admin config --server-url=https://admin:***&#64;server.com
    

    The following is an exert of the logs, I can only find journalctl logs, not sure if there are better logs somewhere else?
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.131+00:00] Using /usr/local/percona/pmm2/exporters/node_exporter  component=main
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.131+00:00] Using /usr/local/percona/pmm2/exporters/mysqld_exporter  component=main
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.131+00:00] Using /usr/local/percona/pmm2/exporters/mongodb_exporter  component=main
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.132+00:00] Using /usr/local/percona/pmm2/exporters/postgres_exporter  component=main
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.132+00:00] Using /usr/local/percona/pmm2/exporters/proxysql_exporter  component=main
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.132+00:00] Using /usr/local/percona/pmm2/exporters/rds_exporter  component=main
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.132+00:00] Starting local API server on http://127.0.0.1:7777/ ...  component=local-server/JSON
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.133+00:00] Starting...                                   component=client
    Feb 05 11:37:13 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:13.133+00:00] Connecting to https://admin:***&#64;server.com:443/ ...  component=client
    Feb 05 11:37:14 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:14.134+00:00] Started.                                      component=local-server/JSON
    Feb 05 11:37:18 client.com pmm-agent[24259]: ERRO[2020-02-05T11:37:18.133+00:00] Failed to connect to server.com:443: timeout.  component=client
    Feb 05 11:37:18 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:18.916+00:00] Connecting to https://admin:***&#64;server.com:443/ ...  component=client
    Feb 05 11:37:23 client.com pmm-agent[24259]: ERRO[2020-02-05T11:37:23.916+00:00] Failed to connect to server.com:443: timeout.  component=client
    Feb 05 11:37:25 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:25.327+00:00] Connecting to https://admin:***&#64;server.com:443/ ...  component=client
    Feb 05 11:37:30 client.com pmm-agent[24259]: ERRO[2020-02-05T11:37:30.327+00:00] Failed to connect to server.com:443: timeout.  component=client
    Feb 05 11:37:32 client.com pmm-agent[24259]: INFO[2020-02-05T11:37:32.244+00:00] Connecting to https://admin:***&#64;server.com:443/ ...  component=client
    Feb 05 11:37:37 client.com pmm-agent[24259]: ERRO[2020-02-05T11:37:37.244+00:00] Failed to connect to server.com:443: timeout.  component=client
    

    I'm slightly worried that the client may not be trying the correct URL as the failure message states "server.com:443" which is NOT the same as "https:server.com"

    The following is also a wget from the client to the server to confirm that the page can be reached:
    user&#64;client:~$ wget https://server.com
    converted 'https://server.com' (ISO-8859-1) -> 'https://server.com' (UTF-8)
    --2020-02-05 11:49:04--  https://server.com/
    Resolving server.com (server.com)... 217.33.88.85
    Connecting to server.com (server.com)|217.33.88.85|:443... connected.
    HTTP request sent, awaiting response... 302 Moved Temporarily
    Location: http://server.com/graph/ [following]
    converted 'http://server.com/graph/' (ISO-8859-1) -> 'http://server.com/graph/' (UTF-8)
    --2020-02-05 11:49:04--  http://server.com/graph/
    Connecting to server.com (server.com)|217.33.88.85|:80... connected.
    HTTP request sent, awaiting response... 301 Moved Permanently
    Location: https://server.com/graph/ [following]
    converted 'https://server.com/graph/' (ISO-8859-1) -> 'https://server.com/graph/' (UTF-8)
    --2020-02-05 11:49:04--  https://server.com/graph/
    Connecting to server.com (server.com)|217.33.88.85|:443... connected.
    HTTP request sent, awaiting response... 302 Found
    Location: /graph/login [following]
    converted 'https://server.com/graph/login' (ISO-8859-1) -> 'https://server.com/graph/login' (UTF-8)
    --2020-02-05 11:49:05--  https://server.com/graph/login
    Reusing existing connection to server.com:443.
    HTTP request sent, awaiting response... 200 OK
    Length: unspecified [text/html]
    Saving to: 'index.html.5'
    
    index.html.5                                       [ <=>                                                                                                 ]  34.99K  --.-KB/s   in 0.007s
    
    2020-02-05 11:49:05 (5.01 MB/s) - 'index.html.5' saved [35833]
    
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.