Unable to setup proxy between pmm-client and server

Percona Monitoring and Management (PMM) PMM 2.x
3

Hi,
I was working on all the suggestions mentioned in the link you provided. Since I am not that good at Nginx it took me some time to fix the issue. Sharing the details with you.

I have two NGINX proxy servers that are configured with load balancing. All my servers(clients) requests are going through these 2 servers and reach to PMM server(pmm container).

I was able to set up client servers and PMM directly without any issues and the issue was when the requests passing through the proxy-server. I have also configured a proxy domain name for the PMM(All the details are mentioned in my first reply).

Steps I tried.

  1. Tweaked all the firewall rules and make sure that the ports and the IPs are allowed in all servers.
  2. Re-installed the pmm container and assign a custom port for secure connection instead of port 443.
  3. Reconfigured the proxy servers according to the new port
  4. Tried the grpc settings mentioned by you.
  5. Also the SSL settings I tried partially. This means, Installed my domain’s SSL cert on pmm container. Also the same was using on both nginx servers for the proxy domain.(I am not sure these are the steps that the percona techie suggested on the other discussion).

But the issue persisted.

Fix:
Anyway, the fix was related to grpc(or related to all the things I did before). The pmm get connected after I configured the below Nginx conf on my proxy servers by referring to the Nginx documents.

server {
listen 9116 ssl http2;

server_name proxy.mydomain.net;

ssl_certificate /etc/nginx/certificates/wildcard.mydomain.chained.crt;
ssl_certificate_key /etc/nginx/certificates/mydomain.key;
ssl_session_cache shared:SSL:10m;

ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
location / {
grpc_pass grpcs://165.145.588.89:9116;
}
}

References:

Architecture - Percona Monitoring and Management
https://www.percona.com/blog/2020/12/01/foiled-by-the-firewall-a-tale-of-transition-from-prometheus-to-victoriametrics/
Introducing gRPC Support with NGINX 1.13.10 - NGINX
Pmm-agent can not connect to pmm-server when using reverse proxy

4 Likes