Percona Everest v0.8.0. Issue with PMM and certificate

Percona Everest
, ,
1

Hi all,
I want to report a bug I found when I try to create a monitoring-instances.
POST request from the FE is missing an “apiKey” and this is why I cannot create a monitoring endpoints. After that, I use Postman to create the endpoints and it’s works. I dont sure it’s bug from code or my knowledge but I want to share if someone get it.

perconabugg
perconabugg1858×1080 321 KB

2

I remember that we had a problem when the URL contains / at the end.
But I think it’s been fixed.
Try the URL without the “/”.

Anyway, we’ve passed on to the team in Slack and someone will check in tomorrow.

1 Like
3

I had try with and without “/”, but just only when I attach an apiKey, this request has succeeded.

1 Like
4

Hello @just3tee

You don’t really need to send the PMM API key if the following 2 conditions are true:

  1. The Everest API server can reach the PMM server
  2. The PMM user has permission to create API keys

This is because when the Everest API server receives the POST /monitoring-instances request it will try to log in to PMM using the provided credentials and create an API key for itself. If any of the 2 conditions above aren’t satisfied it will return an error saying “Could not create an API key in PMM”.

We do have plans to add the option for users to specify the PMM API key directly in the UI (similar to what you’ve done via Postman) but condition 1. will still need to be needed, Everest will always need to be able to communicate with the PMM server.

1 Like
5

Hi @Diogo_Recharte
Thanks for your answer!
I want to provide some information, I use the admin account provided from PMM Server and getting error like what I screenshot. My Everest run inside a DOKS and I expose it through Nginx Ingress with LB IP, PMM server run inside a Docker in the other host but has same vpc network. This is my own setup, is it the reason? T4R

6

I can’t know for certain because I don’t know how your VPC is configured but if you’re using the admin account you definitely have enough permissions so most likely you just can’t reach it from within the k8s cluster.

You can quickly troubleshoot this by running a pod with some networking tools and test it out.

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
  namespace: default
spec:
  containers:
  - name: dnsutils
    image: registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3
    command:
      - sleep
      - "infinity"
    imagePullPolicy: IfNotPresent
  restartPolicy: Always
EOF

Then exec into this pod with kubectl exec -it dnsutils -- sh and run a couple of commands:

  1. you can run nslookup pmm.thedreamhome.click to understand if k8s cluster knows about that DNS name.
  2. ping pmm.thedreamhome.click (or the IP directly) to understand if you can reach it.
1 Like
7

@just3tee I just remembered one other thing. We currently have a limitation when connecting to PMM. We enforce the TLS certificate validation so if your PMM server is using a self-signed certificate you can’t use https to connect to it (the certificate isn’t signed by a known entity so it’s rejected).
The workaround for this is to use http instead https in the endpoint URL.

1 Like
8

image
K8S can reach the pmm server, I already monitoring some db instances run inside k8s cluster, I can see the metrics in the pmm dashboard. So, I think the step registration a monitoring instances have some issues, because, swagger defined a POST request include returned apiKey key but in fe of everest isn’t handle it?, and got the 500 status code, all my knowledge is the request body is not match with the server :frowning: . Sorry if I lack of knowledge about this platform :frowning:

9

It the cert from Let’s Encrypt, ZeroSSL valid? I signed the cert and key from 2 platform

image
image684×541 17.1 KB

10

Well, we fetch the trusted CAs from the standard alpine repos when building the image.
I’m pretty sure that it should be part of chain signed by those trusted CAs :confused:

Just to check if this is a certificate issue can you please try to add the PMM server again using the everest UI and show me the logs of the percona-everest-xxxxx..... pod in the everest-system namespace? If that’s the case, it should log something regarding not being able to verify the cert.

1 Like
11

I will check it and will response in the next week. Happy weekend!

12

Hi @Diogo_Recharte
Thank you for answering me many questions.I troubleshoot follow your suggestion and reinstall pmm-server with this tutorial Automate the SSL Certificate Lifecycle of your Percona Monitoring and Management Server, finally this works. I new to K8S and Percona, so I am happy with the results I have gotten from your suports. Thank you very much!!! :heart_eyes:

1 Like
13

Hi @just3tee

We could really use information from you on how you plan to use Percona Everest, how you use the API.

We would be very grateful if you could share a use case. You can do it privately in a private message to me.

1 Like