# Percona Everest v0.8.0. Issue with PMM and certificate

**URL:** https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001
**Category:** Percona Everest
**Tags:** pmm, bugs, api
**Created:** [March 6, 2024, 4:46pm UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001 "2024-03-06T16:46:42Z")
**Posts on this page:** 13
**Page:** 1

<div class="post-metadata">

### Author: ![just3tee](https://avatars.discourse-cdn.com/v4/letter/j/a183cd/32.png) [@just3tee](https://forums.percona.com/u/just3tee)
#### Post date: [March 6, 2024, 4:46pm UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/1 "2024-03-06T16:46:42Z")

</div>

Hi all,  
I want to report a bug I found when I try to create a monitoring-instances.  
POST request from the FE is missing an “apiKey” and this is why I cannot create a monitoring endpoints. After that, I use Postman to create the endpoints and it’s works. I dont sure it’s bug from code or my knowledge but I want to share if someone get it.

 ![perconabugg](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/6/6ea47a4afb678e00f76d762f5ef56b4b02666f9d.jpeg)

---

<div class="post-metadata">

### Author: ![daniil.bazhenov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/daniil.bazhenov/32/62_2.png) [@daniil.bazhenov](https://forums.percona.com/u/daniil.bazhenov)
#### Post date: [March 6, 2024, 7:19pm UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/2 "2024-03-06T19:19:04Z")

</div>

I remember that we had a problem when the URL contains / at the end.  
But I think it’s been fixed.  
Try the URL without the “/”.

Anyway, we’ve passed on to the team in Slack and someone will check in tomorrow.

---

<div class="post-metadata">

### Author: ![just3tee](https://avatars.discourse-cdn.com/v4/letter/j/a183cd/32.png) [@just3tee](https://forums.percona.com/u/just3tee)
#### Post date: [March 7, 2024, 6:43am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/3 "2024-03-07T06:43:59Z")

</div>

I had try with and without “/”, but just only when I attach an apiKey, this request has succeeded.

---

<div class="post-metadata">

### Author: ![Diogo\_Recharte](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/diogo_recharte/32/8004_2.png) [@Diogo\_Recharte](https://forums.percona.com/u/Diogo_Recharte)
#### Post date: [March 8, 2024, 9:36am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/4 "2024-03-08T09:36:43Z")

</div>

Hello @just3tee

You don’t really need to send the PMM API key if the following 2 conditions are true:

1. The Everest API server can reach the PMM server
2. The PMM user has permission to create API keys

This is because when the Everest API server receives the POST /monitoring-instances request it will try to log in to PMM using the provided credentials and create an API key for itself. If any of the 2 conditions above aren’t satisfied it will return an error saying “Could not create an API key in PMM”.

We do have plans to add the option for users to specify the PMM API key directly in the UI (similar to what you’ve done via Postman) but condition 1. will still need to be needed, Everest will always need to be able to communicate with the PMM server.

---

<div class="post-metadata">

### Author: ![just3tee](https://avatars.discourse-cdn.com/v4/letter/j/a183cd/32.png) [@just3tee](https://forums.percona.com/u/just3tee)
#### Post date: [March 8, 2024, 9:47am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/5 "2024-03-08T09:47:46Z")

</div>

Hi @Diogo_Recharte  
Thanks for your answer!  
I want to provide some information, I use the admin account provided from PMM Server and getting error like what I screenshot. My Everest run inside a DOKS and I expose it through Nginx Ingress with LB IP, PMM server run inside a Docker in the other host but has same vpc network. This is my own setup, is it the reason? T4R

---

<div class="post-metadata">

### Author: ![Diogo\_Recharte](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/diogo_recharte/32/8004_2.png) [@Diogo\_Recharte](https://forums.percona.com/u/Diogo_Recharte)
#### Post date: [March 8, 2024, 9:57am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/6 "2024-03-08T09:57:39Z")

</div>

I can’t know for certain because I don’t know how your VPC is configured but if you’re using the admin account you definitely have enough permissions so most likely you just can’t reach it from within the k8s cluster.

You can quickly troubleshoot this by running a pod with some networking tools and test it out.

```auto
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: dnsutils
  namespace: default
spec:
  containers:
  - name: dnsutils
    image: registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3
    command:
      - sleep
      - "infinity"
    imagePullPolicy: IfNotPresent
  restartPolicy: Always
EOF

```

Then exec into this pod with `kubectl exec -it dnsutils -- sh` and run a couple of commands:

1. you can run `nslookup pmm.thedreamhome.click` to understand if k8s cluster knows about that DNS name.
2. `ping pmm.thedreamhome.click` (or the IP directly) to understand if you can reach it.

---

<div class="post-metadata">

### Author: ![Diogo\_Recharte](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/diogo_recharte/32/8004_2.png) [@Diogo\_Recharte](https://forums.percona.com/u/Diogo_Recharte)
#### Post date: [March 8, 2024, 10:03am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/7 "2024-03-08T10:03:57Z")

</div>

@just3tee I just remembered one other thing. We currently have a limitation when connecting to PMM. We enforce the TLS certificate validation so if your PMM server is using a self-signed certificate you can’t use https to connect to it (the certificate isn’t signed by a known entity so it’s rejected).  
The workaround for this is to use http instead https in the endpoint URL.

---

<div class="post-metadata">

### Author: ![just3tee](https://avatars.discourse-cdn.com/v4/letter/j/a183cd/32.png) [@just3tee](https://forums.percona.com/u/just3tee)
#### Post date: [March 8, 2024, 10:07am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/8 "2024-03-08T10:07:16Z")

</div>

![image](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/f/fb5db5b66b68e33e87a7733f327934da5e1f99af.png)  
K8S can reach the pmm server, I already monitoring some db instances run inside k8s cluster, I can see the metrics in the pmm dashboard. So, I think the step registration a monitoring instances have some issues, because, swagger defined a POST request include returned apiKey key but in fe of everest isn’t handle it?, and got the 500 status code, all my knowledge is the request body is not match with the server ☹ . Sorry if I lack of knowledge about this platform ☹

---

<div class="post-metadata">

### Author: ![just3tee](https://avatars.discourse-cdn.com/v4/letter/j/a183cd/32.png) [@just3tee](https://forums.percona.com/u/just3tee)
#### Post date: [March 8, 2024, 10:09am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/9 "2024-03-08T10:09:09Z")

</div>

It the cert from Let’s Encrypt, ZeroSSL valid? I signed the cert and key from 2 platform

 ![image](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/0/072a6faa30d96dc6a7ae7c2719b2d789af23336d.png)

---

<div class="post-metadata">

### Author: ![Diogo\_Recharte](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/diogo_recharte/32/8004_2.png) [@Diogo\_Recharte](https://forums.percona.com/u/Diogo_Recharte)
#### Post date: [March 8, 2024, 10:25am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/10 "2024-03-08T10:25:37Z")

</div>

Well, we fetch the trusted CAs from the standard alpine repos when [building the image](https://github.com/percona/everest/blob/bd5c11044d19e8e55a703f5de8436ee3d78b6fd2/Dockerfile#L7).  
I’m pretty sure that it should be part of chain signed by those trusted CAs 😕

Just to check if this is a certificate issue can you please try to add the PMM server again using the everest UI and show me the logs of the `percona-everest-xxxxx.....` pod in the `everest-system` namespace? If that’s the case, it should log something regarding not being able to verify the cert.

---

<div class="post-metadata">

### Author: ![just3tee](https://avatars.discourse-cdn.com/v4/letter/j/a183cd/32.png) [@just3tee](https://forums.percona.com/u/just3tee)
#### Post date: [March 8, 2024, 10:45am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/11 "2024-03-08T10:45:55Z")

</div>

I will check it and will response in the next week. Happy weekend!

---

<div class="post-metadata">

### Author: ![just3tee](https://avatars.discourse-cdn.com/v4/letter/j/a183cd/32.png) [@just3tee](https://forums.percona.com/u/just3tee)
#### Post date: [March 8, 2024, 3:29pm UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/12 "2024-03-08T15:29:53Z")

</div>

Hi @Diogo_Recharte  
Thank you for answering me many questions.I troubleshoot follow your suggestion and reinstall pmm-server with this tutorial [Automate the SSL Certificate Lifecycle of your Percona Monitoring and Management Server](https://www.percona.com/blog/automate-the-ssl-certificate-lifecycle-percona-monitoring-and-management-server/), finally this works. I new to K8S and Percona, so I am happy with the results I have gotten from your suports. Thank you very much!!! 😍

---

<div class="post-metadata">

### Author: ![daniil.bazhenov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/daniil.bazhenov/32/62_2.png) [@daniil.bazhenov](https://forums.percona.com/u/daniil.bazhenov)
#### Post date: [March 11, 2024, 6:30am UTC](https://forums.percona.com/t/percona-everest-v0-8-0-issue-with-pmm-and-certificate/29001/13 "2024-03-11T06:30:51Z")

</div>

Hi @just3tee

We could really use information from you on how you plan to use Percona Everest, how you use the API.

We would be very grateful if you could share a use case. You can do it privately in a private message to me.
