# PMM Server not receiving pod resource data from in kubernetes cluster

**URL:** <https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656>\
**Category:** PMM 2.x\
**Tags:** pmm, mysql, kubernetes\
**Created:** [November 18, 2022, 1:24pm UTC](https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656 "2022-11-18T13:24:28Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![SCHrodrigo](https://avatars.discourse-cdn.com/v4/letter/s/3ab097/32.png) [@SCHrodrigo](https://forums.percona.com/u/SCHrodrigo)\
**Post date:** [November 18, 2022, 1:24pm UTC](https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656/1 "2022-11-18T13:24:28Z")

</div>

Hi,

My team has been trying to implement PMM Server to monitor our Percona XtraDB Clusters through PMM Clients. Most of the data about the MySQL instances are being displayed but not the CPU, RAM, disk and network information on the main dashboard.

 ![Main PMM Server Dashboard with missing data from PMM Clients](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/0/02eb7069096fa17073dd1ff3ee83af094747c219.png)

Meanwhile, on the dashboard template MySQL/MySQL Instance Summary, all of the graphs have data, except Node Summary.

 ![MySQL Instace Summary with no Node Summary data](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/d/deac306f7314c8d86d1f4c2d83027c391aa5aa0a.png)

PMM Server image: percona/pmm-server:2.31.0  
PMM Clients image: percona/pmm-client:2.28.0  
K8s: v1.22.14-gke.300

Is this a RBAC problem, or a YAML misconfiguration? We are using Helm to install the server, and the generated templates to install the clusters. A couple of changes we did with the templates are the version set to 1.10.0 and the containerSecurityContext for the pmm-client. The last one was required for Kubernetes to recognize the user as nonRoot.

Here are the files used for both Server and Clusters:  
[pmm-values.yaml.pdf](https://forums.percona.com/uploads/short-url/9BOZBuDnMcd87EXOJSkFHPtoxza.pdf) (63.0 KB)  
[pxc-db-chart.yaml.pdf](https://forums.percona.com/uploads/short-url/oby791HnCuZOvpl90U8vJBYmxXu.pdf) (75.2 KB)

---

<div class="post-metadata">

**Author:** ![SCHrodrigo](https://avatars.discourse-cdn.com/v4/letter/s/3ab097/32.png) [@SCHrodrigo](https://forums.percona.com/u/SCHrodrigo)\
**Post date:** [November 18, 2022, 2:24pm UTC](https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656/2 "2022-11-18T14:24:17Z")

</div>

I’m not sure if it is related, but there is an error popping up from the pmm-client:

 ![image](https://us1.discourse-cdn.com/flex019/uploads/percona1/original/2X/8/8b686634074f815621bf29b0bbf58842a13b3c2c.png)

The serverUser and secrets.passwords were not altered.

---

<div class="post-metadata">

**Author:** ![steve.hoffman](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/steve.hoffman/32/66_2.png) [@steve.hoffman](https://forums.percona.com/u/steve.hoffman)\
**Post date:** [November 18, 2022, 6:37pm UTC](https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656/3 "2022-11-18T18:37:10Z")

</div>

I have to phone a friend here…K8s is my kryptonite!

Here’s what I do know:  
In a typical configuration, all the data you’re missing comes from Node Exporter…so when I see that blank but database stats are flowing that means that the instance was added as a “remote instance” (where the PMM server is reaching out to the DB service port and scraping what data it can from port 3306 in mysql’s case) but since there’s not an external service port available for node stats, they’re just blank.

I just browsed through the release notes and Helm support wasn’t officially added until 2.29.0 so you might first try to upgrade your client image to at least 2.29.0 but for maximum compatibility I’d go right to 2.31.0 to match your PMM server.

I know that there was some changes made (or to be made) about the way PMM handles K8s monitoring since the node\_exporter’s original premise was monitoring the host that a DB is running on but you need more levels of data (the pods available resources and consumption, the nodes available resource and consumption and the clusters available resources and consumption) that node\_exporter wasn’t able to look at (and may still not be able to…not quite sure).

I’ll put this post in front of our K8s experts to see if they have ideas and/or suggestions on where to start looking.

---

<div class="post-metadata">

**Author:** ![Denys\_Kondratenko](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/denys_kondratenko/32/2798_2.png) [@Denys\_Kondratenko](https://forums.percona.com/u/Denys_Kondratenko)\
**Post date:** [November 21, 2022, 2:24pm UTC](https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656/4 "2022-11-21T14:24:01Z")

</div>

Hi @SCHrodrigo ,

did you auth that pmm client as described in PXC doc?

> **[Monitor with Percona Monitoring and Management (PMM) - Percona Operator for...](https://docs.percona.com/percona-operator-for-mysql/pxc/monitoring.html)**
>
> Documentation

Operator needs to know PMM auth.  
As client is not connected - there are no metrics.

In PMM DBaaS when you create DB there - it configures everything for you.

If you add that k8s cluster to the PMM DBaaS - PMM would install VM operator and would start to gather cAdvisor metrics as well. From 2.32 kube-state metrics as well.

Thanks,  
Denys

---

<div class="post-metadata">

**Author:** ![SCHrodrigo](https://avatars.discourse-cdn.com/v4/letter/s/3ab097/32.png) [@SCHrodrigo](https://forums.percona.com/u/SCHrodrigo)\
**Post date:** [November 22, 2022, 11:25am UTC](https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656/5 "2022-11-22T11:25:02Z")

</div>

@Denys_Kondratenko and @steve.hoffman, thank you for answering!

We combined both of the informations, we set the versions to be the latest and changed the authorization to be token-based. Since then, the registration error stopped popping up, and the metrics started rolling in.

Thank you!

---

<div class="post-metadata">

**Author:** ![Denys\_Kondratenko](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/denys_kondratenko/32/2798_2.png) [@Denys\_Kondratenko](https://forums.percona.com/u/Denys_Kondratenko)\
**Post date:** [November 22, 2022, 11:38am UTC](https://forums.percona.com/t/pmm-server-not-receiving-pod-resource-data-from-in-kubernetes-cluster/18656/6 "2022-11-22T11:38:12Z")

</div>

nice! Thanks you for the update!
