# PMM Client Upgrade Issues 2.x to 3.x **URL:** \ **Category:** Percona Monitoring and Management (PMM)\ **Created:** [June 18, 2026, 5:21pm UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933 "2026-06-18T17:21:10Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![dclark](https://avatars.discourse-cdn.com/v4/letter/d/b782af/32.png) [@dclark](https://forums.percona.com/u/dclark)\ **Post date:** [June 18, 2026, 5:21pm UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933/1 "2026-06-18T17:21:10Z")
## Description: I have recently upgraded our kubernetes mongodb deployment to use PMM 3.8.1. I have 4 deployments in our NP environment that all talk back to the same PMM install. The PMM migration went well. When I went to upgrade the PMM client for the first mongodb deployment, the pmm-client kept crashing. Upon further investigation I found that the operator appears to be injecting the wrong values into the stateful sets. Specifically PMM\_AGENT\_CONFIG\_FILE is being pushed as /usr/local/percona/pmm2/config/pmm-agent.yaml instead of /usr/local/percona/pmm/config/pmm-agent.yaml. The helm values for the mongodb deployment inject PMM\_API\_KEY, PMM\_ADMIN\_USER and PMM\_ADMIN\_PASSWORD as part of the terragrunt build. I have read that because of support for pmm2 and pmm3 together this may be what is causing the issue. Looking through the operator code there is this piece. secret.Data[api.PMMServerToken] However, I do not see that as an option in the helm value examples for mongo at [percona-helm-charts/charts/psmdb-db/values.yaml at main · percona/percona-helm-charts · GitHub](https://github.com/percona/percona-helm-charts/blob/main/charts/psmdb-db/values.yaml) . If I edit the stateful set and change the PMM\_AGENT\_CONFIG\_FILE to /usr/local/percona/pmm/config/pmm-agent.yaml and scale up the set, it connects correctly. I am sure I am missing something simple but can’t seem to find docs around how exactly to get the new pmm-agents to use API tokens instead of Keys which I am assuming is why the operator is thinking PMM2 when I really want PMM3 and is then injecting the pmm2 path into the stateful sets PMM\_AGENT\_CONFIG\_FILE variable. **Operator** Chart: psmdb-operator-1.21.3 APP VERSION: 1.21.2 **PSMDB** crVersion: 1.21.2 Chart: 1.21.2 pmm-agent: 3.5 but tried a few **PMM** Chart 1.8.1 Any thoughts or advice would be greatly appreciated. Thank you, ## ---
**Author:** ![dclark](https://avatars.discourse-cdn.com/v4/letter/d/b782af/32.png) [@dclark](https://forums.percona.com/u/dclark)\ **Post date:** [June 18, 2026, 5:45pm UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933/2 "2026-06-18T17:45:44Z")
Ok, I need to do some testing but think I figured it out. If someone can just confirm it would be great. It looks like if the value of PMM\_SERVER\_API\_KEY is an api key, then it assumes pmm2, if it is an API token it assumes pmm3. I tested by manually updating the secret to a new token, let the operator recycle the containers and then ran kubectl get sts -n -o yaml | grep -A5 -B5 PMM\_AGENT\_CONFIG\_FILE, this resulted in; name: PMM\_AGENT\_CONFIG\_FILE value: /usr/local/percona/pmm/config/pmm-agent.yaml and not name: PMM\_AGENT\_CONFIG\_FILE value: /usr/local/percona/pmm2/config/pmm-agent.yaml ---
**Author:** ![dclark](https://avatars.discourse-cdn.com/v4/letter/d/b782af/32.png) [@dclark](https://forums.percona.com/u/dclark)\ **Post date:** [June 19, 2026, 12:28pm UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933/3 "2026-06-19T12:28:45Z")
So did another environment and think that did not work. It did not automatically change from pmm2 to pmm. Is there something that needs to be passed to ensure the use of pmm3? I don’t like the idea of needing to edit the stateful sets to change PMM\_AGENT\_CONFIG\_FILE from /usr/local/percona/pmm2/config/pmm-agent.yaml to /usr/local/percona/pmm/config/pmm-agent.yaml. ---
**Author:** ![dclark](https://avatars.discourse-cdn.com/v4/letter/d/b782af/32.png) [@dclark](https://forums.percona.com/u/dclark)\ **Post date:** [June 19, 2026, 7:15pm UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933/4 "2026-06-19T19:15:46Z")
Ok, pretty sure I got it now. use PMM\_SERVER\_TOKEN instead of PMM\_SERVER\_API\_KEY. Am I just completely missing something in the docs or is this not well documented? ---
**Author:** ![Yunus](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/yunus/32/6430_2.png) [@Yunus](https://forums.percona.com/u/Yunus)\ **Post date:** [June 20, 2026, 6:02am UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933/5 "2026-06-20T06:02:08Z")
Right, PMM 3 will use service account token > **[Automatic migration of API keys - Percona Monitoring and Management - Service...](https://docs.percona.com/percona-monitoring-and-management/3/api/authentication.html#automatic-migration-of-api-keys)** > > Deprecation notice | Documentation Hope that helps ---
**Author:** ![dclark](https://avatars.discourse-cdn.com/v4/letter/d/b782af/32.png) [@dclark](https://forums.percona.com/u/dclark)\ **Post date:** [June 22, 2026, 12:23pm UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933/6 "2026-06-22T12:23:32Z")
That’s great and all, but no where in that does it mention that the helm values need to change. That you need to change **PMM\_SERVER\_API\_KEY** to **PMM\_SERVER\_TOKEN** for it to assume pmm 3.0. **PMM\_SERVER\_TOKEN** is not even in the helm values file in git at [percona-helm-charts/charts/psmdb-db/values.yaml at main · percona/percona-helm-charts · GitHub](https://github.com/percona/percona-helm-charts/blob/main/charts/psmdb-db/values.yaml) . It wasn’t until I did some testing and completely removed all PMM related helm values that I was able to determine the issue because the operator started throwing an error message; 2026-06-19T19:05:51.440Z ERROR secret is missing the required PMM credentials {“controller”: “psmdb-controller”, “controllerGroup”: “[psmdb.percona.com](http://psmdb.percona.com)”, “controllerKind” : “PerconaServerMongoDB”, “PerconaServerMongoDB”: {“name”:“xxx”,“namespace”:“xxx”}, “namespace”: “xxx”, “name”: “xxx”, “reconcileID”: “d970cbac-2eb1-4cb7-978b-160ef23e02c5”, “error”: “PMM is enabled and requires the configuration of either PMM\_SERVER\_TOKEN for PMM3 or PMM\_SERVER\_API\_KEY for PMM2”} ---
**Author:** ![Denis\_Subbota](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/denis_subbota/32/23607_2.png) [@Denis\_Subbota](https://forums.percona.com/u/Denis_Subbota)\ **Post date:** [June 24, 2026, 10:12am UTC](https://forums.percona.com/t/pmm-client-upgrade-issues-2-x-to-3-x/40933/7 "2026-06-24T10:12:46Z")
Hey dclark, you nailed it - and you found a real gap, so it’s not just you. The operator goes by which key is set, not the value: `PMM_SERVER_API_KEY` → it assumes PMM2 (old `/pmm2/` path), `PMM_SERVER_TOKEN` → PMM3 (`/pmm/` path). That’s what your error is saying too. So once the key is `PMM_SERVER_TOKEN`, no more hand-editing the StatefulSet. It’s also why your earlier test was flaky — you changed the value but kept the `PMM_SERVER_API_KEY` key. And yep, the chart only shows the commented `PMM_SERVER_API_KEY` - `PMM_SERVER_TOKEN` isn’t in there at all. This needs a bug so the docs and chart get fixed. Mind opening one in Percona’s Jira? Thanks for the great write-up!