Pmm-admin config gives 500 error, PMM logs show 403 for 127.0.0.1:3000/api/auth/keys

Percona Monitoring and Management (PMM) PMM 2.x
1

Hi,

I’m getting the following error trying to register a PMM node:

[root@db1 ~]# pmm-admin config --server-url=https://admin:$PASS@pmm.internal.host --force --debug

Checking local pmm-agent status...
pmm-agent is running.
Registering pmm-agent on PMM Server...
Failed to register pmm-agent on PMM Server: Internal server error..

And in the pmm-managed.log:

time="2023-10-18T15:15:21.911+00:00" level=info msg="Starting RPC /server.Server/Readiness ..." request=27d3d77b-6dc9-11ee-99cb-0242ac110003
time="2023-10-18T15:15:21.919+00:00" level=info msg="RPC /server.Server/Readiness done in 8.192346ms." request=27d3d77b-6dc9-11ee-99cb-0242ac110003
time="2023-10-18T15:15:24.256+00:00" level=info msg="Starting RPC /management.Node/RegisterNode ..." request=XXXXXXX
time="2023-10-18T15:15:24.313+00:00" level=error msg="RPC /management.Node/RegisterNode done in 56.83741ms with unexpected error: clientError: POST http://127.0.0.1:3000/api/auth/keys -> 403 {\"accessErrorId\":\"ACE3156772839\",\"message\":\"You'll need additional permissions to perform this action. Permissions needed: apikeys:create\",\"title\":\"Access denied\"}\n\ngithub.com/percona/pmm/managed/services/grafana.(*Client).do\n\t/home/builder/rpm/BUILD/pmm-XXXXX/src/github.com/percona/pmm/managed/services/grafana/client.go:149\ngithub.com/percona/pmm/managed/services/grafana.

Can anyone shed some light on what the problem might be?

[root@db1 ~]# pmm-admin  -v
ProjectName: pmm-admin
Version: 2.40.0
PMMVersion: 2.40.0
Timestamp: 2023-09-28 22:11:09 (UTC)
FullCommit: 29fbba7984b26fb68ecfe65e44c1c6b761c14e28

Version for the PMM server is the same.

2

Hi @jdlawrie .

Please confirm if the user you’re using an ADMIN user.
You can confirm if the user is an ADMIN user in the Server Admin / Users in PMM GUI.

Best,
Mauricio.

3

Thank you Mauricio, the admin user had somehow been set to a viewer