Hide Password in Percona Config

Percona Monitoring and Management (PMM) PMM 2.x
1

Sooooo Got Percona setup and it’s working great, BUUUUT there is an issue.

The main admin account password is in plain text in:
/usr/local/percona/pmm2/config/pmm-agent.yaml

Which is fine for systems where a controlled set of people are root.

BUT I host some servers for other people, and I only want them to have VIEW access to Percona via their user account.

How can I hide/obfuscate or otherwise eliminate this password being in there in plain text?

SSH Key? Something like that? or anything? :frowning:

Thanks,
Matt

1 Like
2

So I thought I had found a way around this with Orgs and Users,

So I created a new Org, made a new user, made the user an Admin of the new Org, but pmm-agent says invalid password when I update the config on that.

And when I switch to that Org all the dashboards say they cannot be found.

The node still belongs to the Primary Org, and I don’t see any way to move it to the new Org, so that the Admin access would apply.

Thanks,
Matt

1 Like
3

unfortunately orgs doesn’t work the way you (or I) would like them too…they’re sort of pseudo multi-tenancy but no real isolation of the data…just limiting what you’re exposed to…the downside is that you must install dashboards and graphs for each org and we do not do that by default (so you create a new ord, that’s an empty shell you can put dashboards that look at more (or less) filtered data compared to another org and you can limit peoples access/association to orgs…but not really security.

There is a way to hide that password and for the life of me I can’t find it…so I’m pinging a colleague but he’s 6 hrs ahead and should be enjoying family time so I scheduled the ping for first thing his morning…if he doesn’t post directly here i’ll post what he does to avoid users with (pmm-client) server access from reading that file and granting themselves pmm-server admin rights.

1 Like
4

Hi @Matt_Westfall

I am afraid that I’m not sure what Steve is thinking about to hide the password.

Which is fine for systems where a controlled set of people are root.

Are you allowing users to be root that shouldn’t have access to read the config? The file is written out as 0660 for the pmm-agent user and group, but if you are allowing users superuser access then you are going to be up against it in terms of securing the file.

However, there is no reason to be using the admin account in the first place. The user for the agent simply needs Grafana admin, which is not a full admin account and so PMM-specific admin features are not visible.

2022-06-09-1654761327

To improve on that further, you could create an API key, where API keys have a TTL and thus allows you to rotate tokens with automation, e.g. create a process that does the following:

  1. Create API key with TTL x
  2. Register/configure instance using the API key
  3. Schedule task at interval x - y to call process

N.B. the API keys need admin privileges, but you can’t login to the UI with them

To take that a step further, you could generate API keys per instance to limit the scope of its usage not only in time but also to be able to revoke a single instance’s access without the need to reconfigure all of the other instances.

For reference:

  • create the user via Server Admin → Users
  • create the API keys via Configuration → Users → API keys

Going back to your original concern about the visibility of the config file content, you could try the following (completely untested):

  • Override the systemd unit file to add ExecStartPre and ExecStartPost
  • ExecStartPre will read from a secure provider to retrieve the password and update the YAML
  • ExecStartPost will remove the password from the YAML
1 Like
5

Well these are customer machines, so they have root on their machines, but I want to put them in Percona. And I don’t care if they can see everything IN Percona, just not change and mess with things.

API Key should be perfect.

Do I just put the API Key in place of the password in the pmm-agent config file?

Thanks,
Matt

1 Like
6

Actually just using Grafana admin should work fine, because they would obviously know their own password to login and view anyway :smiley:

1 Like
7

So, the API keys are used as follows… whichever way that you are doing things:

managing the config file

server:
  username: api_key
  password: <insert the base64-encoded key that is generated>

pmm-admin

pmm-admin config --server-url=https://api_key@<insert the base64-encoded key that is generated>@<server ip>:<server port>/

pmm-agent

pmm-agent setup ... --server-username=api_key --server-password=<insert the base64-encoded key that is generated> ...

N.B. the API keys need admin privileges, but you can’t login to the UI with them

However, as these do require the admin role, they could be used with the API to change things.

Actually just using Grafana admin should work fine, because they would obviously know their own password to login and view anyway :smiley:

:+1:

1 Like