Connect to a pmm-server with authentication (is it a bug?!)

Percona Monitoring and Management (PMM) PMM 1.x
1

Hi Everyone,

I have created a pmm-server with:

docker run -d -p 85:80 --volumes-from pmm-data --name pmm-server --restart always -e SERVER_USER=someusername -e SERVER_PASSWORD=somepassword percona/pmm-server:1.0.7

So far so good - I can access it via web and it’s asking for the given credentials.

Now, when I want to set up monitoring, I do

pmm-admin config --server 127.0.0.1:85 --server-user someusername --server-password somepassword

I get a message:

Unable to connect to PMM server by address: 127.0.0.1:85

  • Check if the configured address is correct.
  • If server is running on non-default port, ensure it was specified along with the address.
  • If server is enabled for SSL or self-signe

I’ve tried to search the documentation and googling, but the --server-user and --server-password options are quite poorly documented.
What am I missing?

Client version is pmm-client-1.0.7-1.x86_64

Best,
Istvan Prosinger

2

Possible issues: firewall, selinux etc.
Do you run pmm-server on the host where your database run?
can you try to use IP address (not 127.0.0.1)?

3

Yes, I’ve just tried it, same issue. I must note that all this works flawlessly without a password.
[HR][/HR]
When I try to configure the client without a user/pass, it detects that the server is password protected and asks me to configure the user and pass, but bump:

[I][root@vt-api ~]# pmm-admin config --server 88.99.31.108:85
Unable to connect to PMM server by address: 88.99.31.108:85

Looks like the server is password protected.
Use ‘pmm-admin config’ to define server user and password.[/I]

Suma sumarum, this is what I get again:

[I]root@vt-api ~]# pmm-admin config --server 88.99.31.108:85 --server-user someuser --server-password somepassword
Unable to connect to PMM server by address: 88.99.31.108:85

  • Check if the configured address is correct.
  • If server is running on non-default port, ensure it was specified along with the address.
  • If server is enabled for SSL or self-signed SSL, enable the corresponding option.
  • You may also check the firewall settings[/I].

Thanks for your efforts!

Istvan

Edit:
There is not SeLinux (it’s disabled), and the firewall is open for this case - loging into the server works perfectly with password and user when I access it with the web browser.

4

First of all - I am a bit stressed, I double checked, I am sure, it is impossible to get password unprotected service by the following commands

docker create -v /opt/prometheus/data -v /opt/consul-data -v /var/lib/mysql -v /var/lib/grafana --name pmm-data percona/pmm-server:1.0.7 /bin/true
docker run -d -p 85:80 --volumes-from pmm-data --name pmm-server --restart always -e SERVER_USER=someusername -e SERVER_PASSWORD=somepassword percona/pmm-server:1.0.7

anyway,
Can you please check that pmm-admin version is completely the same to the PMM Server version? can you update it to 1.0.7?

5

What I meant here, is that it works flawlessly when I run the server from docker without a defined SERVER_USER and SERVER_PASSWORD. In that case the client connects to the server (without password) with no problem.

Docker says:
REPOSITORY TAG IMAGE ID CREATED SIZE
docker.io/percona/pmm-server 1.0.7 a91f4f6237a9 4 weeks ago 714.4 MB

Not sure how else to check this

6

I see, server version is 1.0.7

please run the following command on the database server

pmm-admin --version
7

Thanks for that, pmm-admin also says that it’s version 1.0.7

BTW, were you able to replicate this error? I think that the setup is quite straight forward…

8

I checked, but I cannot reproduce, situation looks very-very strange.

can you run the following command on the client


curl http://someusername:somepassword@88.99.31.108:85/prometheus/

expected output is

<a href="/prometheus/graph">Found</a>.
9

No, I got


<html>
<head><title>401 Authorization Required</title></head>
<body bgcolor="white">
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx</center>
</body>
</html>

Access to the server requires authorization in this setup, I find this normal

Edit:
Woops sorry, I’ve changed the credentials. Yes, I’ve got


<a href="/prometheus/graph">Found</a>.
10

Hi,

I cannot understand what is going, may you have some kind of proxy or something else.
During “config” command, pmm-admin makes simple http request to PMM server (to URL “http://someusername:somepassword@88.99.31.108:85/”).

I prepared the patch which can add additional debug information into “Unable to connect” error message - github link

What is your operation system?
I’ll prepare simple copy-paste instruction “How to build pmm-admin” for your OS version.

11

No, there is no proxy infront of it. That’s why I moved it to port 85, because I have Apache on port 80. As I can tell, it just created a NAT rule in iptables towards the container that still runs on port 80 in docker’s realm. I think everything is ok there.
I’d rather say that the problem is somewhere somewhere around


[I]pmm-admin config --server 88.99.31.108:85 --server-user someuser --server-password somepassword[/I]

not handling correctly the username and password given. BTW, is it a problem if I have a dash at the end of the password? Like “somepassword#”

I’m running it on CentOS 7. It’s wouldn’t be a problem for me to delete the container and make a new one, but this worked out of the box when I was doing it fro the Instructions provided by the site. The problems started when I have stopped the server and added

-e SERVER_USER=someusername -e SERVER_PASSWORD=somepassword

when running it, and then tryed to configure pmm-admin to use the user/pass.

Best, Istvan

12

Hi Istvan,

yes, issue is related to hash sign. issue will be fixed in the next release.
I prepared temporary fix for you - github link

you can build inside vagrant box on your desktop and copy binary to production or compile binary directly on production
start vagrant


vagrant init centos/7
vagrant up --provider virtualbox
vagrant ssh

how-to compile pmm-client


# install git and golang
yum -y install \
git \
https://repo.percona.com/laboratory/7/RPMS/x86_64/golang-1.7.3-2.el7.centos.x86_64.rpm \
https://repo.percona.com/laboratory/7/RPMS/x86_64/golang-bin-1.7.3-2.el7.centos.x86_64.rpm \
https://repo.percona.com/laboratory/7/RPMS/x86_64/golang-src-1.7.3-2.el7.centos.noarch.rpm

# prepare eng
mkdir /tmp/pmm-client
cd /tmp/pmm-client
export GOPATH=/tmp/pmm-client

# get fix
go get github.com/delgod/pmm-client
cd src/github.com/delgod/pmm-client
git checkout 41c6b196
cd /tmp/pmm-client
cp src/github.com/delgod/pmm-client/pmm/main.go \
src/github.com/percona/pmm-client/pmm/main.go

# build
cd src/github.com/percona/pmm-client
go build -o pmm-admin pmm-admin.go

# copy to production
scp pmm-admin REMOTE-SERVER:

# on remote server
sudo cp pmm-admin /sbin/pmm-admin
13

It works now. Thanks a lot for this.

Best,
Istvan