PMM Error: "context deadline exceeded". Need to update Scrape_timeout.

MySQL & MariaDB Other MySQL® Questions
1

I have a PMM docker container deployed and I’m trying to monitor Ubuntu server. On my prometheus/targets page, I keep getting the error: Context Deadline Exceeded. I would really appreciate any help.

I read in prometheus forums that I might need to change the “scrape_timeout” variable on prometheus. This is being set by the /etc/prometheus.yml file.
How would I edit the scrape_timeout variable?

2

I have same issue and would appreciate if a solution could be posted.

3

My issue is still present for all the MySql 8 database instances in my farm.

Could someone address this request?

4

Hi,

My installation of PMM is monitoring a mix of MySql 5.5 to MySql 8.0.11 instances.

For all the instances that are not MySql 8.0.11 I have full functionality of PMM.

For only the MySql 8.0.11 instances (6 of them). I only see the OS (LINUX), and Query Analytics data. Nothing for the MySql Overview.

In Prometheus I see for the servers
[TABLE]
[TR]
[TD][URL]https://192.168.72.192:42002/metrics-hr[/URL][/TD]
[TD]DOWN[/TD]
[TD]instance=“mysql_lcormysqld01”[/TD]
[TD]888ms ago[/TD]
[TD]no token found[/TD]
[/TR]
[/TABLE]

When running

[baasp@lcormysqlp01 ~]$ sudo pmm-admin check-network –-no-emoji
PMM Network Status

Server Address | XXX.XXX.XXX.XXX
Client Address | XXX.XXX.XXX.XXX

  • System Time
    NTP Server (0.pool.ntp.org) | unable to get ntp time: %!s()
    PMM Server | 2018-12-13 16:24:53 +0000 GMT
    PMM Client | 2018-12-13 11:29:25 -0500 EST
    PMM Client to PMM Server Time Drift | 272s
    Time is out of sync. Please make sure the server time is correct to see the metrics.

  • Connection: Client → Server

SERVER SERVICE STATUS

Consul API OK
Prometheus API OK
Query Analytics API OK

Connection duration | 1.501739ms
Request duration | -739.678µs
Full round trip | 762.061µs

  • Connection: Client ← Server

SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD

linux:metrics linux_lcormysqlp01 XXX.XXX.XXX.XXX:42000 OK YES YES
mysql:metrics mysql_lcormysqlp01 XXX.XXX.XXX.XXX:42002 DOWN YES YES

When an endpoint is down it may indicate that the corresponding service is stopped (run ‘pmm-admin list’ to verify).
If it’s running, check out the logs /var/log/pmm-*.log

When all endpoints are down but ‘pmm-admin list’ shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.

Also you can check the endpoint status by the URL: [url]http://192.168.145.100/prometheus/targets[/url]

How do I fix this?

Thanks

Peter

5

I get the same thing “context deadline exceeded”. Is there any solution to this yet? This pmm server looks great but with no data who cares!!!

when i issue my pmm-admin list, all 3 services are ok. when I issue the pmm-admin check-network, the linux:metrics and the mysql:metrics are down, however I can telnet from my pmm-server to the client on port 42000 and 42002. makes no sense.

logs have nothing. If I goto the prometheus page: [url]https://myhost/prometheus/targets[/url], I can hit all links that are showing context deadline exceeded

where are you percona??? need help.

6

You should really raise a new post. We’re not reviewing posts from 2018 on a regular basis, and even if there was a fix available then, or specific advice to be offered, this could be a different issue arising on a later version of software and in a different environment.

There’s a post here that explains the kind of information that we need. [URL]https://www.percona.com/forums/questions-discussions/percona-monitoring-and-management/50690-pmm-troubleshooting-and-how-to-report-a-bug[/URL]

For the best attention, you should put this question in the PMM forum channel, as it’s a PMM issue not a Percona Server issue. This is very likely to be why it has been overlooked in the past.

Please do these things, provide full information (versions etc) and then I will try to get someone to take a look at the post. I will watch out for your new post to appear.