Mykola, how can i “disable query examples”?
- Open “Query Analytics” interface
- choose database host
- click on SETTINGS button
- disable (uncheck) “Send real query examples” flag
- set “Collect from” field to “Performance Schema”
- click apply button
please notify me if any bugs still exists
You were right, it work fine with disabled “Send real query examples”
Is there some way to disable “Send real query examples” from command line? I just use ansible to add hosts to PMM, so i need to add it without GUI.
currently it is imposible to add node with disabled “Send real query examples” option.
but, I added such functionality as experimental feature (it is not reviewed and not tested) - [url]https://github.com/percona/pmm-client/pull/24[/url]
if you want you can compile it and test
sudo yum install golang git
mkdir /tmp/pmm-admin
cd /tmp/pmm-admin
git clone -b query-examples https://github.com/percona/pmm-client src/github.com/percona/pmm-client
export GOPATH=$(pwd)
go build ./src/github.com/percona/pmm-client/
sudo mv pmm-client /usr/sbin/pmm-admin
sudo pmm-admin remove mysql
sudo pmm-admin add mysql --disable-queryexamples
anyway it is workaround, we are searching root of the issue.
Hi Alexksey,
after all, I created the issue [url][PMM-589] capturing query examples from perf schema freezes the agent - Percona JIRA
feel free to add any additional information to jira ticket.
Ok, thanks.
Adventure does not end. Everything works fine, until slow_query_log disabled. In spite of the fact usage of performance_schema. Same problem, bad connection driver.
sorry, can you clarify one more time.
when slow_query_log=OFF, performance_schema=ON, query_examples=OFF, Do you have any issues?
Right now “slow_query_log=OFF, performance_schema=ON, query_examples=OFF”.
Problem exist.
A lot of WARNING in pmm-mysql-queries-0.log:
2017/02/17 08:51:00.001570 WARNING qan-analyzer-f122d8f8-worker Interval out of sequence: got 85, expected 81
2017/02/17 08:52:00.001470 WARNING qan-analyzer-f122d8f8 Skipping interval ‘86 2017-02-17 06:51:00 UTC to 2017-02-17 06:52:00 UTC (0-0)’ because interval ‘85 2017-02-17 06:50:00 UTC to 2017-02-17 06:51:00 UTC (0-0)’ is still being parsed
2017/02/17 08:53:00.001461 WARNING qan-analyzer-f122d8f8 Skipping interval ‘87 2017-02-17 06:52:00 UTC to 2017-02-17 06:53:00 UTC (0-0)’ because interval ‘85 2017-02-17 06:50:00 UTC to 2017-02-17 06:51:00 UTC (0-0)’ is still being parsed
2017/02/17 08:54:00.001591 WARNING qan-analyzer-f122d8f8 Skipping interval ‘88 2017-02-17 06:53:00 UTC to 2017-02-17 06:54:00 UTC (0-0)’ because interval ‘85 2017-02-17 06:50:00 UTC to 2017-02-17 06:51:00 UTC (0-0)’ is still being parsed
2017/02/17 08:55:00.001422 WARNING qan-analyzer-f122d8f8 Skipping interval ‘89 2017-02-17 06:54:00 UTC to 2017-02-17 06:55:00 UTC (0-0)’ because interval ‘85 2017-02-17 06:50:00 UTC to 2017-02-17 06:51:00 UTC (0-0)’ is still being parsed
2017/02/17 08:56:00.001660 WARNING qan-analyzer-f122d8f8-worker Interval out of sequence: got 90, expected 86
2017/02/17 08:57:00.001457 WARNING qan-analyzer-f122d8f8 Skipping interval ‘91 2017-02-17 06:56:00 UTC to 2017-02-17 06:57:00 UTC (0-0)’ because interval ‘90 2017-02-17 06:55:00 UTC to 2017-02-17 06:56:00 UTC (0-0)’ is still being parsed
After manipulation on settings page - switch to slowlog, switch to performance_schema, set and unset Send real query examples QAN start to work.
these warnings just warnings and shouldn’t affect qan-api functionality.
If i need to reinstall pmm-client, how do i have to clear all settings, caches, tampfiles etc after removing and before installing? Is there some place where all of this stored?
before reinstall please type pmm-admin uninstall
on client side.
Ok, i removed, installed and configured client again, will see. Can i write here problems with grafana, or create other topic?
I think new topic is better
Well, for now QAN usable only with slow_query_log, not with performance_schema. Tested on 1.1.1 version.
what issues do you have with “performance_schema=ON, query_examples=OFF” ?
2017/02/21 10:18:23.795575 main.go:207: API is ready
2017/02/21 11:01:31.792736 WARNING data-sender Waiting for API to ack qan_1487667661266859145: read tcp 192.168.15.9:45680->192.168.15.19:82: i/o timeout
2017/02/21 11:05:00.003461 WARNING qan-analyzer-04bce813-worker Interval out of sequence: got 1, expected 47
2017/02/21 11:06:00.004050 WARNING qan-analyzer-04bce813 Skipping interval ‘2 2017-02-21 09:05:00 UTC to 2017-02-21 09:06:00 UTC (0-0)’ because interval ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:05:00 UTC (0-0)’ is still being parsed
2017/02/21 11:07:00.001403 WARNING qan-analyzer-04bce813 Skipping interval ‘3 2017-02-21 09:06:00 UTC to 2017-02-21 09:07:00 UTC (0-0)’ because interval ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:05:00 UTC (0-0)’ is still being parsed
2017/02/21 11:08:00.018888 WARNING qan-analyzer-04bce813 Skipping interval ‘4 2017-02-21 09:07:00 UTC to 2017-02-21 09:08:00 UTC (0-0)’ because interval ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:05:00 UTC (0-0)’ is still being parsed
2017/02/21 11:09:00.001833 WARNING qan-analyzer-04bce813 Skipping interval ‘5 2017-02-21 09:08:00 UTC to 2017-02-21 09:09:00 UTC (0-0)’ because interval ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:05:00 UTC (0-0)’ is still being parsed
2017/02/21 11:09:23.213428 analyzer.go:426: qan-analyzer-04bce813-worker crashed: ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:05:00 UTC (0-0)’: runtime error: invalid memory address or nil pointer dereference
goroutine 498 [running]:
runtime/debug.Stack(0x4868ec, 0xc42000e0f0, 0x2)
/usr/local/go1.7.4/src/runtime/debug/stack.go:24 +0x79
runtime/debug.PrintStack()
/usr/local/go1.7.4/src/runtime/debug/stack.go:16 +0x22
github.com/percona/qan-agent/qan.(*RealAnalyzer).runWorker.func1(0xc42016cc60, 0xc420a56ea0)
/mnt/workspace/pmm-client-tarball/pmm-client-1.1.1/src/github.com/percona/qan-agent/qan/analyzer.go:427 +0x1f6
panic(0x7176c0, 0xc42000c070)
/usr/local/go1.7.4/src/runtime/panic.go:458 +0x243
database/sql.(*DB).conn(0x0, 0x7f0aae2bc001, 0x0, 0x1100000000, 0xc420045970)
/usr/local/go1.7.4/src/database/sql/sql.go:781 +0x3a
database/sql.(*DB).query(0x0, 0xc420d36c60, 0xa2, 0x0, 0x0, 0x0, 0x1, 0x10, 0x6fe4c0, 0xc420d36c60)
/usr/local/go1.7.4/src/database/sql/sql.go:1074 +0x3b
database/sql.(*DB).Query(0x0, 0xc420d36c60, 0xa2, 0x0, 0x0, 0x0, 0xa2, 0xc4200459c8, 0x4142f8)
/usr/local/go1.7.4/src/database/sql/sql.go:1062 +0x90
database/sql.(*DB).QueryRow(0x0, 0xc420d36c60, 0xa2, 0x0, 0x0, 0x0, 0xa2)
/usr/local/go1.7.4/src/database/sql/sql.go:1143 +0x70
github.com/percona/qan-agent/qan/perfschema.GetDigestText(0x8dca20, 0xc42021b860, 0xc4216d0080, 0x20, 0xc4216d0090, 0x10, 0xc4215ac950, 0x10)
/mnt/workspace/pmm-client-tarball/pmm-client-1.1.1/src/github.com/percona/qan-agent/qan/perfschema/worker.go:188 +0x164
github.com/percona/qan-agent/qan/perfschema.(*RealWorkerFactory).Make.func2(0xc4216d0080, 0x20, 0xc4219cdc80, 0x10, 0x9140e0, 0x0)
/mnt/workspace/pmm-client-tarball/pmm-client-1.1.1/src/github.com/percona/qan-agent/qan/perfschema/worker.go:107 +0x47
github.com/percona/qan-agent/qan/perfschema.(*Worker).getSnapshot(0xc4200e8f00, 0xc4208e31a0, 0x0, 0x0, 0x0)
/mnt/workspace/pmm-client-tarball/pmm-client-1.1.1/src/github.com/percona/qan-agent/qan/perfschema/worker.go:430 +0x92f
github.com/percona/qan-agent/qan/perfschema.(*Worker).Run(0xc4200e8f00, 0x0, 0x0, 0x0)
/mnt/workspace/pmm-client-tarball/pmm-client-1.1.1/src/github.com/percona/qan-agent/qan/perfschema/worker.go:276 +0x38a
github.com/percona/qan-agent/qan.(*RealAnalyzer).runWorker(0xc42016cc60, 0xc420a56ea0)
/mnt/workspace/pmm-client-tarball/pmm-client-1.1.1/src/github.com/percona/qan-agent/qan/analyzer.go:451 +0x305
created by github.com/percona/qan-agent/qan.(*RealAnalyzer).run
/mnt/workspace/pmm-client-tarball/pmm-client-1.1.1/src/github.com/percona/qan-agent/qan/analyzer.go:365 +0x832
2017/02/21 11:09:23.213692 ERROR qan-analyzer-04bce813 qan-analyzer-04bce813-worker crashed: ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:05:00 UTC (0-0)’: runtime error: invalid memory address or nil pointer dereference
2017/02/21 11:10:00.001421 WARNING qan-analyzer-04bce813-worker Interval out of sequence: got 1, expected 2
2017/02/21 11:11:00.001888 WARNING qan-analyzer-04bce813 Skipping interval ‘2 2017-02-21 09:10:00 UTC to 2017-02-21 09:11:00 UTC (0-0)’ because interval ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:10:00 UTC (0-0)’ is still being parsed
2017/02/21 11:12:00.001400 WARNING qan-analyzer-04bce813 Skipping interval ‘3 2017-02-21 09:11:00 UTC to 2017-02-21 09:12:00 UTC (0-0)’ because interval ‘1 0001-01-01 00:00:00 UTC to 2017-02-21 09:10:00 UTC (0-0)’ is still being parsed
Other error, but same result - it does not work with performance_schema for a long time. Less then 1 hour. But it work fine with slow query log.
Buy the way, when you advice me in first time switching to performance schema and disabling query sampling, i forgot to disable slow query log in database. And it works. But after disabling slow query log, without changing configuration in QAN, it stop working in 1 hour or so.