# Multiple Errors from PMM in the MariaDB sql error log file

**URL:** <https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323>\
**Category:** Percona Monitoring and Management (PMM)\
**Created:** [August 9, 2023, 4:17am UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323 "2023-08-09T04:17:43Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Naresh9999](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/naresh9999/32/5568_2.png) [@Naresh9999](https://forums.percona.com/u/Naresh9999)\
**Post date:** [August 9, 2023, 4:17am UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/1 "2023-08-09T04:17:43Z")

</div>

Below are errors flooding our SQL error log file.

Source MariaDB: 10.4

2023-08-08 19:29:52 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1146: Table ‘heartbeat.heartbeat’ doesn’t exist : SELECT UNIX\_TIMESTAMP(ts), UNIX\_TIMESTAMP(NOW(6)), server\_id from `heartbeat`.`heartbeat`  
2023-08-08 19:29:52 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1286: Unknown storage engine ‘TOKUDB’ : SHOW ENGINE TOKUDB STATUS  
2023-08-08 19:29:54 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1193: Unknown system variable ‘slow\_query\_log\_always\_write\_time’ : SELECT /\* pmm-agent:slowlog _/ @@slow\_query\_log\_always\_write\_time  
2023-08-08 19:30:00 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_READ’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_READ  
2023-08-08 19:30:00 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_WRITE’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_WRITE  
2023-08-08 19:30:04 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1193: Unknown system variable ‘slow\_query\_log\_always\_write\_time’ : SELECT /_ pmm-agent:slowlog _/ @@slow\_query\_log\_always\_write\_time  
2023-08-08 19:30:10 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_READ’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_READ  
2023-08-08 19:30:10 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_WRITE’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_WRITE  
2023-08-08 19:30:14 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1193: Unknown system variable ‘slow\_query\_log\_always\_write\_time’ : SELECT /_ pmm-agent:slowlog _/ @@slow\_query\_log\_always\_write\_time  
2023-08-08 19:30:20 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_READ’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_READ  
2023-08-08 19:30:20 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_WRITE’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_WRITE  
2023-08-08 19:30:24 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1193: Unknown system variable ‘slow\_query\_log\_always\_write\_time’ : SELECT /_ pmm-agent:slowlog _/ @@slow\_query\_log\_always\_write\_time  
2023-08-08 19:30:30 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_READ’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_READ  
2023-08-08 19:30:30 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_WRITE’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_WRITE  
2023-08-08 19:30:34 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1193: Unknown system variable ‘slow\_query\_log\_always\_write\_time’ : SELECT /_ pmm-agent:slowlog _/ @@slow\_query\_log\_always\_write\_time  
2023-08-08 19:30:40 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_READ’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_READ  
2023-08-08 19:30:40 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_WRITE’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_WRITE  
2023-08-08 19:30:44 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1193: Unknown system variable ‘slow\_query\_log\_always\_write\_time’ : SELECT /_ pmm-agent:slowlog \*/ @@slow\_query\_log\_always\_write\_time  
2023-08-08 19:30:50 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_READ’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_READ  
2023-08-08 19:30:50 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1109: Unknown table ‘QUERY\_RESPONSE\_TIME\_WRITE’ in information\_schema : SELECT TIME, COUNT, TOTAL FROM INFORMATION\_SCHEMA.QUERY\_RESPONSE\_TIME\_WRITE

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [August 9, 2023, 4:09pm UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/2 "2023-08-09T16:09:44Z")

</div>

It looks like the ‘SQL error log file’ is a plugin unique to MariaDB which logs SQL errors. This plugin, unfortunately, has no filtering capabilities.

The SQL errors you see are coming from PMM, attempting to query basic information about your system such as query response times, any tokudb engine stats, and replication heartbeat. There are also some typical system variables that MariaDB doesn’t appear to support.

I will pass this info along to our PMM team for a possible feature request to not run these types of queries against MariaDB. Until then, I’m afraid these queries will continue to execute.

---

<div class="post-metadata">

**Author:** ![Naresh9999](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/naresh9999/32/5568_2.png) [@Naresh9999](https://forums.percona.com/u/Naresh9999)\
**Post date:** [August 9, 2023, 5:55pm UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/3 "2023-08-09T17:55:03Z")

</div>

Hi @matthewb

Thanks for the update.

They are consistently writing to the sql error log, thus it would be better if we could disable this option.

I can see the following error in the slow query log. Could you also look at these slow log related errors?

\*_2023-08-08 19:30:44 pmm\_user[pmm\_user] @ [192.168.1.1] ERROR 1193: Unknown system variable ‘slow\_query\_log\_always\_write\_time’ : SELECT / pmm-agent:slowlog /_

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [August 9, 2023, 6:07pm UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/4 "2023-08-09T18:07:57Z")

</div>

> [@Naresh9999](#):
>
> They are consistently writing to the sql error log, thus it would be better if we could disable this option.

Unfortunately, disabling isn’t an option in PMM as of today. This is a feature that would need to be added to PMM, tested, QA’d, etc. I wouldn’t expect a fix within the next several weeks, possibly a month.

The error in your slow log is the same issue as the others, MariaDB doesn’t have that variable.

---

<div class="post-metadata">

**Author:** ![Naresh9999](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/naresh9999/32/5568_2.png) [@Naresh9999](https://forums.percona.com/u/Naresh9999)\
**Post date:** [August 9, 2023, 6:18pm UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/5 "2023-08-09T18:18:17Z")

</div>

Got it @matthewb

I appreciate the thorough explanation. Could you kindly ask that we fix this problem internally, preferably for the next release 2.40 ?

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [August 9, 2023, 6:36pm UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/6 "2023-08-09T18:36:31Z")

</div>

I’ve created a JIRA ticket for the PMM team and referenced this forum post. It is out of my hands at this point as I’m not in a position to tell that team what to focus on. You are welcome to engage with us on a support contract which could absolutely help speed things along.

[https://jira.percona.com/browse/PMM-12407](https://jira.percona.com/browse/PMM-12407)

---

<div class="post-metadata">

**Author:** ![Naresh9999](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/naresh9999/32/5568_2.png) [@Naresh9999](https://forums.percona.com/u/Naresh9999)\
**Post date:** [August 9, 2023, 6:43pm UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/7 "2023-08-09T18:43:06Z")

</div>

Thanks @matthewb for the ticket.

---

<div class="post-metadata">

**Author:** ![Naresh9999](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/naresh9999/32/5568_2.png) [@Naresh9999](https://forums.percona.com/u/Naresh9999)\
**Post date:** [August 10, 2023, 7:19pm UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/8 "2023-08-10T19:19:16Z")

</div>

@matthewb Is there any way that we can turn off these checks at the PMM agent level?

---

<div class="post-metadata">

**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\
**Post date:** [August 11, 2023, 3:36am UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/9 "2023-08-11T03:36:02Z")

</div>

Unfortunately, no. These queries come from various sub-systems within the PMM agent. Some are for the query analytics exporter, some are core to the mysqld exporter.

All of Percona’s software, including everything PMM related, is 100% opensource. You can pull down the exporter codebase and comment out all of these queries and recompile the exporter for your unique environment until this is fixed within PMM itself.

---

<div class="post-metadata">

**Author:** ![Naresh9999](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/naresh9999/32/5568_2.png) [@Naresh9999](https://forums.percona.com/u/Naresh9999)\
**Post date:** [August 11, 2023, 3:43am UTC](https://forums.percona.com/t/multiple-errors-from-pmm-in-the-mariadb-sql-error-log-file/24323/10 "2023-08-11T03:43:19Z")

</div>

Many thanks for the information. @matthewb

Actually, I am not sure about the coding part 😀, but if you could please request it next one or two months if we can fix this issues would be great.
