pt-query-digest: MySQLProtocolParser caused an error

Polyglot Projects Percona Toolkit
#1

I used to run this command:

tcpdump -i lo port 3306 -s 65535 -x -n -q -tttt > pt.log; pt-query-digest --type=tcpdump pt.log

But it stopped working for me. I cannot put my finger on the problem because I experience it on different Debian versions and different percona server versions, different percona-toolkit version and I can even reproduce with mariadb.

The error I get:

Pipeline process 4 (MySQLProtocolParser) caused an error: Argument “” isn’t numeric in multiplication () at /usr/bin/pt-que
ry-digest line 4505, <$fh> line 8.
Will retry pipeline process 3 (MySQLProtocolParser) 100 more times.
Pipeline process 4 (MySQLProtocolParser) caused an error: Argument “” isn’t numeric in multiplication () at /usr/bin/pt-que
ry-digest line 4505, <$fh> line 10.
Will retry pipeline process 3 (MySQLProtocolParser) 99 more times.

…

Will retry pipeline process 3 (MySQLProtocolParser) 1 more time.
The pipeline caused an error: Pipeline process 4 (MySQLProtocolParser) caused an error: Argument “” isn’t numeric in multipl
ication (*) at /usr/bin/pt-query-digest line 4505, <$fh> line 210.
Terminating pipeline because process 3 (MySQLProtocolParser) caused too many errors.

What is wrong?

#2

Hello,
Could you share the dump file?

#3

I published a log which should contain only 1 query at https://ctors.net/pub/pt.log

#4

I am having this exact same problem. Interestingly, if I change the date on my computer’s clock back to 2019 and collect a tcpdump, pt-query-digest has no problem parsing that file. But any file collected now with the correct date gives errors like tvlooy posted above. I can’t seem to figure out why a new year might be throwing things off though.

I have attached a 7z archive with two pcaps, one from last year (working), and one from today (not working). Remove the .txt extension.

#5

Hi usafbeach, thanks for reporting, I’ll bring it to the attention of the team.
I have, though, deleted the attachment as we don’t accept archive files. You can either leave it at that or you can use an external host for the pcaps
Let’s see if the team need them?

#6

Hi again, could you upload those pcaps to/as gist please? https://gist.github.com/
Then I can get the dev team to review. THANKS!

#7

Hi Lorraine, thanks for the reply. Hopefully you don’t mind, but it was easier to upload the files to a github repo instead:

https://github.com/0xBEAKER/0xBEAKER.github.io/blob/master/files/tcpdump2019.pcap
https://github.com/0xBEAKER/0xBEAKER.github.io/blob/master/files/tcpdump2020.pcap

#8

That should work too, I will ping the dev team :slight_smile:

#9

I had this same problem with a new pcap. I opened a small pcap in vim and changed all of the dates from ‘2020-01-08’ to ‘2019-01-08’. pt-query-digest no longer errors.
We are too far into the future. Please make a time machine so we can go back a few years when pt-query-digest still worked. Or whatever you see fit.

Thank you

#10

Hello,
There is a fix in progress. I think it is related to this issue: https://github.com/percona/percona-toolkit/pull/434

#11

The fix works. Thanks a lot everyone!

#12

The fix is already merged in the 3.0 branch and it will be released in the next version

#13

Hey guys!
That method worked for me too for a while. Day 04 Fev was the last time I executed a pt-query-digest command and it worked fine!
Today, 07 Fev it returns me a new error and I didn’t touch in anything. Its a crontab script… the error is:

Pipeline process 3 (TcpdumpParser) caused an error: Use of uninitialized value $source in pattern match (m//) at /usr/bin/pt-query-digest line 3683, <$fh> chunk 1.
Will retry pipeline process 2 (TcpdumpParser) 100 more times.
TCP session 10.8.0.2:57218 had errors, will save them in /tmp/pt-query-digest-errors.L9yj1ui
Pipeline process 3 (TcpdumpParser) caused an error: Use of uninitialized value $source in pattern match (m//) at /usr/bin/pt-query-digest line 3683, <$fh> chunk 17.
Will retry pipeline process 2 (TcpdumpParser) 99 more times.

What should I do?
Its awkward…

Thank you!!

#14

Hello,

That’s a different error. I would need steps of a tcpdump file to reproduce it.
Could you provide it?

Thanks

#15

Hey! Sorry for the time, but the error still up…

Crontab -e

&#64;reboot screen -dmS bTCP; sleep 5; screen -S bTCP -X stuff '/usr/sbin/tcpdump -i any port 3306 -s 65535 -x -nn -q -tttt> /etc/path/to/tcpdump_3306.out\n'

I execute this command trough a script:

os.system("pt-query-digest --report-format query_report --type tcpdump /etc/path/to/tcpdump_3306.out > /etc/path/to/log-mysql.txt")

The error:

Pipeline process 3 (TcpdumpParser) caused an error: Use of uninitialized value $source in pattern match (m//) at /usr/bin/pt-query-digest line 3683, <$fh> chunk 1.
Will retry pipeline process 2 (TcpdumpParser) 100 more times.
Pipeline process 3 (TcpdumpParser) caused an error: substr outside of string at /usr/bin/pt-query-digest line 3701, <$fh> chunk 59.
Will retry pipeline process 2 (TcpdumpParser) 99 more times.

What is happening? :confused:

EDIT:

LINE 3683 have the following content:

$args{oktorun}->(0) if $args{oktorun};

LINE 3701 have the following content:

my $ip_hlen = hex(substr($data, 1, 1)); # Num of 32-bit words in header.

Thats because I add a comment on line 3682 and its probably different from the original one.

[COLOR=#FF0000]EDIT 2:

New error appear:

Pipeline process 4 (MySQLProtocolParser) caused an error: substr outside of string at /usr/bin/pt-query-digest line 4602, <$fh> line 483.

That line have the following content:

PTDEBUG && _d('data:', $data, 'first byte:', $first_byte);
#16

carlos.salguero Hey buddy! Could you please help on this? I really need to use Percona Toolkit… thank you.

#17

I really need to fix this issue… please.

#18

I notice that Carlos requested a tcpdump a couple of weeks ago and I suspect he can’t help you much more without that.

With this being the free open source forum we cannot commit to responding in any given time frame as the team may have higher priority tasks such as working on new software project developments or with customers. I can understand that this may be frustrating, but it is a reality nonetheless.

Without the tcpdump I suspect time spent tracking down the issue is fruitless.

If the problem is business affecting and you need our paid support urgently then I am able to put you in touch with someone that can help you with that.

#19

I understand but this is a problem that I can’t really solve. Thats why I need some help, I’m not able to pay at the moment. I’ll wait for a new update on package…

#20

I’m seeing the same error as hmota, tcpdump attached
Wondering if it’s related to IPv6?

mysqltcpdump.txt.zip (3.8 MB)

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright © 2006 - 2021 Percona LLC. All rights reserved.