pt-query-digest: MySQLProtocolParser caused an error

tvlooytvlooy EntrantActive Member Participant
I used to run this command:

tcpdump -i lo port 3306 -s 65535 -x -n -q -tttt > pt.log; pt-query-digest --type=tcpdump pt.log

But it stopped working for me. I cannot put my finger on the problem because I experience it on different Debian versions and different percona server versions, different percona-toolkit version and I can even reproduce with mariadb.

The error I get:

Pipeline process 4 (MySQLProtocolParser) caused an error: Argument "" isn't numeric in multiplication (*) at /usr/bin/pt-que
ry-digest line 4505, <$fh> line 8.
Will retry pipeline process 3 (MySQLProtocolParser) 100 more times.
Pipeline process 4 (MySQLProtocolParser) caused an error: Argument "" isn't numeric in multiplication (*) at /usr/bin/pt-que
ry-digest line 4505, <$fh> line 10.
Will retry pipeline process 3 (MySQLProtocolParser) 99 more times.

...


Will retry pipeline process 3 (MySQLProtocolParser) 1 more time.
The pipeline caused an error: Pipeline process 4 (MySQLProtocolParser) caused an error: Argument "" isn't numeric in multipl
ication (*) at /usr/bin/pt-query-digest line 4505, <$fh> line 210.
Terminating pipeline because process 3 (MySQLProtocolParser) caused too many errors.

What is wrong?

Comments

  • carlos.salguerocarlos.salguero Percona Toolkit Developer Percona
    Hello,
    Could you share the dump file?
  • tvlooytvlooy Entrant Active Member Participant
    I published a log which should contain only 1 query at https://ctors.net/pub/pt.log
  • usafbeachusafbeach Entrant Active Member Contributor
    I am having this exact same problem. Interestingly, if I change the date on my computer's clock back to 2019 and collect a tcpdump, pt-query-digest has no problem parsing that file. But any file collected now with the correct date gives errors like tvlooy posted above. I can't seem to figure out why a new year might be throwing things off though.

    I have attached a 7z archive with two pcaps, one from last year (working), and one from today (not working). Remove the .txt extension.
  • lorraine.pocklingtonlorraine.pocklington Percona Community Manager Reader Mentor
    Hi usafbeach, thanks for reporting, I'll bring it to the attention of the team.
    I have, though, deleted the attachment as we don't accept archive files. You can either leave it at that or you can use an external host for the pcaps
    Let's see if the team need them?
  • lorraine.pocklingtonlorraine.pocklington Percona Community Manager Reader Mentor
    Hi again, could you upload those pcaps to/as gist please? https://gist.github.com/
    Then I can get the dev team to review. THANKS!
  • usafbeachusafbeach Entrant Active Member Contributor
    Hi Lorraine, thanks for the reply. Hopefully you don't mind, but it was easier to upload the files to a github repo instead:

    https://github.com/0xBEAKER/0xBEAKER.github.io/blob/master/files/tcpdump2019.pcap
    https://github.com/0xBEAKER/0xBEAKER.github.io/blob/master/files/tcpdump2020.pcap
  • lorraine.pocklingtonlorraine.pocklington Percona Community Manager Reader Mentor
    That should work too, I will ping the dev team :)
  • jermjerm Entrant Active Member Participant
    I had this same problem with a new pcap. I opened a small pcap in vim and changed all of the dates from '2020-01-08' to '2019-01-08'. pt-query-digest no longer errors.
    We are too far into the future. Please make a time machine so we can go back a few years when pt-query-digest still worked. Or whatever you see fit.

    Thank you
  • carlos.salguerocarlos.salguero Percona Toolkit Developer Percona
    Hello,
    There is a fix in progress. I think it is related to this issue: https://github.com/percona/percona-toolkit/pull/434
  • tvlooytvlooy Entrant Active Member Participant
    The fix works. Thanks a lot everyone!
  • carlos.salguerocarlos.salguero Percona Toolkit Developer Percona
    The fix is already merged in the 3.0 branch and it will be released in the next version
  • hmotahmota Entrant Active Member Participant
    Hey guys!
    That method worked for me too for a while. Day 04 Fev was the last time I executed a pt-query-digest command and it worked fine!
    Today, 07 Fev it returns me a new error and I didn't touch in anything. Its a crontab script.. the error is:

    Pipeline process 3 (TcpdumpParser) caused an error: Use of uninitialized value $source in pattern match (m//) at /usr/bin/pt-query-digest line 3683, <$fh> chunk 1.
    Will retry pipeline process 2 (TcpdumpParser) 100 more times.
    TCP session 10.8.0.2:57218 had errors, will save them in /tmp/pt-query-digest-errors.L9yj1ui
    Pipeline process 3 (TcpdumpParser) caused an error: Use of uninitialized value $source in pattern match (m//) at /usr/bin/pt-query-digest line 3683, <$fh> chunk 17.
    Will retry pipeline process 2 (TcpdumpParser) 99 more times.


    What should I do?
    Its awkward...

    Thank you!!
  • carlos.salguerocarlos.salguero Percona Toolkit Developer Percona
    Hello,

    That's a different error. I would need steps of a tcpdump file to reproduce it.
    Could you provide it?

    Thanks
  • hmotahmota Entrant Active Member Participant
    Hey! Sorry for the time, but the error still up..

    Crontab -e
    &#64;reboot screen -dmS bTCP; sleep 5; screen -S bTCP -X stuff '/usr/sbin/tcpdump -i any port 3306 -s 65535 -x -nn -q -tttt> /etc/path/to/tcpdump_3306.out\n'
    
    I execute this command trough a script:
    os.system("pt-query-digest --report-format query_report --type tcpdump /etc/path/to/tcpdump_3306.out > /etc/path/to/log-mysql.txt")
    

    The error:
    Pipeline process 3 (TcpdumpParser) caused an error: Use of uninitialized value $source in pattern match (m//) at /usr/bin/pt-query-digest line 3683, <$fh> chunk 1.
    Will retry pipeline process 2 (TcpdumpParser) 100 more times.
    Pipeline process 3 (TcpdumpParser) caused an error: substr outside of string at /usr/bin/pt-query-digest line 3701, <$fh> chunk 59.
    Will retry pipeline process 2 (TcpdumpParser) 99 more times.
    

    What is happening? :/



    EDIT:

    LINE 3683 have the following content:
    $args{oktorun}->(0) if $args{oktorun};
    


    LINE 3701 have the following content:
    my $ip_hlen = hex(substr($data, 1, 1)); # Num of 32-bit words in header.
    


    Thats because I add a comment on line 3682 and its probably different from the original one.


    EDIT 2:

    New error appear:

    Pipeline process 4 (MySQLProtocolParser) caused an error: substr outside of string at /usr/bin/pt-query-digest line 4602, <$fh> line 483.
    

    That line have the following content:
    PTDEBUG && _d('data:', $data, 'first byte:', $first_byte);
    
    
  • hmotahmota Entrant Active Member Participant
    carlos.salguero Hey buddy! Could you please help on this? I really need to use Percona Toolkit.. thank you.
  • hmotahmota Entrant Active Member Participant
    I really need to fix this issue.. please.
  • lorraine.pocklingtonlorraine.pocklington Percona Community Manager Reader Mentor
    I notice that Carlos requested a tcpdump a couple of weeks ago and I suspect he can't help you much more without that.

    With this being the free open source forum we cannot commit to responding in any given time frame as the team may have higher priority tasks such as working on new software project developments or with customers. I can understand that this may be frustrating, but it is a reality nonetheless.

    Without the tcpdump I suspect time spent tracking down the issue is fruitless.

    If the problem is business affecting and you need our paid support urgently then I am able to put you in touch with someone that can help you with that.
  • hmotahmota Entrant Active Member Participant
    I notice that Carlos requested a tcpdump a couple of weeks ago and I suspect he can't help you much more without that.

    With this being the free open source forum we cannot commit to responding in any given time frame as the team may have higher priority tasks such as working on new software project developments or with customers. I can understand that this may be frustrating, but it is a reality nonetheless.

    Without the tcpdump I suspect time spent tracking down the issue is fruitless.

    If the problem is business affecting and you need our paid support urgently then I am able to put you in touch with someone that can help you with that.

    I understand but this is a problem that I can't really solve. Thats why I need some help, I'm not able to pay at the moment. I'll wait for a new update on package....
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.