Not the answer you need?
Register and ask your own question!

Xtradb crash on ubuntu upgrade

quetzalquetzal EntrantCurrent User Role Beginner
Hello,
We are trying to upgrade our ubuntu cluster from Ubuntu 14.04 to Ubuntu 16.04.
We got a problem with percona xtradb cluster.
We use, on the old cluster, percona xtradb 5.6.41-28.28-1.trusty package from percona repo. If we upgrade ubuntu with this package, this package run fine on ubuntu 16.04. But if we upgrade to 5.6.41-28.28-1.xenial, all seem to work good (clustercheck is ok, sst is ok, etc...) but as soon as we direct some trafic to the upgraded node, it crash.
The only message we have in logs is:

Thread pointer: 0x2969f80
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 7f7f351ace38 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x3b)[0x92b9bb]
/usr/sbin/mysqld(handle_fatal_signal+0x49a)[0x691bba]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f83117d9390]
/lib/x86_64-linux-gnu/libc.so.6(+0x14dec2)[0x7f8310caaec2]
/usr/sbin/mysqld(_Z21find_or_create_digestP10PFS_threadPK18sql_digest_storagePKcj+0x39b)[0x93ef1b]
/usr/sbin/mysqld[0x968843]
/usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x386)[0x7209b6]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x244)[0x7243e4]
/usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x182)[0x6e85b2]
/usr/sbin/mysqld(handle_one_connection+0x40)[0x6e87c0]
/usr/sbin/mysqld(pfs_spawn_thread+0x146)[0x964416]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7f83117cf6ba]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f8310c6441d]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0): is an invalid pointer
Connection ID (thread ID): 75
Status: NOT_KILLED

When we try to execute manually some requests (select, inserts, etc...) we have no problem, but as soon as we add the real trafic it crash. We have try to log all request, and to replay requests concerned by the thread ID that crash, without any crash.
How is it possible that trusty version have no problem on xenial but xenial version have ? How can we upgrade ?

Thanks !

Comments

  • przemekprzemek Percona Support Engineer Percona Staff Role
    Hello,

    This crash is identical as in https://jira.percona.com/browse/PXC-1909 and this function:
    /usr/sbin/mysqld(_Z21find_or_create_digestP10PFS_threadPK18s ql_digest_storagePKcj+0x39b)[0x93ef1b]
    
    refers to EVENTS_STATEMENTS_SUMMARY_BY_DIGEST in Performance Schema.

    Unfortunately, we are missing the test case in this bug report. I have tried to trigger this crash by enabling all the P_S instruments and consumers, and running various sysbench tests in multiple threads, and in the same time, concurrent queries to performance_schema.events_statements_summary_by_digest, but I am not able to trigger any crash.
    Used the following packages:
    root@ubuntu-xenial:~# dpkg -l|grep -i percona
    ii  percona-release                   0.1-6.xenial                               all          Package to install Percona gpg key and APT repo
    ii  percona-xtrabackup                2.3.10-1.xenial                            amd64        Open source backup tool for InnoDB and XtraDB
    ii  percona-xtradb-cluster-client-5.6 5.6.41-28.28-1.xenial                      amd64        Percona XtraDB Cluster database client binaries
    ii  percona-xtradb-cluster-common-5.6 5.6.41-28.28-1.xenial                      amd64        Percona XtraDB Cluster database common files (e.g. /etc/mysql/my.cnf)
    ii  percona-xtradb-cluster-galera-3   3.28-1.xenial                              amd64        Metapackage for latest version of galera3.
    ii  percona-xtradb-cluster-galera-3.x 3.28-1.xenial                              amd64        Galera components of Percona XtraDB Cluster
    ii  percona-xtradb-cluster-server-5.6 5.6.41-28.28-1.xenial                      amd64        Percona XtraDB Cluster database server binaries
    


    Can you send more details on what your workload looks like? What kind of queries are you executing for performance_schema tables? What are your P_S settings?
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.