We successfully upgraded Percona MySQL server version 5.6.29-76.2 to 5.7.16-10 (first) but it was crashing then we upgraded to 5.7.17-11. We see 5.7.17-11 is also crashing.
So far we are not able to figure out anything except that there is something to do with lob/text data.
Following is error and stack trace.
00:01:39 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
Please help us make Percona Server better by reporting any
bugs at [url]System Dashboard - Percona JIRA
key_buffer_size=268435456
read_buffer_size=1048576
max_used_connections=417
max_threads=1001
thread_count=408
connection_count=406
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1556244 K bytes of memory
Hope that’s ok; if not, decrease some variables in the equation.
Thread pointer: 0x7ef95c03c880
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = 7ef8e6df7d40 thread_stack 0x40000
/usr/sbin/mysqld(my_print_stacktrace+0x2c)[0xed056c]
/usr/sbin/mysqld(handle_fatal_signal+0x461)[0x7a0631]
/lib64/libpthread.so.0[0x31d080f7e0]
/usr/sbin/mysqld(_ZN10Field_blob15copy_blob_valueEP11st_mem_root+0x28)[0x7e3d08]
/usr/sbin/mysqld(_Z25mysql_prepare_blob_valuesP3THDR4ListI4ItemEP11st_mem_root+0x2b8)[0xe1b468]
/usr/sbin/mysqld(Z12write_recordP3THDP5TABLEP9COPY_INFOS4+0x87d)[0xe1c02d]
/usr/sbin/mysqld(_ZN14Sql_cmd_insert12mysql_insertEP3THDP10TABLE_LIST+0x82d)[0xe1ca9d]
/usr/sbin/mysqld(_ZN14Sql_cmd_insert7executeEP3THD+0xc2)[0xe1d2d2]
/usr/sbin/mysqld(_Z21mysql_execute_commandP3THDb+0x18d7)[0xca9e67]
/usr/sbin/mysqld(_ZN18Prepared_statement7executeEP6Stringb+0x357)[0xcd9e57]
/usr/sbin/mysqld(ZN18Prepared_statement12execute_loopEP6StringbPhS2+0xca)[0xcda25a]
/usr/sbin/mysqld(_Z19mysqld_stmt_executeP3THDmmPhm+0x13b)[0xcda58b]
/usr/sbin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x190f)[0xcb197f]
/usr/sbin/mysqld(_Z10do_commandP3THD+0x1b7)[0xcb24a7]
/usr/sbin/mysqld(handle_connection+0x2a0)[0xd76740]
/usr/sbin/mysqld(pfs_spawn_thread+0x1b4)[0xeed474]
/lib64/libpthread.so.0[0x31d0807aa1]
/lib64/libc.so.6(clone+0x6d)[0x31d00e8aad]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (7ef95c12bec0): is an invalid pointer
Connection ID (thread ID): 4170
Status: NOT_KILLED
After converting it in readable format then we see this.
0xed056c my_print_stacktrace + 44
0x7a0631 handle_fatal_signal + 1121
0x31d080f7e0 _end + -829069216
0x7e3d08 Field_blob::copy_blob_value(st_mem_root*) + 40
0xe1b468 mysql_prepare_blob_values(THD*, List&, st_mem_root*) + 696
0xe1c02d write_record(THD*, TABLE*, COPY_INFO*, COPY_INFO*) + 2173
0xe1ca9d Sql_cmd_insert::mysql_insert(THD*, TABLE_LIST*) + 2093
0xe1d2d2 Sql_cmd_insert::execute(THD*) + 194
0xca9e67 mysql_execute_command(THD*, bool) + 6359
0xcd9e57 Prepared_statement::execute(String*, bool) + 855
0xcda25a Prepared_statement::execute_loop(String*, bool, unsigned char*, unsigned char*) + 202
0xcda58b mysqld_stmt_execute(THD*, unsigned long, unsigned long, unsigned char*, unsigned long) + 315
0xcb197f dispatch_command(THD*, COM_DATA const*, enum_server_command) + 6415
0xcb24a7 do_command(THD*) + 439
0xd76740 handle_connection + 672
0xeed474 pfs_spawn_thread + 436
0x31d0807aa1 _end + -829101279
0x31d00e8aad _end + -836568275
When we read stack trace from bottom to top then we can see that MySQL is working on some record which has something to do with blob data.
Has someone faced the same issue?
I have also filed a bug [url]https://bugs.launchpad.net/percona-server/+bug/1667552[/url] but so far no response.
Any pointer/clue highly appreciated.
Thanks