MySQL crash recovery automatically...Can someone suggest me on this?

18:48:57 UTC - mysqld got signal 11 ;
Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware.
Thread pointer: 0x7fdc7493fea0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = 7fe79c058d10 thread_stack 0x46000
/usr/sbin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x3d) [0x5599d215849d]
/usr/sbin/mysqld(handle_fatal_signal+0x2fb) [0x5599d110222b]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12980) [0x7fe7dafd0980]
/usr/sbin/mysqld(+0xf293ce) [0x5599d0fa53ce]
/usr/sbin/mysqld(+0xf295f2) [0x5599d0fa55f2]
/usr/sbin/mysqld(+0xf29623) [0x5599d0fa5623]
/usr/sbin/mysqld(JOIN::finalize_table_conditions()+0xeb) [0x5599d0fab87b]
/usr/sbin/mysqld(JOIN::replace_index_subquery()+0x107) [0x5599d0fabd27]
/usr/sbin/mysqld(JOIN::optimize()+0x2765) [0x5599d0fbf1b5]
/usr/sbin/mysqld(SELECT_LEX::optimize(THD*)+0xb6) [0x5599d101bc46]
/usr/sbin/mysqld(SELECT_LEX_UNIT::optimize(THD*, TABLE*, bool)+0x7b) [0x5599d108a3eb]
/usr/sbin/mysqld(SELECT_LEX::optimize(THD*)+0x115) [0x5599d101bca5]
/usr/sbin/mysqld(SELECT_LEX_UNIT::optimize(THD*, TABLE*, bool)+0x7b) [0x5599d108a3eb]
/usr/sbin/mysqld(TABLE_LIST::optimize_derived(THD*)+0x40) [0x5599d145ddd0]
/usr/sbin/mysqld(JOIN::optimize()+0x46c) [0x5599d0fbcebc]
/usr/sbin/mysqld(SELECT_LEX::optimize(THD*)+0xb6) [0x5599d101bc46]
/usr/sbin/mysqld(SELECT_LEX_UNIT::optimize(THD*, TABLE*, bool)+0x7b) [0x5599d108a3eb]
/usr/sbin/mysqld(Sql_cmd_dml::execute_inner(THD*)+0x3d) [0x5599d101a65d]
/usr/sbin/mysqld(Sql_cmd_dml::execute(THD*)+0x598) [0x5599d10246e8]
/usr/sbin/mysqld(mysql_execute_command(THD*, bool)+0xaae) [0x5599d0fc746e]
/usr/sbin/mysqld(sp_instr_stmt::exec_core(THD*, unsigned int*)+0x51) [0x5599d0f43e71]
/usr/sbin/mysqld(sp_lex_instr::reset_lex_and_exec_core(THD*, unsigned int*, bool)+0x17b) [0x5599d0f45d4b]
/usr/sbin/mysqld(sp_lex_instr::validate_lex_and_execute_core(THD*, unsigned int*, bool)+0xde) [0x5599d0f46cfe]
/usr/sbin/mysqld(sp_instr_stmt::execute(THD*, unsigned int*)+0xc3) [0x5599d0f48543]
/usr/sbin/mysqld(sp_head::execute(THD*, bool)+0x3be) [0x5599d0f3d0de]
/usr/sbin/mysqld(sp_head::execute_procedure(THD*, mem_root_deque<Item*>)+0x7e3) [0x5599d0f40423]
/usr/sbin/mysqld(Sql_cmd_call::execute_inner(THD
)+0x136) [0x5599d1453106]
/usr/sbin/mysqld(Sql_cmd_dml::execute(THD*)+0x598) [0x5599d10246e8]
/usr/sbin/mysqld(mysql_execute_command(THD*, bool)+0xaae) [0x5599d0fc746e]
/usr/sbin/mysqld(dispatch_sql_command(THD*, Parser_state*)+0x3f9) [0x5599d0fcad29]
/usr/sbin/mysqld(dispatch_command(THD*, COM_DATA const*, enum_server_command)+0x15ed) [0x5599d0fcc80d]
/usr/sbin/mysqld(do_command(THD*)+0x1c4) [0x5599d0fcdfe4]
/usr/sbin/mysqld(+0x1077020) [0x5599d10f3020]
/usr/sbin/mysqld(+0x2640091) [0x5599d26bc091]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fe7dafc56db]

Usually sig 11 is due to Out Of Memory (aka OOM). Check your syslogs and dmesg and see if the oomkiller reaped MySQL.

2 Likes

@matthewb

I got this only can you suggest me :

57 prod-dbn-01 kernel: [1672253.593585] kauditd_printk_skb: 265 callbacks suppressed
Jun 2 14:48:57 prod-dbn-01 kernel: [1672253.593587] audit: type=1400 audit(1654195737.445:24597): apparmor=“DENIED” operation=“open” profile=“/usr/sbin/mysqld” name=“/proc/119316/task/119403/mem” pid=119316 comm=“mysqld” requested_mask=“r” denied_mask=“r” fsuid=112 ouid=112
Jun 2 14:48:59 ppd01 systemd[1]: mysql.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jun 2 14:48:59 ppd01 systemd[1]: mysql.service: Failed with result ‘exit-code’.
Jun 2 14:48:59 ppd01 systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
Jun 2 14:48:59 ppd01 systemd[1]: mysql.service: Scheduled restart job, restart counter is at 1.
Jun 2 14:48:59 ppd01 systemd[1]: Stopped MySQL Community Server.
Jun 2 14:48:59 ppd01 systemd[1]: Starting MySQL Community Server…

1 Like

Looks like apparmor denied some operation on MySQL? Try disabling apparmor and repeat your crashing scenario.

1 Like

Also, what version are you in ? Try patching to latest minor path of your major version (i assume 5.7) as it could potentially be a bug

1 Like

@CTutte I am using 8.0.23

1 Like

@matthewb I Stopped/disabled apparmor no luck?

12:23:40 prod-dbn-01 kernel: [446075.741485] audit: type=1400 audit(1655915020.952:15): apparmor=“DENIED” operation=“open” profile=“/usr/sbin/mysqld” name=“/proc/1460/task/2198/mem” pid=1460 comm=“mysqld” requested_mask=“r” denied_mask=“r” fsuid=112 ouid=112
12:23:42 server-01 systemd[1]: mysql.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
12:23:42 server-01 systemd[1]: mysql.service: Failed with result ‘exit-code’.
12:23:42 server-01 systemd[1]: mysql.service: Service hold-off time over, scheduling restart.
12:23:42 server-01 systemd[1]: mysql.service: Scheduled restart job, restart counter is at 1.
12:23:42 server-01 systemd[1]: Stopped MySQL Community Server.
12:23:42 server-01 systemd[1]: Starting MySQL Community Server…

1 Like

Looks like apparmor is still running.

Quick google: 16.04 - MySQL won't start because of AppArmor? - Ask Ubuntu

1 Like

@matthewb apparmor service found disabled.

It could be hardware issue or does upgrade MySQL from 8.0.23 to 8.0.25 will help here.

1 Like

It won’t hurt to upgrade to the latest 8.0.28. Hardware is rarely the problem.

1 Like