We have been experiencing this issue at least once a day for the last 5 days:
2017-04-18T00:11:49.499793Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 11624ms. The settings might not be optimal. (flushed=200, during the time.) 2017-04-18T00:11:49.649596Z 14 [ERROR] /usr/sbin/mysqld: Binary logging not possible. Message: An error occurred during flush stage of the commit. 'binlog_error_action' is set to 'ABORT_SERVER'. Hence aborting the server. 00:11:49 UTC - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. Attempting to collect some information that could help diagnose the problem. As this is a crash and something is definitely wrong, the information collection process might fail. Please help us make Percona Server better by reporting any bugs at http://bugs.percona.com/ key_buffer_size=8388608 read_buffer_size=131072 max_used_connections=189 max_threads=215 thread_count=101 connection_count=101 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 93386 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x7f04ff816000 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7f0da45dce00 thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x2c)[0xec707c] /usr/sbin/mysqld(handle_fatal_signal+0x461)[0x79d941] /lib/x86_64-linux-gnu/libpthread.so.0(+0x10330)[0x7f0dadae3330] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x37)[0x7f0dacf24c37] /lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7f0dacf28028] /usr/sbin/mysqld[0x761bc0] /usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG33handle_binlog_flush_or_sync_errorEP3THDb+0x19b)[0xe5d1ab] /usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG14ordered_commitEP3THDbb+0x131)[0xe65691] /usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG6commitEP3THDb+0xcb3)[0xe676e3] /usr/sbin/mysqld(_Z15ha_commit_transP3THDbb+0x131)[0x7fd571] /usr/sbin/mysqld(_Z12trans_commitP3THD+0x39)[0xd51cd9] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THDb+0xc92)[0xca4f12] /usr/sbin/mysqld(_Z11mysql_parseP3THDP12Parser_state+0x5c5)[0xcaba75] /usr/sbin/mysqld(_Z16dispatch_commandP3THDPK8COM_DATA19enum_server_command+0x92f)[0xcac42f] /usr/sbin/mysqld(_Z10do_commandP3THD+0x1b7)[0xcaddc7] /usr/sbin/mysqld(handle_connection+0x2a0)[0xd6f8c0] /usr/sbin/mysqld(pfs_spawn_thread+0x1b4)[0x1246fc4] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7f0dadadb184] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f0dacfe837d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7f04ff828030): COMMIT Connection ID (thread ID): 14 Status: NOT_KILLED You may download the Percona Server operations manual by visiting http://www.percona.com/software/percona-server/. You may find information in the manual which will help you identify the cause of the crash. 2017-04-18T00:11:51.272796Z mysqld_safe Number of processes running now: 0 2017-04-18T00:11:51.274351Z mysqld_safe mysqld restarted 2017-04-18T00:11:51.296136Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000) 2017-04-18T00:11:51.296265Z 0 [Warning] Changed limits: max_connections: 214 (requested 300) 2017-04-18T00:11:51.296278Z 0 [Warning] Changed limits: table_open_cache: 400 (requested 2000) 2017-04-18T00:11:51.443192Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2017-04-18T00:11:51.444043Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.14-7-log) starting as process 8607 ... 2017-04-18T00:11:51.454822Z 0 [Warning] InnoDB: Using innodb_support_xa is deprecated and the parameter may be removed in future releases. Only innodb_support_xa=ON is allowed. 2017-04-18T00:11:51.454848Z 0 [Warning] InnoDB: Using innodb_file_format is deprecated and the parameter may be removed in future releases. See http://dev.mysql.com/doc/refman/5.7/en/innodb-file-format.html 2017-04-18T00:11:51.454912Z 0 [Note] InnoDB: PUNCH HOLE support available 2017-04-18T00:11:51.454921Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2017-04-18T00:11:51.454925Z 0 [Note] InnoDB: Uses event mutexes 2017-04-18T00:11:51.454929Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier 2017-04-18T00:11:51.454933Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8 2017-04-18T00:11:51.454937Z 0 [Note] InnoDB: Using Linux native AIO 2017-04-18T00:11:51.455688Z 0 [Note] InnoDB: Number of pools: 1 2017-04-18T00:11:51.455770Z 0 [Note] InnoDB: Using CPU crc32 instructions 2017-04-18T00:11:51.456478Z 0 [Note] InnoDB: Initializing buffer pool, total size = 32G, instances = 8, chunk size = 128M 2017-04-18T00:11:52.181194Z 0 [Note] InnoDB: Completed initialization of buffer pool 2017-04-18T00:11:52.312596Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2017-04-18T00:11:52.324805Z 0 [Note] InnoDB: Recovering partial pages from the parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite 2017-04-18T00:11:52.405692Z 0 [Note] InnoDB: Highest supported file format is Barracuda. 2017-04-18T00:11:52.542600Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 303947279741 2017-04-18T00:11:52.613070Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 303952522240 2017-04-18T00:11:52.681503Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 303957765120 2017-04-18T00:11:52.841734Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 303948327936 2017-04-18T00:11:52.919589Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 303953570816 2017-04-18T00:11:53.005760Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 303958813696 2017-04-18T00:11:53.082855Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 303964056576 2017-04-18T00:11:53.086403Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 303964205271 2017-04-18T00:11:53.086696Z 0 [Note] InnoDB: Database was not shutdown normally! 2017-04-18T00:11:53.086701Z 0 [Note] InnoDB: Starting crash recovery. 2017-04-18T00:11:53.338831Z 0 [Note] InnoDB: Created parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite, size 31457280 bytes 2017-04-18T00:11:53.427281Z 0 [Note] InnoDB: Transaction 432564072 was in the XA prepared state. 2017-04-18T00:11:53.428568Z 0 [Note] InnoDB: Transaction 432564072 was in the XA prepared state. 2017-04-18T00:11:53.428713Z 0 [Note] InnoDB: 1 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo 2017-04-18T00:11:53.428726Z 0 [Note] InnoDB: Trx id counter is 432564480 2017-04-18T00:11:53.428740Z 0 [Note] InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
I have no idea why this happens or how to solve it. Any help will be appreciated.