06:20:38 UTC - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Please help us make Percona Server better by reporting any bugs at http://bugs.percona.com/ key_buffer_size=33554432 read_buffer_size=131072 max_used_connections=414 max_threads=502 thread_count=81 connection_count=81 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 232495 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x36fb450 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 7fa997040e88 thread_stack 0x40000 /usr/sbin/mysqld(my_print_stacktrace+0x2c)[0x8c10bc] /usr/sbin/mysqld(handle_fatal_signal+0x469)[0x649249] /lib/x86_64-linux-gnu/libpthread.so.0(+0xf8d0)[0x7fb15df7d8d0] /usr/sbin/mysqld(_ZN8MDL_lock11Ticket_list13remove_ticketEP10MDL_ticket+0x11)[0x63b981] /usr/sbin/mysqld(_ZN8MDL_lock13remove_ticketEMS_NS_11Ticket_listEP10MDL_ticket+0x88)[0x63be48] /usr/sbin/mysqld(_ZN11MDL_context12release_lockE17enum_mdl_durationP10MDL_ticket+0x1a)[0x63ceea] /usr/sbin/mysqld(_ZN18Global_backup_lock18release_protectionEP3THD+0x1c)[0x7bee6c] /usr/sbin/mysqld(_ZN13MYSQL_BIN_LOG6commitEP3THDb+0x4c5)[0x86e645] /usr/sbin/mysqld(_Z15ha_commit_transP3THDbb+0x310)[0x591290] /usr/sbin/mysqld(_Z17trans_commit_stmtP3THD+0x29)[0x751679] /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x2eb9)[0x6c9c99] /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x5e8)[0x6ccfe8] /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0xd2b)[0x6ce3cb] /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x1a2)[0x69b1d2] /usr/sbin/mysqld(handle_one_connection+0x40)[0x69b270] /usr/sbin/mysqld(pfs_spawn_thread+0x146)[0x8fc596] /lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4)[0x7fb15df760a4] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fb15bfb662d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (7fa86246b900): is an invalid pointer Connection ID (thread ID): 46143427 Status: NOT_KILLED You may download the Percona Server operations manual by visiting http://www.percona.com/software/percona-server/. You may find information in the manual which will help you identify the cause of the crash. 180324 11:50:41 mysqld_safe Number of processes running now: 0 180324 11:50:41 mysqld_safe mysqld restarted 2018-03-24 11:50:41 0 [Warning] options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --slow-query-log is not set 2018-03-24 11:50:41 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details). 2018-03-24 11:50:41 0 [Warning] Insecure configuration for --secure-file-priv: Current value does not restrict location of generated files. Consider setting it to a valid, non-empty path. 2018-03-24 11:50:41 0 [Note] /usr/sbin/mysqld (mysqld 5.6.34-79.1-log) starting as process 19914 ... 2018-03-24 11:50:41 19914 [Warning] Using unique option prefix myisam-recover instead of myisam-recover-options is deprecated and will be removed in a future release. Please use the full name instead. 2018-03-24 11:50:45 19914 [Note] InnoDB: Using atomics to ref count buffer pool pages 2018-03-24 11:50:45 19914 [Note] InnoDB: The InnoDB memory heap is disabled 2018-03-24 11:50:45 19914 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2018-03-24 11:50:45 19914 [Note] InnoDB: Memory barrier is not used 2018-03-24 11:50:45 19914 [Note] InnoDB: Compressed tables use zlib 1.2.8 2018-03-24 11:50:45 19914 [Note] InnoDB: Using Linux native AIO 2018-03-24 11:50:45 19914 [Note] InnoDB: Using CPU crc32 instructions 2018-03-24 11:50:45 19914 [Note] InnoDB: Initializing buffer pool, size = 26.0G 2018-03-24 11:50:45 19914 [Note] InnoDB: Completed initialization of buffer pool 2018-03-24 11:50:46 19914 [Note] InnoDB: Highest supported file format is Barracuda. 2018-03-24 11:50:46 19914 [Note] InnoDB: Log scan progressed past the checkpoint lsn 2055424107682 2018-03-24 11:50:46 19914 [Note] InnoDB: Database was not shutdown normally! 2018-03-24 11:50:46 19914 [Note] InnoDB: Starting crash recovery. 2018-03-24 11:50:46 19914 [Note] InnoDB: Reading tablespace information from the .ibd files... 2018-03-24 11:50:50 19914 [Note] InnoDB: Restoring possible half-written data pages 2018-03-24 11:50:50 19914 [Note] InnoDB: from the doublewrite buffer... InnoDB: Doing recovery: scanned up to log sequence number 2055429350400 InnoDB: Doing recovery: scanned up to log sequence number 2055434593280 InnoDB: Doing recovery: scanned up to log sequence number 2055439836160 InnoDB: Doing recovery: scanned up to log sequence number 2055445079040 InnoDB: Doing recovery: scanned up to log sequence number 2055450321920 InnoDB: Doing recovery: scanned up to log sequence number 2055455564800 InnoDB: Doing recovery: scanned up to log sequence number 2055460807680 InnoDB: Doing recovery: scanned up to log sequence number 2055466050560 InnoDB: Doing recovery: scanned up to log sequence number 2055471293440 InnoDB: Doing recovery: scanned up to log sequence number 2055476536320 InnoDB: Doing recovery: scanned up to log sequence number 2055481779200 InnoDB: Doing recovery: scanned up to log sequence number 2055487022080 InnoDB: Doing recovery: scanned up to log sequence number 2055492264960 InnoDB: Doing recovery: scanned up to log sequence number 2055497507840 InnoDB: Doing recovery: scanned up to log sequence number 2055502750720 InnoDB: Doing recovery: scanned up to log sequence number 2055507993600 InnoDB: Doing recovery: scanned up to log sequence number 2055513236480 InnoDB: Doing recovery: scanned up to log sequence number 2055518479360 InnoDB: Doing recovery: scanned up to log sequence number 2055523722240 InnoDB: Doing recovery: scanned up to log sequence number 2055528965120 InnoDB: Doing recovery: scanned up to log sequence number 2055534208000 InnoDB: Doing recovery: scanned up to log sequence number 2055539450880 InnoDB: Doing recovery: scanned up to log sequence number 2055544693760 InnoDB: Doing recovery: scanned up to log sequence number 2055549936640 InnoDB: Doing recovery: scanned up to log sequence number 2055555179520 InnoDB: Doing recovery: scanned up to log sequence number 2055560422400 InnoDB: Doing recovery: scanned up to log sequence number 2055565665280 InnoDB: Doing recovery: scanned up to log sequence number 2055570908160 InnoDB: Doing recovery: scanned up to log sequence number 2055576151040 InnoDB: Doing recovery: scanned up to log sequence number 2055581393920 InnoDB: Doing recovery: scanned up to log sequence number 2055586636800 InnoDB: Doing recovery: scanned up to log sequence number 2055591879680 InnoDB: Doing recovery: scanned up to log sequence number 2055597122560 InnoDB: Doing recovery: scanned up to log sequence number 2055602365440 InnoDB: Doing recovery: scanned up to log sequence number 2055607608320 InnoDB: Doing recovery: scanned up to log sequence number 2055612851200 InnoDB: Doing recovery: scanned up to log sequence number 2055615477363 InnoDB: Transaction 3565956337 was in the XA prepared state. InnoDB: 1 transaction(s) which must be rolled back or cleaned up InnoDB: in total 0 row operations to undo InnoDB: Trx id counter is 3565956608 2018-03-24 11:50:57 19914 [Note] InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 InnoDB: Apply batch completed InnoDB: Last MySQL binlog file position 0 501536700, file name mysql-bin.000642 2018-03-24 11:51:24 19914 [Note] InnoDB: 128 rollback segment(s) are active. InnoDB: Starting in background the rollback of uncommitted transactions 2018-03-24 11:51:24 7fca945fb700 InnoDB: Rollback of non-prepared transactions completed 2018-03-24 11:51:24 19914 [Note] InnoDB: Waiting for purge to start 2018-03-24 11:51:24 19914 [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.34-79.1 started; log sequence number 2055615477363 2018-03-24 11:51:24 19914 [Note] Recovering after a crash using /var/lib/mysql/mysql-live-db/mysql-bin 2018-03-24 11:51:28 19914 [Note] Starting crash recovery... 2018-03-24 11:51:28 7fd1c8108740 InnoDB: Starting recovery for XA transactions... 2018-03-24 11:51:28 7fd1c8108740 InnoDB: Transaction 3565956337 in prepared state after recovery 2018-03-24 11:51:28 7fd1c8108740 InnoDB: Transaction contains changes to 1 rows 2018-03-24 11:51:28 7fd1c8108740 InnoDB: 1 transactions in prepared state after recovery 2018-03-24 11:51:28 19914 [Note] Found 1 prepared transaction(s) in InnoDB 2018-03-24 11:51:28 19914 [Note] Crash recovery finished. 2018-03-24 11:51:28 19914 [Note] RSA private key file not found: /var/lib/mysql/mysql-live-db//private_key.pem. Some authentication plugins will not work. 2018-03-24 11:51:28 19914 [Note] RSA public key file not found: /var/lib/mysql/mysql-live-db//public_key.pem. Some authentication plugins will not work. 2018-03-24 11:51:28 19914 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3306 2018-03-24 11:51:28 19914 [Note] - '0.0.0.0' resolves to '0.0.0.0'; 2018-03-24 11:51:28 19914 [Note] Server socket created on IP: '0.0.0.0'. 2018-03-24 11:51:28 19914 [ERROR] /usr/sbin/mysqld: Table './mysql/servers' is marked as crashed and should be repaired 2018-03-24 11:51:28 19914 [Warning] Checking table: './mysql/servers' 2018-03-24 11:51:28 19914 [ERROR] 1 client is using or hasn't closed the table properly 2018-03-24 11:51:28 19914 [Note] Event Scheduler: Loaded 0 events 2018-03-24 11:51:28 19914 [Note] /usr/sbin/mysqld: ready for connections. Version: '5.6.34-79.1-log' socket: '/var/run/mysqld/mysqld.sock' port: 3306 Percona Server (GPL), Release 79.1, Revision 1c589f9 2018-03-24 11:51:31 19914 [Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. The statement is unsafe because it uses a LIMIT clause. This is unsafe because the set of rows included cannot be predicted. Statement: Update sms_out_queue Set api_flag='97e31736-2f2b-11e8-afa1-0cc47ac77a72', api_flag_date_time=now(), service_instance_id='xml1' where (api_flag='' or api_flag is null) and sch_dt <= now() and exp_dt > now() and 11 between st_hr and end_hr order by priority limit 500