# Unable to perform Percona MySQL version upgrade to 5.7 **URL:** \ **Category:** Percona Server for MySQL 5.7\ **Created:** [April 10, 2023, 7:20pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301 "2023-04-10T19:20:01Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 10, 2023, 7:20pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/1 "2023-04-10T19:20:02Z")
I am trying to upgrade my Percona MySQL server to 5.7. The current version on Percona MySQL is 5.6.51-91.0 and the version that I am trying to upgrade is 5.7.40-43. I am performing an in-place upgrade having the following steps: - Backup the data and my.cnf file - Remove the package of v5.6 - Install the package of v5.7 - Executing the mysql\_upgrade checker, the upgrade the is getting failed at this point and the error log says: 2023-04-10T18:22:44.763494Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.41-44-log) starting as process 31924 … 2023-04-10T18:22:44.798744Z 0 [Note] InnoDB: PUNCH HOLE support available 2023-04-10T18:22:44.798793Z 0 [Note] InnoDB: Mutexes and rw\_locks use GCC atomic builtins 2023-04-10T18:22:44.798804Z 0 [Note] InnoDB: Uses event mutexes 2023-04-10T18:22:44.798812Z 0 [Note] InnoDB: GCC builtin \_\_atomic\_thread\_fence() is used for memory barrier 2023-04-10T18:22:44.798821Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.12 2023-04-10T18:22:44.798830Z 0 [Note] InnoDB: Using Linux native AIO 2023-04-10T18:22:44.803122Z 0 [Note] InnoDB: Number of pools: 1 2023-04-10T18:22:44.808970Z 0 [Note] InnoDB: Using CPU crc32 instructions 2023-04-10T18:22:44.819849Z 0 [Note] InnoDB: Initializing buffer pool, total size = 300G, instances = 8, chunk size = 128M 2023-04-10T18:22:52.880252Z 0 [Note] InnoDB: Completed initialization of buffer pool 2023-04-10T18:22:54.811564Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-04-10T18:22:54.826993Z 0 [Note] InnoDB: Crash recovery did not find the parallel doublewrite buffer at /mysqlData/mysql/data/xb\_doublewrite 2023-04-10T18:22:54.829936Z 0 [Note] InnoDB: Highest supported file format is Barracuda. **2023-04-10T18:22:54.830997Z 0 [ERROR] InnoDB: Upgrade after a crash is not supported. This redo log was created before MySQL 5.7.9, and we did not find a valid checkpoint. Please follow the instructions at [http://dev.mysql.com/doc/refman/5.7/en/upgrading.html](http://dev.mysql.com/doc/refman/5.7/en/upgrading.html)** **2023-04-10T18:22:54.831036Z 0 [ERROR] InnoDB: Plugin initialization aborted with error Generic error** **2023-04-10T18:22:59.936051Z 0 [ERROR] Plugin ‘InnoDB’ init function returned error.** **2023-04-10T18:22:59.936100Z 0 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed.** **2023-04-10T18:22:59.936173Z 0 [ERROR] Failed to initialize builtin plugins.** **2023-04-10T18:22:59.936184Z 0 [ERROR] Aborting** I also found that the parameter innodb\_log\_block\_size is replaced now with innodb\_log\_write\_ahead\_size in v5.7 so I made those changes as well in the my.cnf file but it’s not working and making the database crash. Below is the my.cnf : [client] socket=/mysqlData/mysql/mysql.sock [mysqld] #require\_secure\_transport=ON #Settings related to Teleport configuration ssl-cert=/etc/teleport/va-db-replica-015.crt ssl-key=/etc/teleport/va-db-replica-015.key ssl-ca=/etc/teleport/va-db-replica-015.cas #default-time-zone=EST5EDT datadir=/mysqlData/mysql/data socket=/mysqlData/mysql/mysql.sock max\_connections=4000 # Key buffer size is used for MyISAM table, hence disabling the same #key\_buffer\_size=32M sort\_buffer\_size=64M read\_buffer\_size=64M join\_buffer\_size=64M table\_open\_cache=4096 thread\_cache\_size=500 connect\_timeout=60 wait\_timeout=600 interactive\_timeout=120 open\_files\_limit=40000 innodb\_open\_files=800 event\_scheduler=OFF max\_allowed\_packet=128M tmp\_table\_size=1G max\_heap\_table\_size=1G ####################### # SLAVE SETTINGS ####################### #log-slave-updates skip-slave-start #slave-skip-errors=1062, 1032 #skip-slave-start read-only server-id=577 report-host=va-db-replica-015 log-bin=/mysqlData/mysqlbinlog/va-db-replica-015-binlog relay-log=/mysqlData/mysqlbinlog/va-db-replica-015-relay-bin #slave\_compressed\_protocol=1 binlog\_format=ROW expire\_logs\_days=1 #auto-increment-increment=2 #auto-increment-offset=1 ####################### # QUERY CACHE is deprecated from MySQL version 5.6 ####################### #query\_cache\_size=0 #query\_cache\_type=1 #query\_cache\_strip\_comments=1 skip-name-resolve ####################### # LOGS ####################### log\_error=/mysqlData/log/mysqld.log #slow\_query\_log\_file=/mysqlData/log/slow.log #slow\_query\_log=OFF #slow\_launch\_time=1 #log\_queries\_not\_using\_indexes=1 #long\_query\_time=10 #log\_slow\_verbosity=fu #query\_response\_time\_stats=ON ####################### # Innodb Performance ####################### #innodb\_force\_recovery=0 innodb\_file\_per\_table innodb\_buffer\_pool\_size=300G #innodb\_additional\_mem\_pool\_size=20M innodb\_data\_file\_path = ibdata1:1024M:autoextend innodb\_flush\_method=O\_DIRECT innodb\_log\_file\_size=1G innodb\_flush\_log\_at\_trx\_commit=2 # Dynamic innodb\_io\_capacity=500 # Dynamic innodb\_concurrency\_tickets=4000 innodb\_read\_io\_threads=40 innodb\_write\_io\_threads=30 #Controls the number of background threads used by the InnoDB storage engine to perform purging and garbage collection of old data from the database. innodb\_purge\_threads=4 #innodb\_flush\_neighbor\_pages=none #innodb\_adaptive\_flushing\_method=keep\_average #Transaction related parameters that are dynamic #innodb\_log\_block\_size=8192 #innodb\_log\_block\_size has been replaced by innodb\_log\_write\_ahead\_size variable in v5.7. innodb\_log\_write\_ahead\_size=8192 innodb\_log\_buffer\_size=1G explicit\_defaults\_for\_timestamp=1 # Let InnoDB uses an adaptive algorithm to determine the optimal number of threads based on the available system resources and workload - Dynamic #innodb\_thread\_concurrency=0 # wait longer for client to read - Dynamic innodb\_sync\_spin\_loops=30 # Disabling the below parameter as the working set of data may go beyond 300G so we will let InnoDB storage engine dynamically load data into the buffer pool as needed innodb\_buffer\_pool\_populate=0 [mysqld\_safe] err-log=/mysqlData/log/mysqld.log ####################### # NUMA READY SYSTEM / PERCONA SERVER 5.5+ ####################### flush\_caches=1 numa\_interleave=1 Can someone help me here please? Note: I did the upgrade with database schemas without data and keeping the default my.cnf and it worked without any issues so I believe the issue is something related to the parameters changes. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [April 10, 2023, 9:41pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/2 "2023-04-10T21:41:29Z")
> 2023-04-10T18:22:54.830997Z 0 [ERROR] InnoDB: Upgrade after a crash is not supported. This redo log was created before MySQL 5.7.9, and we did not find a valid checkpoint. If you know your 5.6 shut down cleanly, then you should remove the InnoDB redo logs (ib\_logfile0, ib\_logfile1) and let 5.7 create new ones. **DO NOT DELETE ibdata1!!!** ---
**Author:** ![CTutte](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ctutte/32/1341_2.png) [@CTutte](https://forums.percona.com/u/CTutte)\ **Post date:** [April 11, 2023, 3:07pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/3 "2023-04-11T15:07:02Z")
before upgrading binaries to 5.7 , you should do a “slow\_shutdown” just in case set global innodb\_fast\_shutdown = 0; and then stop the 5.6 mysql server. reference: [https://dev.mysql.com/doc/refman/5.7/en/server-shutdown.html](https://dev.mysql.com/doc/refman/5.7/en/server-shutdown.html) Also as Matthew suggested, do not touch/modify the redo logs and let MySQL itself handle the modifications Regards ---
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 11, 2023, 4:16pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/4 "2023-04-11T16:16:12Z")
I did performed a slow shutdown and removed the redo logs as Matthew suggested. I started the upgrade again and found the following in the error logs : 2023-04-11T15:49:09.612940Z 0 [Warning] Could not increase number of max\_open\_files to more than 5000 (request: 40000) 2023-04-11T15:49:09.622196Z 0 [Warning] Changed limits: table\_open\_cache: 495 (requested 4096) 2023-04-11T15:49:09.823756Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.40-43-log) starting as process 38042 … 2023-04-11T15:49:09.855689Z 0 [Note] InnoDB: PUNCH HOLE support available 2023-04-11T15:49:09.855745Z 0 [Note] InnoDB: Mutexes and rw\_locks use GCC atomic builtins 2023-04-11T15:49:09.855755Z 0 [Note] InnoDB: Uses event mutexes 2023-04-11T15:49:09.855763Z 0 [Note] InnoDB: GCC builtin \_\_atomic\_thread\_fence() is used for memory barrier 2023-04-11T15:49:09.855771Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.12 2023-04-11T15:49:09.855782Z 0 [Note] InnoDB: Using Linux native AIO 2023-04-11T15:49:09.860148Z 0 [Note] InnoDB: Number of pools: 1 2023-04-11T15:49:09.870154Z 0 [Note] InnoDB: Using CPU crc32 instructions 2023-04-11T15:49:09.881608Z 0 [Note] InnoDB: Initializing buffer pool, total size = 300G, instances = 8, chunk size = 128M 2023-04-11T15:49:18.228082Z 0 [Note] InnoDB: Completed initialization of buffer pool 2023-04-11T15:49:20.210871Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-04-11T15:49:20.224403Z 0 [Note] InnoDB: Crash recovery did not find the parallel doublewrite buffer at /mysqlData/mysql/data/xb\_doublewrite 2023-04-11T15:49:20.224827Z 0 [Note] InnoDB: Setting log file ./ib\_logfile101 size to 1024 MB 2023-04-11T15:49:20.225436Z 0 [Note] InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000 2023-04-11T15:49:21.005893Z 0 [Note] InnoDB: Setting log file ./ib\_logfile1 size to 1024 MB 2023-04-11T15:49:21.006834Z 0 [Note] InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000 2023-04-11T15:49:22.265019Z 0 [Note] InnoDB: Renaming log file ./ib\_logfile101 to ./ib\_logfile0 2023-04-11T15:49:22.265173Z 0 [Warning] InnoDB: New log files created, LSN=1129178559769639 2023-04-11T15:49:22.265984Z 0 [Note] InnoDB: Highest supported file format is Barracuda. 2023-04-11T15:49:23.724381Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 1129178559770124 2023-04-11T15:49:23.724425Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 1129178559770133 2023-04-11T15:49:23.724431Z 0 [Note] InnoDB: Database was not shutdown normally! 2023-04-11T15:49:23.724435Z 0 [Note] InnoDB: Starting crash recovery. 2023-04-11T15:49:23.749197Z 0 [Note] InnoDB: Created parallel doublewrite buffer at /mysqlData/mysql/data/xb\_doublewrite, size 31457280 bytes 2023-04-11T15:49:24.149004Z 0 [Note] InnoDB: pre-5.7.2 rseg: 1 holds data to be purged. History length: 1. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.150783Z 0 [Note] InnoDB: pre-5.7.2 rseg: 8 holds data to be purged. History length: 2. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.151501Z 0 [Note] InnoDB: pre-5.7.2 rseg: 9 holds data to be purged. History length: 2. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.152381Z 0 [Note] InnoDB: pre-5.7.2 rseg: 11 holds data to be purged. History length: 1. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.153282Z 0 [Note] InnoDB: pre-5.7.2 rseg: 13 holds data to be purged. History length: 2. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.153908Z 0 [Note] InnoDB: pre-5.7.2 rseg: 14 holds data to be purged. History length: 3. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.154694Z 0 [Note] InnoDB: pre-5.7.2 rseg: 15 holds data to be purged. History length: 2. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.155449Z 0 [Note] InnoDB: pre-5.7.2 rseg: 16 holds data to be purged. History length: 1. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.156180Z 0 [Note] InnoDB: pre-5.7.2 rseg: 18 holds data to be purged. History length: 20. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.156863Z 0 [Note] InnoDB: pre-5.7.2 rseg: 19 holds data to be purged. History length: 18. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.158410Z 0 [Note] InnoDB: pre-5.7.2 rseg: 22 holds data to be purged. History length: 3. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.159108Z 0 [Note] InnoDB: pre-5.7.2 rseg: 23 holds data to be purged. History length: 2. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.159807Z 0 [Note] InnoDB: pre-5.7.2 rseg: 24 holds data to be purged. History length: 17. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.160440Z 0 [Note] InnoDB: pre-5.7.2 rseg: 25 holds data to be purged. History length: 1. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.161192Z 0 [Note] InnoDB: pre-5.7.2 rseg: 26 holds data to be purged. History length: 3. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.162006Z 0 [Note] InnoDB: pre-5.7.2 rseg: 28 holds data to be purged. History length: 1. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.162695Z 0 [Note] InnoDB: pre-5.7.2 rseg: 29 holds data to be purged. History length: 8. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.163447Z 0 [Note] InnoDB: pre-5.7.2 rseg: 30 holds data to be purged. History length: 18. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.164090Z 0 [Note] InnoDB: pre-5.7.2 rseg: 31 holds data to be purged. History length: 1. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:24.164740Z 0 [Note] InnoDB: pre-5.7.2 rseg: 32 holds data to be purged. History length: 17. Recommend slow shutdown with innodb\_fast\_shutdown=0 and restart 2023-04-11T15:49:26.796083Z 0 [Note] InnoDB: Last MySQL binlog file position 0 239022932, file name /data/mysqlbinlog/va-db02-binlog.001308 2023-04-11T15:49:27.508567Z 0 [Note] InnoDB: Successfully reset 12 pre-5.7.2 rseg slots. 2023-04-11T15:49:27.508931Z 0 [Note] InnoDB: Removed temporary tablespace data file: “ibtmp1” 2023-04-11T15:49:27.508949Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-04-11T15:49:27.509020Z 0 [Note] InnoDB: Setting file ‘./ibtmp1’ size to 12 MB. Physically writing the file full; Please wait … 2023-04-11T15:49:27.517697Z 0 [Note] InnoDB: File ‘./ibtmp1’ size is now 12 MB. 2023-04-11T15:49:27.519548Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. 2023-04-11T15:49:27.519569Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active. 2023-04-11T15:49:27.519714Z 0 [Note] InnoDB: page\_cleaner: 1000ms intended loop took 7309ms. The settings might not be optimal. (flushed=0, during the time.) 2023-04-11T15:49:27.519789Z 0 [Note] InnoDB: Creating sys\_virtual system tables. 2023-04-11T15:49:27.524743Z 0 [Note] InnoDB: sys\_virtual table created 2023-04-11T15:49:27.526506Z 0 [Note] InnoDB: Waiting for purge to start 2023-04-11T15:49:27.576627Z 0 [Note] InnoDB: Percona XtraDB ([http://www.percona.com](http://www.percona.com)) 5.7.40-43 started; log sequence number 1129178559770133 2023-04-11T15:49:27.576865Z 0 [Note] InnoDB: Loading buffer pool(s) from /mysqlData/mysql/data/ib\_buffer\_pool **2023-04-11T15:49:27.576932Z 0 [ERROR] InnoDB: Cannot open ‘/mysqlData/mysql/data/ib\_buffer\_pool’ for reading: No such file or directory** 2023-04-11T15:49:27.577130Z 0 [Note] Plugin ‘FEDERATED’ is disabled. 2023-04-11T15:49:27.589502Z 0 [Warning] System table ‘plugin’ is expected to be transactional. **2023-04-11T15:49:27.590607Z 0 [ERROR] unknown variable ‘innodb\_buffer\_pool\_populate=0’** **2023-04-11T15:49:27.590624Z 0 [ERROR] Aborting** 2023-04-11T15:49:27.590663Z 0 [Note] Binlog end 2023-04-11T15:49:27.590751Z 0 [Note] Shutting down plugin ‘ngram’ 2023-04-11T15:49:27.590760Z 0 [Note] Shutting down plugin ‘partition’ 2023-04-11T15:49:27.590763Z 0 [Note] Shutting down plugin ‘BLACKHOLE’ 2023-04-11T15:49:27.590775Z 0 [Note] Shutting down plugin ‘ARCHIVE’ 2023-04-11T15:49:27.590779Z 0 [Note] Shutting down plugin ‘INNODB\_TABLESPACES\_SCRUBBING’ 2023-04-11T15:49:27.590782Z 0 [Note] Shutting down plugin ‘INNODB\_TABLESPACES\_ENCRYPTION’ 2023-04-11T15:49:27.590784Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_VIRTUAL’ 2023-04-11T15:49:27.590787Z 0 [Note] Shutting down plugin ‘INNODB\_CHANGED\_PAGES’ 2023-04-11T15:49:27.590790Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_DATAFILES’ 2023-04-11T15:49:27.590793Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_TABLESPACES’ 2023-04-11T15:49:27.590795Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_FOREIGN\_COLS’ 2023-04-11T15:49:27.590798Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_FOREIGN’ 2023-04-11T15:49:27.590801Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_FIELDS’ 2023-04-11T15:49:27.590804Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_COLUMNS’ 2023-04-11T15:49:27.590807Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_INDEXES’ 2023-04-11T15:49:27.590809Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_TABLESTATS’ 2023-04-11T15:49:27.590812Z 0 [Note] Shutting down plugin ‘INNODB\_SYS\_TABLES’ 2023-04-11T15:49:27.590815Z 0 [Note] Shutting down plugin ‘INNODB\_FT\_INDEX\_TABLE’ 2023-04-11T15:49:27.590818Z 0 [Note] Shutting down plugin ‘INNODB\_FT\_INDEX\_CACHE’ 2023-04-11T15:49:27.590820Z 0 [Note] Shutting down plugin ‘INNODB\_FT\_CONFIG’ 2023-04-11T15:49:27.590823Z 0 [Note] Shutting down plugin ‘INNODB\_FT\_BEING\_DELETED’ 2023-04-11T15:49:27.590826Z 0 [Note] Shutting down plugin ‘INNODB\_FT\_DELETED’ 2023-04-11T15:49:27.590828Z 0 [Note] Shutting down plugin ‘INNODB\_FT\_DEFAULT\_STOPWORD’ 2023-04-11T15:49:27.590831Z 0 [Note] Shutting down plugin ‘INNODB\_METRICS’ 2023-04-11T15:49:27.590834Z 0 [Note] Shutting down plugin ‘INNODB\_TEMP\_TABLE\_INFO’ 2023-04-11T15:49:27.590837Z 0 [Note] Shutting down plugin ‘INNODB\_BUFFER\_POOL\_STATS’ 2023-04-11T15:49:27.590839Z 0 [Note] Shutting down plugin ‘INNODB\_BUFFER\_PAGE\_LRU’ 2023-04-11T15:49:27.590842Z 0 [Note] Shutting down plugin ‘INNODB\_BUFFER\_PAGE’ 2023-04-11T15:49:27.590845Z 0 [Note] Shutting down plugin ‘INNODB\_CMP\_PER\_INDEX\_RESET’ 2023-04-11T15:49:27.590848Z 0 [Note] Shutting down plugin ‘INNODB\_CMP\_PER\_INDEX’ 2023-04-11T15:49:27.590850Z 0 [Note] Shutting down plugin ‘INNODB\_CMPMEM\_RESET’ 2023-04-11T15:49:27.590853Z 0 [Note] Shutting down plugin ‘INNODB\_CMPMEM’ 2023-04-11T15:49:27.590856Z 0 [Note] Shutting down plugin ‘INNODB\_CMP\_RESET’ 2023-04-11T15:49:27.590858Z 0 [Note] Shutting down plugin ‘INNODB\_CMP’ 2023-04-11T15:49:27.590861Z 0 [Note] Shutting down plugin ‘INNODB\_LOCK\_WAITS’ 2023-04-11T15:49:27.590864Z 0 [Note] Shutting down plugin ‘INNODB\_LOCKS’ 2023-04-11T15:49:27.590867Z 0 [Note] Shutting down plugin ‘INNODB\_TRX’ 2023-04-11T15:49:27.590869Z 0 [Note] Shutting down plugin ‘XTRADB\_ZIP\_DICT\_COLS’ 2023-04-11T15:49:27.590872Z 0 [Note] Shutting down plugin ‘XTRADB\_ZIP\_DICT’ 2023-04-11T15:49:27.590875Z 0 [Note] Shutting down plugin ‘XTRADB\_RSEG’ 2023-04-11T15:49:27.590878Z 0 [Note] Shutting down plugin ‘XTRADB\_INTERNAL\_HASH\_TABLES’ 2023-04-11T15:49:27.590880Z 0 [Note] Shutting down plugin ‘XTRADB\_READ\_VIEW’ 2023-04-11T15:49:27.590883Z 0 [Note] Shutting down plugin ‘InnoDB’ 2023-04-11T15:49:27.590947Z 0 [Note] InnoDB: FTS optimize thread exiting. 2023-04-11T15:49:27.591065Z 0 [Note] InnoDB: Starting shutdown… 2023-04-11T15:49:27.691269Z 0 [Note] InnoDB: Dumping buffer pool(s) to /mysqlData/mysql/data/ib\_buffer\_pool 2023-04-11T15:49:27.691582Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 230411 11:49:27 2023-04-11T15:49:28.692196Z 0 [Note] InnoDB: Waiting for page\_cleaner to finish flushing of buffer pool 2023-04-11T15:49:33.442952Z 0 [Note] InnoDB: Shutdown completed; log sequence number 1129178559795442 2023-04-11T15:49:33.443108Z 0 [Note] InnoDB: Removed temporary tablespace data file: “ibtmp1” 2023-04-11T15:49:33.443139Z 0 [Note] Shutting down plugin ‘MEMORY’ 2023-04-11T15:49:33.443145Z 0 [Note] Shutting down plugin ‘MRG\_MYISAM’ 2023-04-11T15:49:33.443161Z 0 [Note] Shutting down plugin ‘CSV’ 2023-04-11T15:49:33.443167Z 0 [Note] Shutting down plugin ‘MyISAM’ 2023-04-11T15:49:33.443185Z 0 [Note] Shutting down plugin ‘PERFORMANCE\_SCHEMA’ 2023-04-11T15:49:33.443255Z 0 [Note] Shutting down plugin ‘sha256\_password’ 2023-04-11T15:49:33.443258Z 0 [Note] Shutting down plugin ‘mysql\_native\_password’ 2023-04-11T15:49:33.443460Z 0 [Note] Shutting down plugin ‘binlog’ 2023-04-11T15:49:33.443646Z 0 [Note] /usr/sbin/mysqld: Shutdown complete 2023-04-11T15:49:34.188435Z 0 [Warning] Could not increase number of max\_open\_files to more than 5000 (request: 40000) 2023-04-11T15:49:34.195301Z 0 [Warning] Changed limits: table\_open\_cache: 495 (requested 4096) 2023-04-11T15:49:34.374386Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.40-43-log) starting as process 38199 … 2023-04-11T15:49:34.386321Z 0 [Note] InnoDB: PUNCH HOLE support available 2023-04-11T15:49:34.386352Z 0 [Note] InnoDB: Mutexes and rw\_locks use GCC atomic builtins 2023-04-11T15:49:34.386357Z 0 [Note] InnoDB: Uses event mutexes 2023-04-11T15:49:34.386362Z 0 [Note] InnoDB: GCC builtin \_\_atomic\_thread\_fence() is used for memory barrier 2023-04-11T15:49:34.386366Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.12 2023-04-11T15:49:34.386370Z 0 [Note] InnoDB: Using Linux native AIO 2023-04-11T15:49:34.388656Z 0 [Note] InnoDB: Number of pools: 1 2023-04-11T15:49:34.398373Z 0 [Note] InnoDB: Using CPU crc32 instructions 2023-04-11T15:49:34.408443Z 0 [Note] InnoDB: Initializing buffer pool, total size = 300G, instances = 8, chunk size = 128M 2023-04-11T15:49:42.595796Z 0 [Note] InnoDB: Completed initialization of buffer pool 2023-04-11T15:49:44.528127Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2023-04-11T15:49:44.541628Z 0 [Note] InnoDB: Crash recovery did not find the parallel doublewrite buffer at /mysqlData/mysql/data/xb\_doublewrite 2023-04-11T15:49:44.542773Z 0 [Note] InnoDB: Highest supported file format is Barracuda. 2023-04-11T15:49:46.229496Z 0 [Note] InnoDB: Created parallel doublewrite buffer at /mysqlData/mysql/data/xb\_doublewrite, size 31457280 bytes 2023-04-11T15:49:49.007051Z 0 [Note] InnoDB: Successfully reset 20 pre-5.7.2 rseg slots. 2023-04-11T15:49:49.007099Z 0 [Note] InnoDB: Creating shared tablespace for temporary tables 2023-04-11T15:49:49.007145Z 0 [Note] InnoDB: Setting file ‘./ibtmp1’ size to 12 MB. Physically writing the file full; Please wait … 2023-04-11T15:49:49.013591Z 0 [Note] InnoDB: File ‘./ibtmp1’ size is now 12 MB. 2023-04-11T15:49:49.014330Z 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active. 2023-04-11T15:49:49.014340Z 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active. 2023-04-11T15:49:49.014465Z 0 [Note] InnoDB: page\_cleaner: 1000ms intended loop took 4487ms. The settings might not be optimal. (flushed=0, during the time.) 2023-04-11T15:49:49.015672Z 0 [Note] InnoDB: Waiting for purge to start 2023-04-11T15:49:49.066026Z 0 [Note] InnoDB: Percona XtraDB ([http://www.percona.com](http://www.percona.com)) 5.7.40-43 started; log sequence number 1129178559795442 2023-04-11T15:49:49.066739Z 0 [Note] InnoDB: Loading buffer pool(s) from /mysqlData/mysql/data/ib\_buffer\_pool 2023-04-11T15:49:49.066974Z 0 [Note] Plugin ‘FEDERATED’ is disabled. 2023-04-11T15:49:49.073320Z 0 [Warning] System table ‘plugin’ is expected to be transactional. **2023-04-11T15:49:49.073848Z 0 [ERROR] unknown variable ‘innodb\_buffer\_pool\_populate=0’** **2023-04-11T15:49:49.073865Z 0 [ERROR] Aborting** I did commented out the parameter innodb\_buffer\_pool\_populate in the my.cnf file and tried starting the service and this time the service got started. I executed the **mysql\_upgrade** checker utility to check for any discrepancies and what I noticed is that the mysql tables are all good, however for many application tables I am getting error message : **error : Table rebuild required. Please do “ALTER TABLE `AccessLevel` FORCE” or dump/reload to fix it!** : mysql\_upgrade -u root -p Enter password: Checking if update is needed. Checking server version. Running queries to upgrade MySQL server. Checking system database. mysql.columns\_priv OK mysql.db OK mysql.engine\_cost OK mysql.event OK mysql.func OK mysql.general\_log OK mysql.gtid\_executed OK mysql.help\_category OK mysql.help\_keyword OK mysql.help\_relation OK mysql.help\_topic OK mysql.host OK mysql.innodb\_index\_stats OK mysql.innodb\_table\_stats OK mysql.ndb\_binlog\_index OK mysql.plugin OK mysql.proc OK mysql.procs\_priv OK mysql.proxies\_priv OK mysql.server\_cost OK mysql.servers OK mysql.slave\_master\_info OK mysql.slave\_relay\_log\_info OK mysql.slave\_worker\_info OK mysql.slow\_log OK mysql.tables\_priv OK mysql.time\_zone OK mysql.time\_zone\_leap\_second OK mysql.time\_zone\_name OK mysql.time\_zone\_transition OK mysql.time\_zone\_transition\_type OK mysql.user OK mysql.user\_info OK Upgrading the sys schema. Checking databases. dbname.AccessLevel error : Table rebuild required. Please do “ALTER TABLE `AccessLevel` FORCE” or dump/reload to fix it! dbname.Action error : Table rebuild required. Please do “ALTER TABLE `Action` FORCE” or dump/reload to fix it! dbname.ActionConversion error : Table rebuild required. Please do “ALTER TABLE `ActionConversion` FORCE” or dump/reload to fix it! dbname.ActionPageGroupMapping error : Table rebuild required. Please do “ALTER TABLE `ActionPageGroupMapping` FORCE” or dump/reload to fix it! We have 1000+ tables and this issue is with ~30% of the tables. Any suggestions on this ? ---
**Author:** ![CTutte](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ctutte/32/1341_2.png) [@CTutte](https://forums.percona.com/u/CTutte)\ **Post date:** [April 11, 2023, 4:33pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/5 "2023-04-11T16:33:07Z")
Hi again, Please take a look at this blogpost: [A Little Trick Upgrading to MySQL 5.7](https://www.percona.com/blog/little-trick-upgrading-mysql-5-7/) You will need to rebuild all the tables Regards ---
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 11, 2023, 4:54pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/6 "2023-04-11T16:54:37Z")
Thank you for your assistance @CTutte and @matthewb , I will go through the same and update here with the end result. ---
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 12, 2023, 8:37pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/7 "2023-04-12T20:37:54Z")
Hi Carlos, I referred the article : [https://www.percona.com/blog/little-trick-upgrading-mysql-5-7/](https://www.percona.com/blog/little-trick-upgrading-mysql-5-7/) and tried the mysqlcheck --check-upgrade --all-databases \> mysql-check.log, however, the list shows me that majority of the tables requires rebuild. Now this seems weird because I have created the replicas that are up and running using the same data that I used on the staging environment then why it is asking me to do a table repair. I am looking for a way wherein the these tables repair can be avoided because our database size is ~700G and it took me ~5 hours when I had kept the mysql\_upgrade running as it is. Any help is appreciated. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [April 12, 2023, 8:56pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/8 "2023-04-12T20:56:42Z")
> [@MahendraB](#): > > then why it is asking me to do a table repair. There are on-disk changes between 5.6 and 5.7. It is not required to rebuild them, but it is best practice to get optimal performance out of MySQL. ---
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 12, 2023, 10:18pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/9 "2023-04-12T22:18:42Z")
Hi Matthew, How do I skip the rebuild part then as mysql\_upgrade checker utility does it automatically which checking for any incompatibilities. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [April 13, 2023, 4:05am UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/10 "2023-04-13T04:05:43Z")
Check --help. There is an option to only process system tables. ---
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 13, 2023, 3:06pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/11 "2023-04-13T15:06:07Z")
Thank you for your help Matthew 🙂 , I used the flag : --upgrade-system-tables with mysql\_upgrade and it worked well. The --upgrade-system-tables option tells mysql\_upgrade to upgrade only the system tables and skip the application tables. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [April 13, 2023, 3:11pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/12 "2023-04-13T15:11:18Z")
Now that you are on 5.7, you need to schedule time to actually fix all those tables so that you can be ready for MySQL 8. Remember, 5.7 EOL in Oct’23 ---
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 13, 2023, 3:31pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/13 "2023-04-13T15:31:50Z")
Hi Matthew, I don’t understand why did we get the error ==\> error : Table rebuild required. Please do “ALTER TABLE `tablename` FORCE” or dump/reload to fix it! I am trying to understand the root cause for this error. Also, what if I am not rebuilding those, will I not be able to upgrade to MySQL 8 ? The reason why I am asking this is because the tables are working well in the current MySQL 5.6 version. ---
**Author:** ![matthewb](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/matthewb/32/34_2.png) [@matthewb](https://forums.percona.com/u/matthewb)\ **Post date:** [April 13, 2023, 4:34pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/14 "2023-04-13T16:34:58Z")
@MahendraB, You got the error because, as I said above, there are on-disk format changes between 5.6 and 5.7. Yes, your old 5.6-format will work in 5.7 but it won’t be optimal. No, you won’t be able to upgrade to 8 until you “upgrade” the tables to the 5.7 format. Look at using [pt-online-schema-change](https://docs.percona.com/percona-toolkit/pt-online-schema-change.html) to accomplish this. ---
**Author:** ![MahendraB](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/mahendrab/32/9108_2.png) [@MahendraB](https://forums.percona.com/u/MahendraB)\ **Post date:** [April 13, 2023, 4:44pm UTC](https://forums.percona.com/t/unable-to-perform-percona-mysql-version-upgrade-to-5-7/21301/15 "2023-04-13T16:44:31Z")
Thanks again Matthew, much appreciated. I will go through the options for this.