# Upgrade MySQL with millions tables **URL:** \ **Category:** Percona Server for MySQL 8.0\ **Created:** [November 26, 2020, 3:31am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315 "2020-11-26T03:31:42Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![demidov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/demidov/32/209_2.png) [@demidov](https://forums.percona.com/u/demidov)\ **Post date:** [November 26, 2020, 3:31am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/1 "2020-11-26T03:31:42Z")
Hello, I want to upgrade Percona Server for MySQL from 8.0.20 to 8.0.21. My server contains 3000 databases with 1000 tables in each database. Yes, I know this is a rare configuration. It is a bit like shared hosting. After starting mysqld on the new version, I see in the log for about 20 minutes: 2020-11-25T18:27:27.571354Z 1 [System] [MY-011090] [Server] Data dictionary upgrading from version ‘80017’ to ‘80021’. And then all the available memory and swap on the server ends, and finally the mysqld process was killed by OOM Killer. Then everything starts all over again. How can I upgrade a server with millions of tables? ---
**Author:** ![Ivan\_Groenewold](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/ivan_groenewold/32/6299_2.png) [@Ivan\_Groenewold](https://forums.percona.com/u/Ivan_Groenewold)\ **Post date:** [November 26, 2020, 6:01am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/2 "2020-11-26T06:01:21Z")
Hi @demidov I am sorry to hear you are running into issues. I suggest you open a bug in our issue tracker so [https://jira.percona.com/projects/PS](https://jira.percona.com/projects/PS/issues/PS-7440?filter=allopenissues) about this ---
**Author:** ![demidov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/demidov/32/209_2.png) [@demidov](https://forums.percona.com/u/demidov)\ **Post date:** [November 26, 2020, 1:28pm UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/3 "2020-11-26T13:28:50Z")
Hi @igroene ! I took a bigger server - with 128 GB of memory. And tryed one more time. At its peak, the mysqld process took about 106 GB. And in the end it was completed in about 1 hour. 2020-11-26T17:38:46.113695Z 1 [System] [MY-011090] [Server] Data dictionary upgrading from version ‘80017’ to ‘80021’. 2020-11-26T18:41:38.549123Z 1 [System] [MY-013413] [Server] Data dictionary upgrade from version ‘80017’ to ‘80021’ completed. Is this normal? If ok, then ok. If not, should I open a bug? ---
**Author:** ![vadimtk](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vadimtk/32/9528_2.png) [@vadimtk](https://forums.percona.com/u/vadimtk)\ **Post date:** [November 26, 2020, 3:37pm UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/4 "2020-11-26T15:37:20Z")
@demidov This is probably normal given amount of tables you have, but I understand the inconvenience. I probably would try to play with [table\_definition\_cache](https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_table_definition_cache) What happens if set it to 10000. Would upgrade perform normally? How much memory it would take in this case, etc… ---
**Author:** ![demidov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/demidov/32/209_2.png) [@demidov](https://forums.percona.com/u/demidov)\ **Post date:** [November 27, 2020, 3:30am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/5 "2020-11-27T03:30:59Z")
@vadimtk In my example above it was: table\_definition\_cache = 30000 I could try to increase this value. ---
**Author:** ![vadimtk](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/vadimtk/32/9528_2.png) [@vadimtk](https://forums.percona.com/u/vadimtk)\ **Post date:** [November 27, 2020, 8:42am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/6 "2020-11-27T08:42:07Z")
actually I would try to decrease that value if the memory consumption is still big ---
**Author:** ![Peter](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/peter/32/2_2.png) [@Peter](https://forums.percona.com/u/Peter)\ **Post date:** [November 27, 2020, 9:43am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/7 "2020-11-27T09:43:07Z")
Interesting if table\_definition\_cache changes it or if data dictionary upgrade operation is not memory efficient with large number of tables. If it were to be the case it is likely upstream MySQL issue. ---
**Author:** ![demidov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/demidov/32/209_2.png) [@demidov](https://forums.percona.com/u/demidov)\ **Post date:** [November 28, 2020, 5:37am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/8 "2020-11-28T05:37:22Z")
@vadimtk , @Peter table\_definition\_cache = 5000 Nothing has changed with this value. Memory consumption is about the same. ---
**Author:** ![lalit.choudhary](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/lalit.choudhary/32/51_2.png) [@lalit.choudhary](https://forums.percona.com/u/lalit.choudhary)\ **Post date:** [December 1, 2020, 2:14am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/9 "2020-12-01T02:14:18Z")
@demidov what type of storage you are using ssd vs hdd ? what the total startup time for mysqld in general ? I’m testing it with ~ 2M innodb tables ( 2000 database with 1000 tables in each) ---
**Author:** ![demidov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/demidov/32/209_2.png) [@demidov](https://forums.percona.com/u/demidov)\ **Post date:** [December 1, 2020, 8:34am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/10 "2020-12-01T08:34:06Z")
@lalit.choudhary SSD. And normal startup time is about 1 minute. ---
**Author:** ![lalit.choudhary](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/lalit.choudhary/32/51_2.png) [@lalit.choudhary](https://forums.percona.com/u/lalit.choudhary)\ **Post date:** [December 1, 2020, 8:43am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/11 "2020-12-01T08:43:00Z")
@demidov Thank you for the details. I see issue in my test (1M tables,1000dbs with 1000 tables in each)) after upgrading from PS 8.0.20 to 8.0.21. Using SSD storage. after upgrade 1st Startup in progress and with time it’s consuming more memory. I can see memory reducing on OS while Data dictionary upgrading from version ‘80017’ to ‘80021’. in progress. I will test with upsteam as well and report this issue. I will update Bug# here later. ---
**Author:** ![lalit.choudhary](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/lalit.choudhary/32/51_2.png) [@lalit.choudhary](https://forums.percona.com/u/lalit.choudhary)\ **Post date:** [December 1, 2020, 10:20am UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/12 "2020-12-01T10:20:05Z")
Here are the bug report for reference. [https://jira.percona.com/browse/PS-7446](https://jira.percona.com/browse/PS-7446) [https://bugs.mysql.com/bug.php?id=101818](https://bugs.mysql.com/bug.php?id=101818) Looks like you already reported this to upstream. My upstream version test still running i will update upstream bug with my test results. ---
**Author:** ![demidov](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/demidov/32/209_2.png) [@demidov](https://forums.percona.com/u/demidov)\ **Post date:** [December 1, 2020, 12:20pm UTC](https://forums.percona.com/t/upgrade-mysql-with-millions-tables/8315/13 "2020-12-01T12:20:41Z")
@lalit.choudhary Thank you!