Problems with starting the mysql.service

Hi, 
I hope someone can help me address the issue I’m stuck with. I must create another MySQL slave instance similar to the one which has been working for a long time already. The only difference is that the old one is “mysql  Ver 8.0.19 for Linux on x86_64 (MySQL Community Server - GPL)”, while the new one is “mysql  Ver 8.0.19-10 for debian-linux-gnu on x86_64 (Percona Server (GPL), Release ‘10’, Revision ‘f446c04’)”. 

So, what’s the problem? The problem is that I can’t start MySQL service on the new Percona Server because of errors “Data Dictionary initialization failed” and “Failed to find valid data directory”. It’s no problem to start the server when the datadir is empty, but it fails when I add files there. Below the steps to reproduce:

1) I use SSHFS on the server 1 with MySQL Community Server to mount a remote file system and forward dumps directly to the server 2 with Percona Server 
2) After that I use ‘xtrabackup --backup --target-dir=/directory/’ followed by ‘xtrabackup --prepare --target-dir=/directory/’
3) When everything is ready, I change permissions and ownership on the server 2 and start the service.
4) The start up fails, so I run “mysqld --initialize”, but it doesn’t help.

The log:
2020-06-19T07:35:48.460188Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.2020-06-19T07:35:48.460235Z 0 [ERROR] [MY-010119] [Server] Aborting2020-06-19T07:35:48.460675Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.19-10)  Percona Server (GPL), Release ‘10’, Revision ‘f446c04’.2020-06-19T07:42:59.701013Z 0 [Warning] [MY-011070] [Server] ‘Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it’ is deprecated and will be removed in a future release.2020-06-19T07:42:59.701032Z 0 [Warning] [MY-010915] [Server] ‘NO_ZERO_DATE’, ‘NO_ZERO_IN_DATE’ and ‘ERROR_FOR_DIVISION_BY_ZERO’ sql modes should be used with strict mode. They will be merged with strict mode in a future release.2020-06-19T07:42:59.702003Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.19-10) starting as process 90222020-06-19T07:42:59.707410Z 1 [ERROR] [MY-011011] [Server] Failed to find valid data directory.2020-06-19T07:42:59.707504Z 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.2020-06-19T07:42:59.707546Z 0 [ERROR] [MY-010119] [Server] Aborting2020-06-19T07:42:59.708023Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.19-10)  Percona Server (GPL), Release ‘10’, Revision ‘f446c04’.

@alexmnsk
 try to start explicitly by providing datadir path and default conf path. 
sudo mysqld --initialize --data-dir=/x/x/ 

also just cross verify the variables mentioned in the cnf file are compatible and supported by the version. 

@vaibhav_upadhyay40 I did this multiple times, but it doesn’t help when I have data in the datadir. I can’t remove 1.1T, it doesn’t make sense for me. Is there any way to handle it without removing data?

can please you share below :

output for command :   
sudo mysqld --initialize --data-dir=/1tb/data_dir/path/
mysql error logs
OS version details