Sudden error 1045 after server reboot

We’ve been running Percona on our server for 2 years now. No issues until today. Our hosting company has detected our CPU was overheating and they’ve fixed the cooling. To do that they had to shut the server down. After starting it up again, something is not right but we can’t pinpoint what. I’ll try to write what we currently know so any ideas are welcome.

service mysql stop fails.

It fails when while executing

/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf shutdown

The full output is:

++ /usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf shutdown
+ shutdown_out='/usr/bin/mysqladmin: connect to server at '\''localhost'\'' failed
error: '\''Access denied for user '\''root'\''@'\''localhost'\'' (using password: NO)'\'''

If I execute “mysqladmin shutdown”, it shuts down properly and I can then do service mysql stop. When starting it back up I get this:

root@xxx001:~# service mysql start
[ ok ] Starting MySQL (Percona Server) database server: mysqld.
[info] Checking for corrupt, not cleanly closed and upgrade needing tables..
root@xxx001:~# ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)

Our root has a password. If I access via mysql -u root -p and enter the password I enter mysql properly and have full access to all databases and tables. MySQL Workbench, setup to connect via SSL connects successfully too.

~/.my.cnf has root passwords in it for [mysqladmin], [client], [mysql], [mysqldump] and [mysqldiff].

/etc/mysql/ contains:

drwxr-xr-x 3 root root 4096 Jul 12 11:23 .
drwxr-xr-x 89 root root 4096 May 16 10:42 ..
drwxr-xr-x 2 root root 4096 Nov 2 2015 conf.d
-rw------- 1 root root 0 Nov 2 2015 debian.cnf
-rwxr-xr-x 1 root root 1286 Nov 2 2015 debian-start
-rw-r--r-- 1 root root 3646 Nov 2 2015 my.cnf

So everything we’ve checked is there. 2 days ago we did service mysql restart and it restarted successfully. No problem with the ‘root’ access. Now it suddenly has it and we can’t resolve it to get our sites back to life.

Sorted out. debian.cnf was empty, but config was inside before. Somehow the file got cleaned of contents when the server was shut off and date of the last change remained the same.