XtraDB 5.7 Cluster ZFS on Linux and parallel doublewrite path as 0_DIRECT

MySQL & MariaDB Percona XtraDB Cluster 5.x
1

We are moving from a legacy MySQL 5.5 Master/Slave environment to 3 node Percona XtraDB 5.7 cluster. We have been using ZFS on Linux for hosting MySQL database file for many years now. We heavily rely on snapshots and have not seen any performance or stability issues with the ZFS filesystem.

In our initial testing we have found the below error in our server logs

[Warning] InnoDB: Failed to set O_DIRECT on file/zfspool/mysql/innodb/xb_doublewrite;create: Invalid argument, continuing anyway. O_DIRECT is known to result in ‘Invalid argument’ on Linux on tmpfs, see MySQL Bug#26662

From this link [url]https://www.percona.com/doc/percona-server/5.7/performance/xtradb_performance_improvements_for_io-bound_highly-concurrent_workloads.html[/url] there is this snippet “Regardless of innodb_flush_method setting, the parallel doublewrite file is opened with O_DIRECT flag to remove OS caching, then its access is further governed by the exact value set:”

We know that ZFS on Linux does not support O_DIRECT and that Percona 5.7 only tries to open innodb_parallel_doublewrite_path as O_DIRECT. So this error makes sense and there does not seem to be anything simple I can do about it.

There 2 questions I have are:

(1) Is this going to be a problem for us. Will Mysql simply open the parallel double write file with another method and go on happily. I am not sure what is actually going on because the message only says that it failed O_DIRECT not that it succeeded with some other method?

(2) Does anyone know of a reason NOT to use ZFS for Percona XtraDB cluster in a general sense.

Thanks!

2

Firstly, as ZFS supports atomic writes, you may actually disable innodb_doublewrite.

Secondly, I am afraid we did observe worse ZFS (on Linux) performance then for example XFS or EXT4. For example: [url]https://www.percona.com/blog/2013/05/24/zfs-on-linux-and-mysql/[/url]
I wonder if you did compare it with one of these by doing some benchmark?

And indeed O_DIRECT seems to be not supported by ZFS. I think that in such case flush method fallbacks to default and succeeds anyway, just the warming is confusing: [url]MySQL Bugs: #70298: [ERROR] InnoDB: Failed to set O_DIRECT on file

Hope this helps.