# XtraDB  5.7 Cluster ZFS on Linux and parallel doublewrite path as 0\_DIRECT

**URL:** <https://forums.percona.com/t/xtradb-5-7-cluster-zfs-on-linux-and-parallel-doublewrite-path-as-0-direct/5468>\
**Category:** Percona XtraDB Cluster 5.x\
**Created:** [March 9, 2017, 2:05pm UTC](https://forums.percona.com/t/xtradb-5-7-cluster-zfs-on-linux-and-parallel-doublewrite-path-as-0-direct/5468 "2017-03-09T14:05:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mselby](https://avatars.discourse-cdn.com/v4/letter/m/ea5d25/32.png) [@mselby](https://forums.percona.com/u/mselby)\
**Post date:** [March 9, 2017, 2:05pm UTC](https://forums.percona.com/t/xtradb-5-7-cluster-zfs-on-linux-and-parallel-doublewrite-path-as-0-direct/5468/1 "2017-03-09T14:05:00Z")

</div>

We are moving from a legacy MySQL 5.5 Master/Slave environment to 3 node Percona XtraDB 5.7 cluster. We have been using ZFS on Linux for hosting MySQL database file for many years now. We heavily rely on snapshots and have not seen any performance or stability issues with the ZFS filesystem.

In our initial testing we have found the below error in our server logs

[Warning] InnoDB: Failed to set O\_DIRECT on file/zfspool/mysql/innodb/xb\_doublewrite;create: Invalid argument, continuing anyway. O\_DIRECT is known to result in ‘Invalid argument’ on Linux on tmpfs, see MySQL Bug#26662

From this link [url][https://www.percona.com/doc/percona-server/5.7/performance/xtradb\_performance\_improvements\_for\_io-bound\_highly-concurrent\_workloads.html[/url]](https://www.percona.com/doc/percona-server/5.7/performance/xtradb_performance_improvements_for_io-bound_highly-concurrent_workloads.html%5B/url%5D) there is this snippet “Regardless of innodb\_flush\_method setting, the parallel doublewrite file is opened with O\_DIRECT flag to remove OS caching, then its access is further governed by the exact value set:”

We know that ZFS on Linux does not support O\_DIRECT and that Percona 5.7 only tries to open [innodb\_parallel\_doublewrite\_path](https://www.percona.com/doc/percona-server/5.7/performance/xtradb_performance_improvements_for_io-bound_highly-concurrent_workloads.html#innodb_parallel_doublewrite_path) as O\_DIRECT. So this error makes sense and there does not seem to be anything simple I can do about it.

There 2 questions I have are:

(1) Is this going to be a problem for us. Will Mysql simply open the parallel double write file with another method and go on happily. I am not sure what is actually going on because the message only says that it failed O\_DIRECT not that it succeeded with some other method?

(2) Does anyone know of a reason NOT to use ZFS for Percona XtraDB cluster in a general sense.

Thanks!

---

<div class="post-metadata">

**Author:** ![przemek](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/przemek/32/3_2.png) [@przemek](https://forums.percona.com/u/przemek)\
**Post date:** [March 30, 2017, 3:01pm UTC](https://forums.percona.com/t/xtradb-5-7-cluster-zfs-on-linux-and-parallel-doublewrite-path-as-0-direct/5468/2 "2017-03-30T15:01:08Z")

</div>

Firstly, as ZFS supports atomic writes, you may actually disable innodb\_doublewrite.

Secondly, I am afraid we did observe worse ZFS (on Linux) performance then for example XFS or EXT4. For example: [url][https://www.percona.com/blog/2013/05/24/zfs-on-linux-and-mysql/[/url]](https://www.percona.com/blog/2013/05/24/zfs-on-linux-and-mysql/%5B/url%5D)  
I wonder if you did compare it with one of these by doing some benchmark?

And indeed O\_DIRECT seems to be not supported by ZFS. I think that in such case flush method fallbacks to default and succeeds anyway, just the warming is confusing: [url][MySQL Bugs: #70298: [ERROR] InnoDB: Failed to set O\_DIRECT on file](https://bugs.mysql.com/bug.php?id=70298%5B/url%5D)

Hope this helps.