# Xtrabackup cannot keep up with binlogs **URL:** \ **Category:** Percona XtraBackup\ **Created:** [August 8, 2013, 3:27pm UTC](https://forums.percona.com/t/xtrabackup-cannot-keep-up-with-binlogs/2867 "2013-08-08T15:27:14Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![sysmonkey](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sysmonkey](https://forums.percona.com/u/sysmonkey)\ **Post date:** [August 8, 2013, 3:27pm UTC](https://forums.percona.com/t/xtrabackup-cannot-keep-up-with-binlogs/2867/1 "2013-08-08T15:27:14Z")
Hello, We’re having a problem with backups failing on one of our slaves. The backups fail soon after beginning with the following error message about not being able to find a binlog file. Our guess was that we’re replicating so much data (over a terabyte, 6000+ qps) that percona can’t keep up with it and the binlog file rotates before it can copy it. Our work around has been to stop the slave for about 10 minutes and then restart after the log file shows files are being copied. Does anyone have suggestions about tuning or some feature I’m not implementing? xtrabackup\_51 version 2.0.6 for MySQL server 5.1.59 unknown-linux-gnu (x86\_64) (revision id: undefined) xtrabackup: uses posix\_fadvise(). xtrabackup: cd to /var/lib/mysql xtrabackup: Target instance is assumed as followings. xtrabackup: innodb\_data\_home\_dir = ./ xtrabackup: innodb\_data\_file\_path = ibdata1:10M:autoextend xtrabackup: innodb\_log\_group\_home\_dir = ./ xtrabackup: innodb\_log\_files\_in\_group = 2 xtrabackup: innodb\_log\_file\_size = 268435456 xtrabackup: using O\_DIRECT 130808 16:09:24 InnoDB: Operating system error number 2 in a file operation. InnoDB: The error means the system cannot find the path specified. InnoDB: File name ./mysqld-relay-bin.119788 InnoDB: File operation call: ‘stat’. InnoDB: Error: os\_file\_readdir\_next\_file() returned -1 in InnoDB: directory . InnoDB: Crash recovery may have failed for some .ibd files! 130808 16:09:41 InnoDB: Operating system error number 2 in a file operation. InnoDB: The error means the system cannot find the path specified. InnoDB: File name ./mysqld-relay-bin.119785 InnoDB: File operation call: ‘stat’. InnoDB: Error: os\_file\_readdir\_next\_file() returned -1 in InnoDB: directory . ---
**Author:** ![dim0](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/dim0/32/950_2.png) [@dim0](https://forums.percona.com/u/dim0)\ **Post date:** [August 13, 2013, 2:24am UTC](https://forums.percona.com/t/xtrabackup-cannot-keep-up-with-binlogs/2867/2 "2013-08-13T02:24:30Z")
Hi sysmonkey, Could you provide us with the command you are running and the my.cnf active for your environment? It sounds quite weird that xtrabackup is currently failing to stat those files even with that amount of transactions coming through. Thanks!!! ---
**Author:** ![dim0](https://sea1.discourse-cdn.com/flex019/user_avatar/forums.percona.com/dim0/32/950_2.png) [@dim0](https://forums.percona.com/u/dim0)\ **Post date:** [August 13, 2013, 2:48am UTC](https://forums.percona.com/t/xtrabackup-cannot-keep-up-with-binlogs/2867/3 "2013-08-13T02:48:17Z")
Could you place the relay-logs outside the datadirectory as a workaround. It seems xtrabackup exits during the relay-log rotation in the datadirectory. xtrabackup keeps an eye on the files which are in the datadir and it currently doesn’t like the relay-logs being removed during a backup. Anyway, I have created a bug for you on [url][https://bugs.launchpad.net/percona-xtrabackup/+bug/1211669[/url]](https://bugs.launchpad.net/percona-xtrabackup/+bug/1211669%5B/url%5D) , so we can have a look on how to change this type of behaviour. ---
**Author:** ![sysmonkey](https://avatars.discourse-cdn.com/v4/letter/s/5e9695/32.png) [@sysmonkey](https://forums.percona.com/u/sysmonkey)\ **Post date:** [August 15, 2013, 8:55am UTC](https://forums.percona.com/t/xtrabackup-cannot-keep-up-with-binlogs/2867/4 "2013-08-15T08:55:24Z")
Hello, Thank you for the workaround, it worked. Here is the start command /usr/bin/ionice -c2 -n4 /usr/bin/innobackupex-1.5.1 --slave-info --password=xxxxx Here is our my.cnf # # The MySQL database server configuration file. # # You can copy this to one of: # - “/etc/mysql/my.cnf” to set global options, # - “~/.my.cnf” to set user-specific options. # # One can use all long options that the program supports. # Run program with --help to get a list of available options and with # –print-defaults to see which it would actually understand and use. # # For explanations see # [http://dev.mysql.com/doc/mysql/en/server-system-variables.html](http://dev.mysql.com/doc/mysql/en/server-system-variables.html) # This will be passed to all mysql clients # It has been reported that passwords should be enclosed with ticks/quotes # escpecially if they contain “#” chars… # Remember to edit /etc/mysql/debian.cnf when changing the socket location. [client] port = 3306 socket = /var/run/mysqld/mysqld.sock # Here is entries for some specific programs # The following values assume you have at least 32M ram # This was formally known as [safe\_mysqld]. Both versions are currently parsed. [mysqld\_safe] socket = /var/run/mysqld/mysqld.sock nice = 0 [mysqld] # # \* Basic Settings # user = mysql pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock port = 3306 basedir = /usr datadir = /var/lib/mysql tmpdir = /data/tmp default-storage-engine = innodb skip-external-locking # # Instead of skip-networking the default is now to listen only on # localhost which is more compatible and is not less secure. #bind-address = 0.0.0.0 bind-address = [xxx.xxx.xxx.xxx] skip-name-resolve # # \* Fine Tuning # key\_buffer = 16M max\_allowed\_packet = 64M thread\_stack = 192K thread\_cache\_size = 8 # This replaces the startup script and checks MyISAM tables if needed # the first time they are touched myisam-recover = BACKUP max\_connections = 2048 table\_open\_cache = 4000 table\_definition\_cache = 4000 open-files-limit = 12000 #thread\_concurrency = 10 # # \* Query Cache Configuration # query\_cache\_limit = 1M query\_cache\_size = 16M max\_heap\_table\_size = 100M tmp\_table\_size = 100M # # \* Logging and Replication # # Both location gets rotated by the cronjob. # Be aware that this log type is a performance killer. # As of 5.1 you can enable the log at runtime! #general\_log\_file = /var/log/mysql/mysql.log #general\_log = 1 # # Error logging goes to syslog due to /etc/mysql/conf.d/mysqld\_safe\_syslog.cnf. # # Here you can see queries with especially long duration slow\_query\_log = 1 slow\_query\_log\_file = /var/log/mysql/mysql-slow.log long\_query\_time = 2 #log-queries-not-using-indexes log-error = /var/log/mysql/mysql-error.log # # The following can be used as easy to replay backup logs or for replication. # note: if you are setting up a replication slave, see README.Debian about # other settings you may need to change. # Replications settings #skip-slave-start server-id = 60000313 master-host = [xxx.xxx.xxx.xxx] master-user = username master-password = password relay-log = /var/log/mysql/relay/mysqld-relay-bin log\_bin = /var/log/mysql/mysql-bin.log log-slave-updates binlog\_format = row expire\_logs\_days = 3 max\_binlog\_size = 100M sync\_binlog = 0 # Logs are getting deleted before backup can get past initial stages. # Set value to double the size of max\_binlog\_size. # See: [http://dev.mysql.com/doc/refman/5.1/en/slave-logs-relaylog.html](http://dev.mysql.com/doc/refman/5.1/en/slave-logs-relaylog.html) # \* InnoDB # # InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/. # Read the manual for more InnoDB related options. There are many! ignore-builtin-innodb plugin-load=innodb=ha\_innodb\_plugin.so innodb\_file\_per\_table = 1 innodb\_file\_format = barracuda innodb\_buffer\_pool\_size=32000M innodb\_log\_file\_size=256M innodb\_log\_buffer\_size=4M innodb\_flush\_method=O\_DIRECT innodb\_flush\_log\_at\_trx\_commit=2 # # \* Security Features # # Read the manual, too, if you want chroot! # chroot = /var/lib/mysql/ # # For generating SSL certificates I recommend the OpenSSL GUI “tinyca”. # # ssl-ca=/etc/mysql/cacert.pem # ssl-cert=/etc/mysql/server-cert.pem # ssl-key=/etc/mysql/server-key.pem [mysqldump] quick quote-names max\_allowed\_packet = 64M [mysql] #no-auto-rehash # faster start of mysql but no tab completition [isamchk] key\_buffer = 16M # # \* IMPORTANT: Additional settings that can override those from this file! # The files must end with ‘.cnf’, otherwise they’ll be ignored. # !includedir /etc/mysql/conf.d/