Mydumper and myloader

MySQL & MariaDB
1

Hello Team ,

I was using CentOS - 7 and i have installed mydumper - mydumper-0.9.5-2.el7.x86_64 in Linux box and was using mydumper by using below syntax.
single Schema = 250 GB Size

mydumper --host=10.xx.xx.xx --user=root --password=xxxxx --database=dbname --outputdir=/backups/bkp --no-views --compress --build-empty-files --threads=4 --compress-protocol --no-backup-locks --less-locking --statement-size=100000000 -G -E -R -k

It was working fine .But when i go /backups/bkp folder , i can’t see table DDL .gz files… i can see all files are schemaname.tablename.sql.gz files …and metadata.partial file.

after 2.5 hrs backup went done and i started restoring by using below command .

time myloader -h 10.xx.xx.xx -u root -ppassword --database=dbname -t 5 -d /backups/bkp

I got the below errors : -

  • (myloader:2170): CRITICAL *: 09:37:32.744: the specified directory is not a mydumper backup

and then

  • (myloader:2348): CRITICAL *: 10:03:29.470: Error restoring sscdb.id_table from file sscdb.id_table.sql.gz: Table ‘sscdb.id_table’ doesn’t exist

and then

  • (mydumper:2522): CRITICAL *: 10:20:20.786: There are queries in PROCESSLIST running longer than 60s, aborting dump,
    use --long-query-guard to change the guard value, kill queries (–kill-long-queries) or use
    different server for dump

And also i was using the below one too.
But no luck
mydumper --host=10.xx.xx.xx --user=root --password=password --database=dbname --threads=4 --compress --triggers --events --routines --outputdir=/backups/bkp
Then i got warning :-

  • (mydumper:2522): CRITICAL *: 10:xx.xx.xx: There are queries in PROCESSLIST running longer than 60s, aborting dump,
    use --long-query-guard to change the guard value, kill queries (–kill-long-queries) or use
    different server for dump

Can anyone help me on this.

2

Hi vasu1,

I’m not sure if the original backup finished successfully.

The file metadata.partial should have been automatically renamed to metadata by mydumper after completing the backup. So everything points to the backup process ending unexpectedly.

The rest of the errors are a consequence of the incomplete backup.

  • No metadata file in dump directory.
  • No schema files with table definitions, DDL.

Could you check the mydumper log? it is likely that there is an error that indicates that the process was killed in some way.

1 Like
3

I have initiated mydumper again … it went fine and able to see schema DDL for all Tables …
But again i seen the below error in the nohup file .

  • (mydumper:1866): CRITICAL *: Couldn’t write metadata file (13)

How to generate mydumper log …can you please share the command .

And i have restored started …

1 Like
4

Hello vasu1,

(mydumper: 1866): CRITICAL *: Couldn’t write metadata file (13)

This error indicates that you don’t have permission to edit the metadata/metadata.partial file.

As you can see in the code, it is one of the first steps of the start_dump function.

It could be that you do not have permissions on the directory where you are writing the dump, or that the metadata.partial file already exists and the user who is doing the dump cannot modify it.

In any case, you will need an updated metadata file to be considered a correct dump, which can be used with myloader.

How to generate mydumper log… can you please share the command.

❯ mydumper --help | grep -i log
   -L, --logfile Log file name to use, by default stdout is used

For example, in your line, you could also add -v 3 to increase the verbosity of the log if you want.

mydumper [...] -v 3 -L /backups/mydumper_dbname.log
1 Like