Need help with go-mydumper?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

xelabs
259 Stars 75 Forks GNU General Public License v3.0 53 Commits 12 Opened issues

Description

A multi-threaded MySQL backup and restore tool, faster than mysqldump

Services available

!
?

Need anything else?

Contributors list

Github Actions Status Github Actions Status Github Actions Status Go Report Card codecov.io

go-mydumper

go-mydumper is a multi-threaded MySQL backup and restore tool, and it is compatible with maxbube/mydumper in the layout.

Build

$git clone https://github.com/xelabs/go-mydumper
$cd go-mydumper
$make build
$./bin/mydumper   -h
$./bin/myloader   -h

Test

$make test

Usage

mydumper

./bin/mydumper -h
Usage: ./bin/mydumper -c conf/mydumper.ini.sample
  -c string
        config file

Examples: $./bin/mydumper -c conf/mydumper.ini.sample 2017/10/25 13:12:52.933391 dumper.go:35: [INFO] dumping.database[sbtest].schema... 2017/10/25 13:12:52.937743 dumper.go:45: [INFO] dumping.table[sbtest.benchyou0].schema... 2017/10/25 13:12:52.937791 dumper.go:168: [INFO] dumping.table[sbtest.benchyou0].datas.thread[1]... 2017/10/25 13:12:52.939008 dumper.go:45: [INFO] dumping.table[sbtest.benchyou1].schema... 2017/10/25 13:12:52.939055 dumper.go:168: [INFO] dumping.table[sbtest.benchyou1].datas.thread[2]... 2017/10/25 13:12:55.611905 dumper.go:105: [INFO] dumping.table[sbtest.benchyou0].rows[633987].bytes[128MB].part[1].thread[1] 2017/10/25 13:12:55.765127 dumper.go:105: [INFO] dumping.table[sbtest.benchyou1].rows[633987].bytes[128MB].part[1].thread[2] 2017/10/25 13:12:58.146093 dumper.go:105: [INFO] dumping.table[sbtest.benchyou0].rows[1266050].bytes[256MB].part[2].thread[1] 2017/10/25 13:12:58.253219 dumper.go:105: [INFO] dumping.table[sbtest.benchyou1].rows[1266054].bytes[256MB].part[2].thread[2]

... [stripped] ...

2017/10/25 13:13:02.939278 dumper.go:182: [INFO] dumping.allbytes[1024MB].allrows[5054337].time[10.01sec].rates[102.34MB/sec]... 2017/10/25 13:13:35.496439 dumper.go:105: [INFO] dumping.table[sbtest.benchyou1].rows[11345659].bytes[2304MB].part[18].thread[2] 2017/10/25 13:13:37.627178 dumper.go:105: [INFO] dumping.table[sbtest.benchyou0].rows[11974624].bytes[2432MB].part[19].thread[1] 2017/10/25 13:13:37.753966 dumper.go:105: [INFO] dumping.table[sbtest.benchyou1].rows[11974630].bytes[2432MB].part[19].thread[2] 2017/10/25 13:13:39.453430 dumper.go:122: [INFO] dumping.table[sbtest.benchyou0].done.allrows[12486842].allbytes[2536MB].thread[1]... 2017/10/25 13:13:39.453462 dumper.go:170: [INFO] dumping.table[sbtest.benchyou0].datas.thread[1].done... 2017/10/25 13:13:39.622390 dumper.go:122: [INFO] dumping.table[sbtest.benchyou1].done.allrows[12484135].allbytes[2535MB].thread[2]... 2017/10/25 13:13:39.622423 dumper.go:170: [INFO] dumping.table[sbtest.benchyou1].datas.thread[2].done... 2017/10/25 13:13:39.622454 dumper.go:188: [INFO] dumping.all.done.cost[46.69sec].allrows[24970977].allbytes[5318557708].rate[108.63MB/s]

The dump files:

$ ls sbtest.sql/
metadata                    sbtest.benchyou0.00009.sql  sbtest.benchyou0.00018.sql   sbtest.benchyou1.00006.sql  sbtest.benchyou1.00015.sql
sbtest.benchyou0.00001.sql  sbtest.benchyou0.00010.sql  sbtest.benchyou0.00019.sql   sbtest.benchyou1.00007.sql  sbtest.benchyou1.00016.sql
sbtest.benchyou0.00002.sql  sbtest.benchyou0.00011.sql  sbtest.benchyou0.00020.sql   sbtest.benchyou1.00008.sql  sbtest.benchyou1.00017.sql
sbtest.benchyou0.00003.sql  sbtest.benchyou0.00012.sql  sbtest.benchyou0-schema.sql  sbtest.benchyou1.00009.sql  sbtest.benchyou1.00018.sql
sbtest.benchyou0.00004.sql  sbtest.benchyou0.00013.sql  sbtest.benchyou1.00001.sql   sbtest.benchyou1.00010.sql  sbtest.benchyou1.00019.sql
sbtest.benchyou0.00005.sql  sbtest.benchyou0.00014.sql  sbtest.benchyou1.00002.sql   sbtest.benchyou1.00011.sql  sbtest.benchyou1.00020.sql
sbtest.benchyou0.00006.sql  sbtest.benchyou0.00015.sql  sbtest.benchyou1.00003.sql   sbtest.benchyou1.00012.sql  sbtest.benchyou1-schema.sql
sbtest.benchyou0.00007.sql  sbtest.benchyou0.00016.sql  sbtest.benchyou1.00004.sql   sbtest.benchyou1.00013.sql  sbtest-schema-create.sql
sbtest.benchyou0.00008.sql  sbtest.benchyou0.00017.sql  sbtest.benchyou1.00005.sql   sbtest.benchyou1.00014.sql

myloader

$ ./bin/myloader --help
Usage: ./bin/myloader -h [HOST] -P [PORT] -u [USER] -p [PASSWORD] -d  [DIR]
  -P int
        TCP/IP port to connect to (default 3306)
  -d string
        Directory of the dump to import
  -h string
        The host to connect to
  -p string
        User password
  -t int
        Number of threads to use (default 16)
  -u string
        Username with privileges to run the loader

Examples: $./bin/myloader -h 192.168.0.2 -P 3306 -u mock -p mock -d sbtest.sql 2017/10/25 13:04:17.396002 loader.go:75: [INFO] restoring.database[sbtest] 2017/10/25 13:04:17.458076 loader.go:99: [INFO] restoring.schema[sbtest.benchyou0] 2017/10/25 13:04:17.516236 loader.go:99: [INFO] restoring.schema[sbtest.benchyou1] 2017/10/25 13:04:17.516389 loader.go:115: [INFO] restoring.tables[benchyou0].parts[00015].thread[1] 2017/10/25 13:04:17.516456 loader.go:115: [INFO] restoring.tables[benchyou0].parts[00005].thread[2]

... [stripped] ...

2017/10/25 13:05:27.783560 loader.go:131: [INFO] restoring.tables[benchyou1].parts[00005].thread[9].done... 2017/10/25 13:05:36.133758 loader.go:181: [INFO] restoring.allbytes[4087MB].time[78.62sec].rates[51.99MB/sec]... 2017/10/25 13:05:44.759183 loader.go:131: [INFO] restoring.tables[benchyou0].parts[00001].thread[3].done... 2017/10/25 13:05:46.133728 loader.go:181: [INFO] restoring.allbytes[4216MB].time[88.62sec].rates[47.58MB/sec]... 2017/10/25 13:05:46.567156 loader.go:131: [INFO] restoring.tables[benchyou1].parts[00016].thread[6].done... 2017/10/25 13:05:50.612200 loader.go:131: [INFO] restoring.tables[benchyou0].parts[00008].thread[10].done... 2017/10/25 13:05:51.131155 loader.go:131: [INFO] restoring.tables[benchyou0].parts[00014].thread[2].done... 2017/10/25 13:05:51.185629 loader.go:131: [INFO] restoring.tables[benchyou0].parts[00011].thread[1].done... 2017/10/25 13:05:51.836354 loader.go:131: [INFO] restoring.tables[benchyou1].parts[00004].thread[0].done... 2017/10/25 13:05:52.286931 loader.go:131: [INFO] restoring.tables[benchyou1].parts[00006].thread[11].done... 2017/10/25 13:05:52.602444 loader.go:131: [INFO] restoring.tables[benchyou0].parts[00019].thread[8].done... 2017/10/25 13:05:52.602573 loader.go:187: [INFO] restoring.all.done.cost[95.09sec].allbytes[5120.00MB].rate[53.85MB/s]

License

go-mydumper is released under the GPLv3. See LICENSE

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.