Not the answer you need?
Register and ask your own question!

pt-Archiver very slow!!

mandmmandm ContributorInactive User Role Beginner
I am trying to archive a 21 Million row table using pt-archiver on a 512MB Amazon EC2 instance
and it is running very slow
the table is actively written into by my applications users as we are archiving it, could this be slowing down the archiving?

here are my archiving options

pt-archiver -u $MyUSER -p$MyPASS --source h=$MyHOST,D=$DEST_DB,t=$DEST_TABLE --purge --primary-key-only --why-quit --progress 10000 --statistics --charset utf8 --retries 3 --commit-each --limit 3000 --where "REQUEST_TIME <= DATE_SUB(NOW(), INTERVAL 60 DAY)"


here is the log output when i kill the job after few hours

TIME ELAPSED COUNT
2014-04-29T15:46:07 0 0
2014-04-29T16:17:21 1874 10000
2014-04-29T16:36:18 3011 20000
2014-04-29T16:55:19 4152 30000
2014-04-29T17:14:17 5289 40000
2014-04-29T17:42:40 6992 50000
2014-04-29T18:11:04 8697 60000
2014-04-29T18:33:02 10015 70000
2014-04-29T18:52:03 11156 80000
2014-04-29T19:15:55 12588 90000
2014-04-29T19:33:00 13613 100000
2014-04-29T19:57:07 15059 110000
2014-04-29T20:25:27 16760 120000
2014-04-29T20:44:56 17929 130000
2014-04-29T21:06:48 19241 140000
2014-04-29T21:35:12 20945 150000
2014-04-29T21:52:58 22011 160000
2014-04-29T22:20:35 23667 170000
2014-04-29T22:39:26 24799 180000

Comments

  • mandmmandm Contributor Inactive User Role Beginner
    Another run and same scenario, it slows down after a particular point
    TIME ELAPSED COUNT
    2014-05-01T12:23:05 0 0
    2014-05-01T12:23:08 2 10000
    2014-05-01T12:23:10 4 20000
    2014-05-01T12:23:13 7 30000
    2014-05-01T12:23:15 9 40000
    2014-05-01T12:23:17 12 50000
    2014-05-01T12:23:20 14 60000
    2014-05-01T12:23:22 17 70000
    2014-05-01T12:23:25 19 80000
    2014-05-01T12:23:27 22 90000
    2014-05-01T12:23:30 24 100000
    2014-05-01T12:23:32 27 110000
    2014-05-01T12:23:35 29 120000
    2014-05-01T12:23:48 42 130000
    2014-05-01T12:24:12 66 140000
    2014-05-01T12:24:36 90 150000
    2014-05-01T12:25:01 116 160000
    2014-05-01T12:25:26 140 170000
    2014-05-01T12:25:49 164 180000
    2014-05-01T12:26:14 188 190000
    2014-05-01T12:26:38 212 200000
    2014-05-01T12:27:02 237 210000
    2014-05-01T12:27:27 261 220000
    2014-05-01T12:27:51 285 230000
    2014-05-01T12:28:15 309 240000
    2014-05-01T12:28:39 334 250000
    2014-05-01T12:29:04 359 260000
    2014-05-01T12:29:28 383 270000
    2014-05-01T12:29:53 407 280000
    2014-05-01T12:30:17 432 290000
    2014-05-01T12:30:42 456 300000
    2014-05-01T12:31:06 481 310000
    2014-05-01T12:31:31 505 320000
    2014-05-01T12:31:55 529 330000
    2014-05-01T13:12:23 2958 340000
    2014-05-01T13:54:45 5499 350000
    2014-05-01T14:32:16 7750 360000
    2014-05-01T15:09:48 10002 370000
    2014-05-01T15:40:48 11863 380000
    2014-05-01T16:18:20 14114 390000
  • mandmmandm Contributor Inactive User Role Beginner
    another output
Sign In or Register to comment.

MySQL, InnoDB, MariaDB and MongoDB are trademarks of their respective owners.
Copyright ©2005 - 2020 Percona LLC. All rights reserved.