pt-query-advisor file size limits

I was unfortunate enough to have run pt-query-advisor on a general query log of 75GB:

pt-query-advisor --type genlog general.log

This (probably expectingly) ate up all the memory on the server and required a hard reboot. Luckily this was a dev server!

Anyway, the server is a 12GB machine running 64-bit RHEL 5.6.

I was wondering if there is a file size limit on this tool and others (pt-index-usage and pt-query-digest comes to mind, if they can handle general query logs).

Or, barring that, is there a tool that will break a general log into smaller manageable files?

That really should NOT have used that much memory. This is a bug for sure. Please file a bug on launchpad.

The workaround I can suggest is to pt-query-digest it first, with --print --no-report and --samples, to capture N distinct samples of each type of query, and then run pt-query-advisor on the result.

Baron, thanks for the response. Your comment made me go back and try it again. It’s running ‘fine’, so it must have been a combination of that and some other backup stuff going on at the time.

Sure does take a while to parse though!

Thanks for the response.