Are you using both InnoDB and MyISAM tables?
Because right now you are assigning:
key_buffer_size=375M (only used by MyISAM)
innodb_buffer_pool_size=339M (only used by InnoDB)
and if you only have 2GB in that server then the combination of these two on top of the other very high settings that you have can very well get the server to start swapping.
Your sort buffer settings are a bit strange:
First of all you are actually setting it twice in the ini file:
set-variable = sort_buffer=2M
But it is also very small.
This forces any query with a result set larger than 2MB and contains a ORDER BY or GROUP BY that can’t use an index to be written to a temporary table on disk.
My recommendation is that you should increase this value and reduce some of the extremely large values that you have in your setup.
You should also check the total amount of used memory on your server during these peek times. Because if the OS starts to swap you will get a drastic reduction of performance. It’s always better to keep a pretty big headroom up to max memory usage than to allow the server to take so much that the OS starts to swap. Because you loose so much more when the OS starts to swap than giving the last MB’s of RAM to MySQL can gain.