Database Memory keep on increase and decrease

Max 250 queries every 5 mins
Max 50 connections
Max 1 concurrent connection every 5 sec
Avg 1.5k Read/Fetch Rows
24GB allocated for Innodb Buffer Pool
32GB Server RAM

I attached my memory usage. There is something wrong with the diagram. It should not keep on increase and reach until 97% usage. I think, there is something not right with the configuration.

Screenshot from 2017-09-12 12-21-03.png