MySQL memory handling for tmp_table_size

I’m trying to determine if it is safe to use a large tmp_table_size when I know that such tables are only occurring sparsely but it may help improve performance. My concern is that when using a connection pool, if the memory allocated for an in-memory table is not released, then there could be a chance of running out of memory. Does anyone know if the memory associated with the temporary table gets released when the table is released at the end of the query?