Currently, the temp space manager is not tuned for small allocations properly. This is so because historically it has been developed primarily for sorting/merging purposes that are known to work with big memory blocks, but now it's also used for temporary blobs and record sets which can be quite small.
First of all, it allocates 1MB of memory even for a few bytes of the requested temporary space. In the concurrent environment, it could be a problem, forcing other processes to swap their temporary data to disk instead of using the available memory.
Second, the internal segment allocator uses the linear search which is sub-optimal when number of segments is large (many small blocks). This issue manifests itself while releasing thousands of tiny temporary blobs at the transaction completion.