Pool based allocations - nginx

Nginx has one core module called "Pool based allocation".   This "pool based allocation" method is in addition to direct heap functions via wrappers such as ngx_alloc(), ngx_calloc and ngx_free. API functions defined by this module has "ngx_p" in the beginning of each function.

Pool based allocation routines are implemented in ngx_palloc.c and ngx_palloc.h.

It appears that these API functions are meant to reduce memory fragmentation that is otherwise a possibility with direct heap based allocations.  Any module or function requiring temporary buffers, then these  'pool based allocation' API functions can be used. Since there is no protection (mutex) primitives in the functions defined in ngx_palloc.c and there is no freeing of buffers until the pool is destroyed, I am guessing that these API functions are mainly would be used by modules for local allocations within one connection processing.  If there is any data required across connections which are to be handled by multiple threads, then these API functions are not suitable to allocate buffer to store that data.
Pool module also provides facility to register callback functions when the pool is being destroyed. Pool module calls the registered functions for application to do any cleanup.

Pool utility module avoids going to heap for every memory allocation required by application modules. It allocates bigger size memory chunks from heap and provides required smaller memory blocks from the big chunks to the application modules through its API. 

Usage is simple:

• Create pool using "ngx_pool_t *ngx_create_pool(size_t size, ngx_log_t *log)".   'size' parameter given to this API function  really a good hint for pool module to allocate memory at the pool creation time. Allocations from this pool using ngx_palloc() function is given from this memory.   As you can see from the code,  if the initially allocated memory chunk is not sufficient to satisfy the application allocations,  pool module has facility to allocate more memory chunks on demand basis.  Pool utility module restricts the amount of memory it can provide to the applications from the preallocated memory chunks. Typically, it is restricted to PAGE SIZE.    But it provided one escape route.  If  ngx_palloc is called with size more than the PAGE SIZE of processor architecture, then it bypasses the pool chunks and goes to the heap directly. These allocations are maintained in separate linked list within pool  in 'large' list.   When ngx_pfree is called, if the block is in the 'large' list it frees the memory to heap. If it does not belong to 'large' list, then it keeps quiet.  Since large list is maintained for each large block allocation and is used by ngx_pfree, I feel there could be impact on performance if there are more large block allocations by the application.  Application module better off calling heap wrapper functions directly.  
• This function also allocates a meta information of type ngx_pool_t and returns pointer to this as handle for other API functions.
  
• Pool can be destroyed using " void ngx_destroy_pool(ngx_pool_t *pool)".  It will free all memory chunks it allocated from heap and then frees the pool meta information block. This function, before destroying the buffers, it calls application registered cleanup functions.
  
• Pool can be reset to its initial state using "void ngx_reset_pool(ngx_pool_t *pool)".  If you want to reuse the pool, this function can be used.  It is similar to destroying the pool and creating a new pool. But using this function is faster than recreating the pool.  It preserves the memory chunks the pool module gets from the heap.  Note that  it frees the memory that is maintained in 'large' list.
• Memory from the pool can be allocated using "void *ngx_palloc(ngx_pool_t *pool, size_t size)" and "void * ngx_pnalloc(ngx_pool_t *pool, size_t size)" :   Function takes pool handle and 'size' of memory required.  These functions checks whether the 'size' requested is less than the PAGE SIZE. If so, it allocates memory from preallocated chunks. If not, it goes to heap and gets the memory and then puts it in 'large' list.  ngx_palloc function is same as ngx_pnalloc, except that the memory it gives is aligned to "NGX_ALIGNMENT" which is defined in nginx with value "sizeof(unsigned long)". Some internal details:  This API function if memory is not found in the preallocated chunks, then it allocates one more big chunk from the heap and provides requested memory from this big chunk.  These chunks are arranged in linked list fashion in the pool handle.  This linked list is used in "Destroy Pool" function to free these  chunks to the heap.
  
• Buffers allocated using this module can be freed using "ngx_int_t ngx_pfree(ngx_pool_t *pool, void *p)".   This function does not do any thing  if the buffer is allocated from the pool chunks. If memory is allocated from heap i.e if memory pointer is in 'large' list,  then it frees the memory to the heap.
• If buffer allocated needs to be initialized to zero, then "void *ngx_pcalloc(ngx_pool_t *pool, size_t size)" can be called. This function is same as ngx_palloc except that it does one additional operation of initializing the content to all zeros.
• Yet times application may require its functions to be called when pool is being destroyed for some buffer allocations.  Pool module provides this function " ngx_pool_cleanup_t * ngx_pool_cleanup_add(ngx_pool_t *p, size_t size)" for applications to get buffer and register cleanup functions for that buffer. This function internally calls ngx_palloc().   In addition, this function allocates some memory to store the cleanup handler and attaches the cleanup  handler to the POOL cleanup list.