Wednesday, December 30, 2009

nginx - Arrays

Ngnix web server provides 'array' utility.  This utility provides set of API functions to create array and push elements.  There is no API function to remove elements in the array. Since it uses pool functions internally to allocate memory, I am guessing that these elements will also be destroyed as part of the pool cleanup. 

Following structure represents the array:

struct ngx_array_s {
    void        *elts;
    ngx_uint_t   nelts;
    size_t       size;
    ngx_uint_t   nalloc;
    ngx_pool_t  *pool;
};

'elts':  Read this as 'elements'. It contains user data elements. Each  user data element size is stored in 'size'. It expects that all user data elements are of same size.
'netls':  Read this as 'number of elements'. It represents the valid number of user data elements pointed by 'elts'.
'size':   Size of user data element.
'nalloc':  Read this as 'number of allocated elements'.  Represents the number of elements that can be stored in 'elts' in total. 
'pool':  Pool that is being used to allocate both ngx_array_t as well as memory to store elements.

API functions :
  • ngx_array_t *ngx_array_create(ngx_pool_t *p, ngx_uint_t n, size_t size) :  This function is used to create the array and allocate memory required to store 'n' elements of size 'size' from the pool 'p'. It returns pointer to ngx_array_t.  User application is expected to store this pointer in its control structures and use this in further functions. 
  • ngx_int ngx_array_init(ngx_array_t *array, ngx_pool_t *pool, ngx_uint_t n, size_t size) : This function is used when application allocated ngx_array_t by itself (by declaring a member of this type in its control structures). This function does every thing as ngx_array_create other than the allocating ngx_array_t.  It returns NGX_OK if every thing is okay.
  • void *ngx_array_push(ngx_array_t *a):  This function gives the pointer to the free 'element'.  Application is expected to typecast its structure to the returned pointer and use it. If there is no space in the 'elts', then it allocates contiguous memory from the pool, copies old data and returns pointer to the free element.  Applications should not be storing the pointer to the 'elts' it its control structures as 'elts' can change with new elements. It is suggested to get 'elts' every time array elements are required for its processing.
  • void *ngx_array_push_n(ngx_array_t *a, ngx_uint_t n):  This function is same as ngx_array_push, except that it maks room for 'n' elements in contiguous memory. It returns pointer to the first free element.

3 comments:

Ravi Chunduru said...
This comment has been removed by the author.
Ravi Chunduru said...

Thanks for the nice article. When looking into the code I found something which looks like a bug.

When pushing new elements, if the array is full then nginx will create a new array and copy all the elements in one case. That looks fine.

But, as they do ngx_palloc() chances are that a new memory block would have been used. Then, pool pointer is no more having the array elements but the array elements are in the pool->current block. If the array->pool is not updated to pool->current , then all the processing on array like pushing beyond the size second time, destroying the array will be wrong from then on.

Ravi Chunduru said...

I described the problem in detail here
http://mailman.nginx.org/pipermail/nginx-devel/2014-January/004826.html