Libnifty Overview

This project contains pthread packages for use in multi-threaded C code. They use a low-overhead, reference-counting mechanism for accessing shared objects, which helps to protect against stale-pointer errors and memory leaks:

Package	Description
nft_list	Linked lists with thread-local free node cache.
nft_pool	Thread pool to execute tasks asynchronously.
nft_queue	Inter-thread event queue or message channel.
nft_rbtree	Balanced red-black btree for associative mapping.
nft_sack	Bulk memory allocator used by nft_list.
nft_task	Schedule tasks to execute at a specified time.
nft_vector	Set operations using sorted arrays for peformance.
nft_core	The base class for the other Nifty packages.
nft_win32	A limited pthread emulation layer for Windows.

Consult the header files in ./include/ for the API definitions. The ANSI C implementations in ./src/ also contain test programs (look for #ifdef MAIN) that illustrate how to use the package.

These packages generally use elastic data structures and optimal algorithms for best performance. All of these packages are thread-safe, and are cancellation-safe using the normal deferred-cancellation mode. All of the blocking calls are deferred cancellation points.

To build, tweak src/Makefile to your liking, then

$ make -C src

To build and run the unit test programs, do

$ make -C src test

This code should build on most modern Unix systems, but you may need to tweak the Makefile, and you may wish to adjust the implementation of the nft_gettime() call in nft_gettime.h to suit your platform.

This file gives a detailed discussion how you can create your own "classes" using libnifty, and why it may be advantageous to do so. See the section Object-oriented development based on nft_core, below.

Note also that there is a lockless implementation of the handle subsystem, which you can enable in the Makefile. This is not yet practical, because it also requires you to fix the size of the handle table. See src/nft_handle.c for more information.

The libnifty packages can be built on WIN32. For more information, refer to the section WIN32 Notes below.

If you have questions or contributions, send them to sean@xenadyne.com.

How to use the Nifty packages

Of the packages that we list above, only the thread pool, message queue, and task scheduler are directly useful. The message queue package is by far the most useful. In any multithreaded program, your threads are going to need to communicate, and message queues are a simple and convenient way to do this.

So, let's discuss the nft_queue package in detail, since it is useful, and it illustrates many common features of the libnifty packages. The source code is in src/nft_queue.c, and like all libnifty packages, there is a test program at the bottom of the .c file, enclosed by #ifdef MAIN ... #endif. This test program demonstrates how to use the nft_queue package.

Let's look at some of the code that you can see in the test program. Here, we create a queue:

    // Create an unlimited queue.
    nft_queue_h Q = nft_queue_new(0);

The parameter 0 means that there is no limit on the queue size, which comes into play when you try to add an item to the queue. More on this later. The _new() function returns a "handle" for the queue, and even though it is defined to be a pointer to a struct, it is really not a pointer value, it is just a number generated by incrementing a counter.

While you cannot dereference the handle returned from nft_queue_create(), you can use it to interact with the queue via the queue's APIs. For example, you can add an item to the end of the queue:

     int rc = nft_queue_add( Q, strdup("hello"));

Because you do not dereference the handle, these APIs help to improve encapsulation, and because the handle is not a pointer, it is very easy to prevent "stale" handles being used. You can convert the handle to a real pointer - you just need to be more careful when doing so. We discuss this in detail, a little further on.

The nft_queue_add() call returns an error code. Most libnifty APIs that return an error code, use zero to indicate success, and use POSIX error codes like EINVAL or ENOMEM to indicate failure. These error codes are defined in errno.h, but libnifty calls do not set the errno variable (though errno may be set by system libraries that libnifty calls). The error codes that a particular call can return, are documented in the package header file.

Most libnifty calls that can block have a timeout parameter, such as this call to pop an item from the head of the queue:

   void * item = nft_queue_pop_wait( Q, -1);

In the example above, if the queue is empty, the nft_queue_pop_wait() call may block indefinitely, if no other thread enqueues an item, because the timeout parameter is negative. If the timeout parameter is zero or positive, then the call would return NULL after that many seconds, if no item were queued in the meantime. So a zero timeout will return immediately, returning NULL if the queue was empty.

When we wish to do away with the queue, we call nft_queue_shutdown():

    rc = nft_queue_shutdown( Q, 10);

The _shutdown call also takes a timeout parameter. Shutdown waits for the queue to become empty, and so this operation may time out. Attempts to add items will fail once the shutdown operation begins, so the shutdown should complete eventually, as long as some thread is popping items from the queue. But it's best to set a timeout, just to be safe.

You may wonder, why we did not name this function nft_queue_destroy, since it appears to be the counterpart to nft_queue_new? In fact, there is a nft_queue_destroy() function, but it is very rarely called directly, because libnifty objects are reference-counted. Because of this, you cannot be certain that the queue has been destroyed, even after nft_queue_shutdown() returns successfully.

Every thread that is blocked within a nft_queue API call, will hold a reference to the underlying object for the duration of that call. Each thread will discard that reference as it returns from the API that it had called. The handle will be invalidated, and the queue destroyed, only when the last reference has been discarded.

This discussion illustrates why libnifty API calls use handles. Reference counting of some kind, is virtually unavoidable when sharing pointers to dynamically-allocated (malloc'ed) memory in a multithreaded program. However, reference counting does not solve every problem - while it can prevent the use of stale references, it is still possible to leak reference counts, and so leak memory.

The virtue of handles, is that all of the reference counting happens inside the API calls, where we can take pains to ensure correctness. Let's look at the implementation of the nft_queue_pop_wait() call, to demonstrate how this works:

void *
nft_queue_pop_wait(nft_queue_h handle, int timeout)
{
    nft_queue * q = nft_queue_lookup(handle);
    if (!q) return NULL;

    pthread_mutex_lock(&q->mutex);
    void * item = nft_queue_dequeue(q, timeout);
    pthread_mutex_unlock(&q->mutex);

    nft_queue_discard(q);
    return item;
}

First, we call nft_queue_lookup() to convert the caller's queue handle to a pointer. If the queue had been destroyed, the lookup operation would return NULL, so the _pop_wait call will reject stale handles. The nft_queue_lookup() operation also will not return an object pointer, unless that object is really a nft_queue, or a nft_queue subclass. So, handles that were incorrectly type-cast are also rejected.

The queue's reference count is incremented atomically by the lookup operation, so the pointer that is returned is guaranteed to be protected by the reference count. Therefore, it is important that we call nft_queue_discard(), to decrement the reference count, before returning. If we wish to be cancellation-safe, we must further ensure that the reference will be discarded by a cancellation cleanup handler, in the event that the thread is cancelled.

In fact, the nft_queue_dequeue() function, which has a timeout and thus can can block, is a deferred cancellation point. It pushes a cancellation cleanup handler that will discard the reference, and also release the mutex, if the thread is cancelled while waiting. So, by using handles, it becomes much easier to create APIs that provide strong guarantees to callers.

The concepts and conventions that we just discussed, are common to the other libnifty packages. I hope that you are persuaded that using the Nifty library will make your pthread programs simpler, and more stable.

Now, if you review the header file nft_queue.h, you may notice that calls such as nft_queue_dequeue(), and nft_queue_destroy(), and indeed, the struct declaration of the nft_queue object itself, are exposed. You might have assumed, after all this talk of encapsulation, that these details would be hidden, and the APIs declared static, within nft_queue.c. The reason that these APIs are published, is to enable subclasses to be built from nft_queue. In fact, the nft_pool package is itself a subclass of nft_queue. In the section that follows, we discuss how this works in detail.

"Object-oriented" development based on nft_core

Many of the Nifty packages are derived in "object-oriented" style from nft_core. This section discusses how this derivation works. Our approach is as simple as possible. The goals are to:

Support only simple single-inheritance,
Produce APIs with strong static type-checking,
Provide effective runtime type-checking,
Use reference-counts to manage shared pointers,
Classes publish handles, keeping pointers private.

The idea is that every object in the framework will inherit from this core class:

    typedef struct nft_core {
        const char  * class;
        unsigned long handle;
        void        (*destroy)(ntf_core *);
    } nft_core;

Inheritance is as simple as it can be - the parent class must be declared as the very first element of the subclass. This is single-inheritance, and in consequence, a pointer to any subclass is also a pointer to its parent class, and grandparent class, etc. To demonstrate, here is an example subclass, consisting of a simple shared (reference-counted) string:

    typedef struct nft_string {
        nft_core core;
        char   * string;
    } nft_string;

The most important feature of the libnifty framework, is that objects derived from nft_core have a "handle", which is an integer that uniquely identifies one object instance. We have already discussed how client code refer to objects via handles, when calling libnifty APIs. Within the API call, we use nft_core_lookup() to obtain a pointer to the object instance:

    nft_core * object_reference = nft_core_lookup( void * handle );

The object's reference count is incremented whenever nft_core_lookup is applied to a valid handle. This ensures that the pointer you receive from the lookup cannot be freed until you explicitly release your reference, via nft_core_discard:

    nft_core_discard( object_reference );

This also means that you must balance every _lookup call with a call to _discard, or the object will never by freed. It is particularly important that cancellation cleanup handlers discard references to nft_core objects where necessary.

Since nft_core_lookup returns a nft_core *, you will need to type-cast the nft_core * to the object's actual type. But, how can you be certain that this pointer really points to an instance of a given class?

That is the purpose of the nft_core.class member. When a subclass is based on nftcore, we arrange that the nft_core.class will be the string "nft_core:subclass". Similarly, if we derive a sub-subclass from subclass, its class string should be "nft_core:subclass:subsubclass". In this way, we are able to define a safe typecast function for subclass, by comparing the object's class name to the desired class. For our shared string class:

    nft_string *
    nft_string_cast(nft_core * object) {
        if (!strncmp(object->class, "nft_core:nft_string", strlen("nft_core:nft_string")))
             return (nft_string *) object;
        else
             return NULL;
    }

The trick, then, is to arrange for the nft_core.class name to be constructed properly when we instantiate subclasses. With these considerations in mind, it should be clear why the nft_core constructor (shown below) takes both class and size parameters - these indicate the subclass that is being created, and its size:

    nft_core *
    nft_core_create(const char * class, size_t size) {
        nft_core * object = malloc(size);
        nft_handle handle = nft_handle_alloc(object);
        *object = { class, handle, nft_destroy };
        return self;
    }

Note that the constructor shown above simplifies the details of handle allocation, but it is enough to illustrate the idea. The core destructor is very simple - it merely frees the memory that was occupied by the object, but we do pass the pointer thru nft_core_cast, to ensure that we are freeing a nft_core object:

    void
    nft_core_destroy(nft_core * pointer) {
        nft_core * object = nft_core_cast(pointer, nft_core_class);
        if (object)
            free(object);
    }

Below, we show the constructor for our nft_string example. Note that it accepts the same size and class parameters that the core constructor does - this enables us to derive subclasses from this subclass. Note that it calls the parent-class constructor as the very first step, passing these same size and class parameters:

    nft_string *
    nft_string_create(const char * class, size_t size, const char * string)
    {
        nft_string  * object = nft_string_cast(nft_core_create(class, size));
        object->core.destroy = nft_string_destroy;
        object->string = strdup(string);
        return object;
    }

You may also have noted that the subclass constructor overrides the destructor. This is how virtual methods work in libnifty - it is up to you to manage them. The subclass destructor must call its parent destructor as the very last step:

    void
    nft_string_destroy(nft_core * pointer)
    {
        nft_string * object = nft_string_cast(pointer);
        if (object)
            free(object->string);
        nft_core_destroy(pointer);
    }

So, to create an instance of nft_string, we simply invoke the constructor, passing the full class name, object size, and the string text:

    nft_string * this = nft_string_create("nft_core:nft_string", sizeof(nft_string), "apple");

That is all there is to it. This is the entire inheritance system in the libnifty framework. We have not yet used a single macro, but of course that will not last, and besides, we can obtain real benefits from a few simple macros. First, we define the nft_core class name using this macro:

    #define nft_core_class    "nft_core"

This macro makes it easy to construct the class name of a subclass, taking advantage of the fact that C will implicitly concatenate two string literals that occur together. For example:

    #define nft_string_class nft_core_class ":nft_string"

This may seem trivial, but it pays off when we derive a subclass from nft_string:

    #define nft_substring_class nft_string_class ":nft_substring"

The benefit should now be apparent: if nft_string_class changes its place in the class hierarchy, these changes will propagate automatically to the value of nft_substring_class.

Given the nft_string_class macro, the nft_string_cast() function that we showed earlier can now be created via macros. The NFT_DECLARE_CAST macro creates the function prototype in a header file, and the NFT_DEFINE_CAST macro creates the corresponding function definition:

    #define NFT_DECLARE_CAST(subclass) \
    subclass * subclass##_cast(nft_core * p);

    #define NFT_DEFINE_CAST(subclass) \
    subclass * \
    subclass##_cast(nft_core * p) { \
        return (subclass *) nft_core_cast(p, subclass##_class); \
    }

We have already discussed, that every object derived from nft_core has a handle that can be used to look up the original object, and we have shown how this makes it possible to keep all reference-counting logic inside the public API. Now we'll talk about how to manipulate handles within these public APIs. Below, we create an instance of nft_string, and save its handle in the variable handle:

    nft_string * object = nft_string_create(nft_string_class, sizeof(nft_string), "my substring");
    nft_handle   handle = object->core.handle;

The handle can be used to obtain a copy of the original object pointer, via nft_core_lookup(). Because nft_core_lookup creates an new reference to the object, the object's reference count is incremented by nft_core_lookup, and so we must discard the reference when we are done with it. You should never assume that lookup will succeed, because your API can easily be given a handle for an object that has been destroyed. so your code should always look like this:

    nft_core * pointer = nft_core_lookup(handle);
    if (pointer != NULL) {
        ...
        nft_core_discard(pointer);
    }

To be type safe when working with subclasses of nft_core, we must use type-safe casting. Using our nft_string example, we cast the nft_core * to a nft_string * before using nft_string calls:

    nft_core * core = nft_core_lookup(handle);
    if (core) {
        nft_string * string = nft_string_cast(core);
        if (string) {
           nft_string_print(string);
        }
        nft_core_discard(core);
    }

We can use macros to create type-safe wrappers that clean up the code above. First, let's give handles of nft_string objects their own type, nft_string_h, so that we can have strong static type-checking on handles:

    typedef struct nft_string_h * nft_string_h;

Now, define a simple wrapper function to cast a nft_string object's handle to type nft_string_h:

    nft_string_h nft_string_handle(const nft_string * string)
    {
        return (nft_string_h) string->core.handle;
    }

Next, a wrapper for nft_core_lookup() in the same style:

    nft_string * nft_string_lookup(nft_string_h handle)
    {
        return nft_string_cast(nft_core_lookup(handle));
    }

And, a wrapper for nft_core_discard():

    void nft_string_discard(nft_string * string)
    {
        nft_core_discard(&string->core);
    }

These wrapper functions provide clean, strongly-typed APIs to manipulate nft_string handles and object references. Using these wrapper functions, our example fragment can be coded much more cleanly:

    nft_string * object = nft_string_lookup(handle);
    if (object != NULL) {
        ...
        nft_string_discard(object);
    }

These wrapper functions can easily be defined via macros. For example, here are two macros, that declare and define the typesafe _cast() function wrapper for a given subclass:

#define NFT_DECLARE_CAST(subclass) \
subclass * subclass##_cast(void *);

#define NFT_DEFINE_CAST(subclass) \
subclass * subclass##_cast(void * vp) { return nft_core_cast(vp, subclass##_class); }

Finally, we gather all these macros into two convenience macros, which provide type-safe wrappers for the four basic operations that all subclasses derived from nft_core support:

#define NFT_DECLARE_WRAPPERS(subclass) \
NFT_TYPEDEF_HANDLE(subclass) \
NFT_DECLARE_CAST(subclass)   \
NFT_DECLARE_HANDLE(subclass) \
NFT_DECLARE_LOOKUP(subclass) \
NFT_DECLARE_DISCARD(subclass)

#define NFT_DEFINE_WRAPPERS(subclass) \
NFT_DEFINE_CAST(subclass)    \
NFT_DEFINE_HANDLE(subclass)  \
NFT_DEFINE_LOOKUP(subclass)  \
NFT_DEFINE_DISCARD(subclass)

Note the first group creates function prototypes, while the second group creates the implementations. These macros are all defined in nft_core.h. Using them, the entire implementation of our nft_string class consists of the following:

typedef struct nft_string
{
    nft_core core;
    char   * string;
} nft_string;

#define nft_string_class nft_core_class ":nft_string"

NFT_DECLARE_WRAPPERS(nft_string)
NFT_DEFINE_WRAPPERS(nft_string))

void
nft_string_destroy(nft_core * core)
{
    nft_string * object = nft_string_cast(core);
    if (object) free(object->string);
    nft_core_destroy(core);
}

nft_string *
nft_string_create(const char * class, size_t size, const char * string)
{
    nft_string  * object = nft_string_cast(nft_core_create(class, size));
    object->core.destroy = nft_string_destroy;
    object->string = strdup(string);
    return object;
}

We demonstrate how to implement and use this example string subclass in src/nft_string.c. Examples using the example class are shown in the unit test that begins with #ifdef MAIN. To build and run the demo, simply do:

cd src ; make nft_string ; ./nft_string

To build and run the full test suite, do:

cd src ; make test

WIN32 Notes

libnifty can be built for WIN32 using MinGW. Earlier versions of libnifty were buildable using Visual Studio 6, and as far as I know, this is still possible, but I have not tried to build this release with Visual Studio.

I recommend the pthread_win32 library to provide the POSIX pthread API. MinGW provides a pthread_win32 package, or you can obtain sources from the project site http://sourceware.org/pthreads-win32/. Alternatively, libnifty provides an optional pthread emulation layer for use on WIN32, which we call nft_win32.

In comparison to pthread_win32, nft_win32 has many limitations, so you should use pthread_win32 if that is at all possible. We have coded the Nifty packages to work within the limitations of nft_win32, and if you wish to use it, you should study the warnings in nft_win32.h. Nifty builds on MinGW using nft_win32 by default, so you will need to make adjustments in order to use pthread_win32.

haifenghuang / Nifty

Libnifty Overview

How to use the Nifty packages

"Object-oriented" development based on nft_core

WIN32 Notes

About

Languages