libqueue is a C library that provides persistent, named data queues for programs. Arbitrarily many queues can co-exist, given they have distinct names.
Arbitrary binary data can be stored. Queues are manipulated using a stack-based API:
push()
-- append data to the queuepop()
-- pop data from the queuepeek()
-- retrieve data from a given stack index
The data itself is stored using leveldb in a compressed hash tree.
/*
Intransparent data type. Nothing to see here...
*/
struct Queue;
/*
Data structure used to store data in, and retrieve data from the
queue
*/
struct QueueData {
void *v;
size_t vlen;
};
/*
Functions return LIBQUEUE_SUCCESS on success, LIBQUEUE_FAILURE on
error or LIBQUEUE_MEM_ERROR as a special failure type for failed
memory allocations.
*/
/*
Open a named queue. Directory location is provided as string
<id>. Queues are persistent and data stored previously will still be
available through the name it was written to. Operations on a queue
must conclude with a call to queue_close().
This function ensures that ${path}/libqueue exist. If it doesn't
and cannot be created OR not written to, this function fails.
*/
struct Queue * queue_open(const char *path);
/*
Same as queue_open but allow one to provde options for various settings internal
to leveldb. The vararg must be terminated by a NULL to indicate end of options
options are to be provided as strings with some requiring args
available options:
"failIfMissing"
if the database doesn't already exist, the database will not be created and error
status is returned
"paranoidChecks"
various paranoid checks will be done with every operation
"writeBufferSize"
must be followed by size_t value. this is the write buffer size of leveldb
"blockSize"
must be followed by size_t value. this is the block size of leveldb
"blockRestartInterval"
must be followed by size_t value. this is the block restart interval of leveldb
"maxOpenFiles"
must be followed by int value. max open file handles used by leveldb
"noCompess"
disables compression
*/
struct Queue * queue_open_with_options(const char *path,..., NULL);
/*
Push a data object onto the queue. struct QueueData must be filled
appropriately: d->v is a void* pointer to the data, d->vlen holds the
data size.
*/
int queue_push(struct Queue *q, struct QueueData *d);
/*
Pop data from the queue. The library will store the data in the passed
QueueData struct. d->v must be freed by the user.
*/
int queue_pop(struct Queue *q, struct QueueData *d);
/*
returns approximate leveldb size if there is at least one element ready
for reading in queue, other wise returns 0
*/
int queue_len(struct Queue *q, int64_t *len);
/*
returns numbers of elements in the queue
*/
int queue_count(struct Queue *q, int64_t *countp);
/*
compacts the queue on file system to use take up less disk space if possible
*/
int queue_compact(struct Queue *q);
/*
Like queue_pop(), but doesn't remove the data object from the queue
and returns the data at queue position s. Indexing is not used
*/
int queue_peek(struct Queue *q, int64_t s, struct QueueData *d);
/*
Like queue_push(), but over-writes the data at queue position s. This
function cannot be used to append new data to the queue. Indexing
starts from zero. s must be positive.
*/
int queue_poke(struct Queue *q, int64_t s, struct QueueData *d);
/*
Closes the queue. Only AFTER this function has been called one can be
sure that the data has been properly stored. Undefined behaviour may
occur if the concluding call to this function is omitted.
*/
int queue_close(struct Queue *q);
struct Queue
is an opaque data type the fields of which
should not be accessed directly.
Dependencies:
- libleveldb (on ubuntu, apt get install libleveldb-dev)
libqueue uses the cmake for building:
cmake . make
It's very simple: Use
#include <queue.h>
in your C program and link it with
-lqueue -lleveldb
given that libqueue was installed into the default linker search path.
The library comes with a number of reference/test/practical utilities that all operate on the queue named 'queueutils-stack0' to be used on the command line:
Pushes all its arguments in onto the queue queueutils-stack0. Exits with an error code of 0 if the push was successful and 1 if it wasn't.
Pops the strings from the queue that were pushed using
qpush
, on element per call to qpush. If there is no element
to pop, exits with the status code 1. If the pop was successfu, it
prints the string to stdout and exists with the status code 0.
if there are no elements, then 0 is returned, otherwise approximate length from leveldb is provided
number of elments in the queue
compact the queue on the file system to use less diskspace
repair a broken leveldb
Peek at elements in the queueutils' queue.
qpeek <INDEX> [<index1>[..<indexN>]]</indexN>]]
Tries to peek at all indizes listed as arguments. Skips over invalid indizes but fails if the queue cannot be accessed properly.
Replace an existing in the queue with another one.
qpoke Fails if we cannot poke the element at the specified index, or if the index is out of bounds.
executes some basic tests
This program is licensed under the GNU Lesser General Public License v3 or later. Note that the LGPL is a set of additional terms that apply on top of the GPL. The text of the GPL3 is provided in LICENSE.GPL3, the text of the LGPL in LICENSE.LGPL3.