NaokiHori / SimpleNpyIO

Simple, C99-compatible, well-tested, well-documented, library-independent, and single-source NPY file reader / writer

SimpleNpyIO

_ _ _

_ _

Overview

A C99-compatible, single-header (snpyio.h) single-source (snpyio.c) library for file reading and writing in the NPY format.

The NPY file format is a binary format utilised for storing multi-dimensional arrays. It comprises a binary dataset preceded by a header that contains the essential metadata to identify the dataset: shape, datatype, and memory order (either row-major or column-major).

Performing I/O operations on NPY files is extremely straightforward: load or dump the metadata, and then read or write the dataset. This library handles the initial step. Due to the simple format, parallel I/O, supported by MPI, can be easily managed.

Dependency

• C compiler

Quick start

1. Prepare workspace:

   mkdir -p /path/to/your/working/directory
   cd /path/to/your/working/directory

2. Checkout this repository, e.g.:

   git clone https://github.com/NaokiHori/SimpleNpyIO
   cd SimpleNpyIO

3. Build and run:

   make
   ./a.out

This gives you:

data (dumped)
  0   1   2   3   4
  5   6   7   8   9
 10  11  12  13  14
header is successfully dumped (size: 64)
header is successfully loaded (size: 64)
data (loaded)
  0   1   2   3   4
  5   6   7   8   9
 10  11  12  13  14

Very simple I/O operations were performed here as a showcase: a two-dimensional data (shown above) is written to a NPY file example.npy, which is reloaded to show the second part. The resulting file example.npy is easily inspected by using Python:

import numpy as np

data = np.load("example.npy")

print(data.shape) # (3, 5)

Please check src/main.c for details.

Practical Usage

Recommended: copy

Copy src/snpyio.c and include/snpyio.h into your project:

include
└── snpyio.h

src
├── main.c
└── snpyio.c

Here, main.c is your code. Now all functions this library offers are ready.

Again, please check src/main.c for APIs.

Advanced: Git submodule

A submodule branch is available, which is specialised to be imported into an existing library, e.g.:

git submodule add --branch submodule https://github.com/NaokiHori/SimpleNpyIO

Make sure to include SimpleNpyIO/include/snpyio.h and compile SimpleNpyIO/src/snpyio.c with your sources.

You can find an example here, where this library is imported and used (check .gitmodules).

Others

For debugging purpose, please compile the source with SNPYIO_ENABLE_LOGGING flag. Please refer to the documentation.