joar / rust-csv-py

Python CSV reader using the csv crate and PyO3

rust-csv_ + PyO3_ = Not much slower than csv_ 🎉

_

Contents

BIG DISCLAIMER

• This is not a production-ready library.
• I'm not a production-ready Rust programmer.
• Either Python 3's csv_ stdlib module is pretty %#!& fast or my Rust code is %#!& slow.

Installation

pip install rustcsv

Examples

CSVReader instance from path

examples/reader_from_path.py:

import tempfile
from rustcsv import CSVReader

# Create a temporary file to put our CSV content in,
# automatically delete it once we're done.
with tempfile.NamedTemporaryFile(mode="w") as writable_fd:
    writable_fd.write(
        """\
spam1,spam2,spam3
spam4,spam5,spam6
"""
    )
    writable_fd.flush()

    for row_number, row in enumerate(CSVReader(writable_fd.name), start=1):
        print(
            "row #{row_number}: {row}".format(row_number=row_number, row=row)
        )

# Prints:
# row #1: ("spam1", "spam2", "spam3")
# row #2: ("spam4", "spam5", "spam6")

CSVReader instance from a binary file object

examples/reader_from_file_object.py:

import tempfile
from rustcsv import CSVReader

# Create a temporary file to put our CSV content in,
# automatically delete it once we're done.
with tempfile.NamedTemporaryFile(mode="w") as writable_fd:
    writable_fd.write(
        """\
spam1,spam2,spam3
spam4,spam5,spam6
"""
    )
    writable_fd.flush()

    readable_fd = open(writable_fd.name, "rb")

    for row_number, row in enumerate(CSVReader(readable_fd), start=1):
        print(
            "row #{row_number}: {row}".format(row_number=row_number, row=row)
        )

# Prints:
# row #1: ("spam1", "spam2", "spam3")
# row #2: ("spam4", "spam5", "spam6")

Development

Development Installation

Install and build the extension locally from e.g. a git checkout.

Requirements

• Pipenv.
• Python 3.6.
• Rust and Cargo nightly (1.30 as of now) - https://rustup.rs/.

Install Python dependencies

pipenv install --dev

Build the rustcsv._rustcsv extension

Either

1. Using the "debug" cargo profile, or

   make develop-debug

2. Using the "release" cargo profile

   make develop-release

Run tests

make test

Run benchmarks

make benchmark

Note: make benchmark will always build the extension using the "release" cargo profile.

Benchmarks

Benchmarks are executed as the last step in the Travis CI project.

You can also run it yourself, see Development and Run benchmarks.

References

• https://github.com/python/cpython/blob/master/Modules/_csv.c
• https://pyo3.rs/v0.4.1/class.html