benhoyt / py-1brc

Optimising the One Billion Row Challenge (1BRC) in Python

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

py-1brc

This repo contains the source code for a talk I gave on Python performance at the Christchurch Python User Group. My talk looked at increasingly faster solutions to the One Billion Row Challenge, showing how to profile and optimise Python as we went.

Some of the solutions import line_profiler, so to run them, first set up a virtual environment and install that package:

$ python3 -m venv venv             # create the virtual environment
$ source venv/bin/activate         # active it
$ pip install -r requirements.txt  # install modules (line_profiler)

Then run a solution as follows (r1 through r7):

$ time python3 r7.py measurements.txt >results.txt

About

Optimising the One Billion Row Challenge (1BRC) in Python

License:MIT License


Languages

Language:Python 100.0%