pushshift / ndjson_processor

High Speed multiprocessing ndjson processor

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

High speed ndjson processor

This is an example of a Python script that reads chunks of data from sys.stdin efficiently and sends them to multiprocessor worker threads for processing.

About

High Speed multiprocessing ndjson processor

License:MIT License


Languages

Language:Python 100.0%