Node parallel benchmarks
Playground for benchmarking Node.js APIs for parallel processing.
Note: This collection of benchmarks is scoped to comparing the speed of parallel processing a large number of text files. Example applications are linting, static analysis, code transformation, etc.
Usage
git clone https://github.com/aaronccasanova/node-parallel-benchmarks.gitnpm run workers-vs-forks
How to adjust the benchmark parameters
Open the config.mjs file and update the following parameters:
| Parameter | Description |
|---|---|
workerPoolSize |
The number of workers to use |
benchmarkIterations |
The number of times to run the benchmark for a given Node API |
processIterations |
The number of times processor.mjs iterates over data.tsx. Use this parameter to adjust how long a single file takes to process |
filesCount |
The number of files to process |
filesChunkSize |
The number of files a worker.mjs instance will process at a time |
How it works
- Each
src/run-<script>.mjsperforms the following operations:- Creates a pool of workers
- Passes a chunk of file paths to each worker
- Each worker performs the following operations:
- Promise.all over each file path:
- Reads the file content
- Passes the file content to the
processor.mjsscript (representative of a generic function to perform any of the example applications listed above) - Each processor script performs the following operations:
- Iterates over each character in the file content
- Computes the frequency of each character
- Returns the results back to the worker
- Returns the results back to the main thread
- Receives another chunk of file paths from the main thread and repeats the above steps
- Promise.all over each file path:
- Each worker performs the following operations:
- Terminates each worker when there are no more file paths to process