worker-nodes
A node.js library to run cpu-intensive tasks in a separate processes and to not block the event loop.
Installation
$ npm install worker-nodesNode.js greater than 6.6.0 highly recommended.
API Reference
WorkerNodes
Kind: global class
- WorkerNodes
- new WorkerNodes(path, [options])
- .call :
Proxy - .ready() ⇒
Promise - .terminate() ⇒
Promise
new WorkerNodes(path, [options])
| Param | Type | Description |
|---|---|---|
| path | String |
An absolute path to the module that will be run in the workers. |
| [options] | Object |
See WorkerNodesOptions for a detailed description. |
workerNodes.call : Proxy
This exposes the api of a module that the worker nodes are working on. If the module is a function, you can call this directly. If the module exports multiple functions, you can call them as they were properties of this proxy.
Kind: instance property of WorkerNodes
workerNodes.ready() ⇒ Promise
A method to check if the minimum required number of workers are ready to serve the calls.
Kind: instance method of WorkerNodes
Returns: Promise - resolves with a WorkerNodes instance
workerNodes.terminate() ⇒ Promise
Starts the process of terminating this instance.
Kind: instance method of WorkerNodes
Returns: Promise - - resolved when the instance is terminated.
WorkerNodesOptions
Describes a WorkerNodes options.
Kind: global class
- WorkerNodesOptions
- .autoStart :
Boolean - .lazyStart :
Boolean - .minWorkers :
Number - .maxWorkers :
Number - .maxTasks :
Number - .maxTasksPerWorker :
Number - .taskTimeout :
Number - .taskMaxRetries :
Number - .workerEndurance :
Number - .workerStopTimeout :
Number
- .autoStart :
options.autoStart : Boolean
Whether should initialize the workers before a first call.
If true, depending on the lazyStart option, it will start the min or max number of workers.
Kind: instance property of WorkerNodesOptions
Default: false
options.lazyStart : Boolean
Whether should start a new worker only if all the others are busy.
Kind: instance property of WorkerNodesOptions
Default: false
options.minWorkers : Number
The minimum number of workers that needs to be running to consider the whole pool as operational.
Kind: instance property of WorkerNodesOptions
Default: 0
options.maxWorkers : Number
The maximum number of workers that can be running at the same time. Defaults to the number of cores the operating system sees.
Kind: instance property of WorkerNodesOptions
options.maxTasks : Number
The maximum number of calls that can be handled at the same time. Exceeding this limit causes MaxConcurrentCallsError to be thrown.
Kind: instance property of WorkerNodesOptions
Default: Infinity
options.maxTasksPerWorker : Number
The number of calls that can be given to a single worker at the same time.
Kind: instance property of WorkerNodesOptions
Default: 1
options.taskTimeout : Number
The number milliseconds after which a call is considered to be lost. Exceeding this limit causes TimeoutError to be thrown and a worker that performed that task to be killed.
Kind: instance property of WorkerNodesOptions
Default: Infinity
options.taskMaxRetries : Number
The maximum number of retries that will be performed over a task before reporting it as incorrectly terminated. Exceeding this limit causes ProcessTerminatedError to be thrown.
Kind: instance property of WorkerNodesOptions
Default: 0
options.workerEndurance : Number
The maximum number of calls that a single worker can handle during its whole lifespan. Exceeding this limit causes the termination of the worker.
Kind: instance property of WorkerNodesOptions
Default: Infinity
options.workerStopTimeout : Number
The timeout value (in milliseconds) for the worker to stop before sending SIGKILL.
Kind: instance property of WorkerNodesOptions
Default: 100
Example
Given /home/joe.doe/workspace/my-module.js:
module.exports = function myTask() {
return 'hello from separate process!';
};you can run it through the worker nodes as follows:
const WorkerNodes = require('worker-nodes');
const myModuleWorkerNodes = new WorkerNodes('/home/joe.doe/workspace/my-module');
myModuleWorkerNodes.call().then(msg => console.log(msg)); // -> 'hello from separate process!'For more advanced examples please refer to the test cases.
Running tests
Check out the library code and then:
$ npm install
$ npm testBenchmarks
To run tests, type:
$ npm install
$ npm run benchmarkIt will run a performance test against the selected libraries:
- data in: an object that consists of a single field that is a 0.5MB random string
- data out: received object stringified and concatenated with another 1MB string
Example results:
results for 100 executions
name time: total [ms] time usr [ms] time sys [ms] worker usr [ms] worker sys [ms] mem rss [MB] worker rss [MB] errors
------------------ ---------------- ------------- ------------- --------------- --------------- ------------ --------------- ------
no-workers 150 239 42 0 0 98 0 0
worker-nodes@1.2.0 1521 646 528 641 367 272 119 0
workerpool@2.1.0 12055 7356 5726 896 212 731 74 0
worker-farm@1.3.1 12124 6711 5501 1577 446 689 74 0
process-pool@0.3.4 12348 6866 5474 1696 458 698 76 0
worker-pool@3.0.2 14029 7633 5604 2285 649 769 104 0
os : Darwin / 15.5.0 / x64
cpu : Intel(R) Core(TM) i7-4578U CPU @ 3.00GHz × 4
node : 6.9.1 / v8: 5.1.281.84See also
sources of inspiration:
License
Copyright 2016 Grupa Allegro Sp. z o.o.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.