Apache 2.0 licensed.
IOpipe is a toolkit for building and orchestrating event-driven and serverless applications. These apps may run locally or in the cloud via AWS Lambda, Google Cloud Functions, or Azure Functions.
IOpipe can:
- Chain AWS Lambda Functions and local functions
- Convert NodeJS functions into serverless functions
- Perform GET and POST HTTP requests
- Parallelize data into serverless workers.
We call our serverless functions "kernels". Kernels take and transform input and communicate over the networking, operating in a fashion to Unix pipes. A kernel may receive input or send output to/from web service requests, functions, or local applications.
The NodeJS SDK provides a generic callback chaining mechanism which allows mixing HTTP(S) requests/POSTs, function calls, and kernels. Callbacks receive the return of the previous function call or HTTP body.
The callback variable received by a function is also an AWS Lambda-compatible "context" object. Because of this, you can chain standard callback-based NodeJS functions, and functions written for AWS Lambda.
var iopipe = require("iopipe")()
/* Get HTTP data, process it with SomeScript, and POST the results.
Note that com.example.SomeScript would be present in .iopipe/filter_cache/ */
iopipe.exec("http://localhost/get-request",
"com.example.SomeScript",
"http://otherhost.post")
// Users may chain functions and HTTP requests.
iopipe.exec(function(_, callback) { callback("something") },
function(arg, callback) { callback(arg) },
"http://otherhost.post",
your_callback)
// A function may also be returned then executed later.
var f = iopipe.define("http://fetch", "https://post")
f()
// A defined function also accepts parameters
var echo = require("iopipe-echo")
var f = iopipe.define(echo, console.log)
f("hello world")
/* Create an AWS Lambda function from any NodeJS function /w callback.
The callback becomes the equivilent of a done or success call on AWS. */
export.handler = iopipe.define(function(event, callback) {
console.log(event)
callback()
})
/* Of course, this method chaining also works for creating AWS Lambda code.
This example will fetch HTTP data from the URL in the event's 'url' key
and return a SHA-256 of the retrieved content. */
var crypto = require("crypto")
export.handler = iopipe.define(iopipe.property("url"),
iopipe.fetch,
(event, callback) => {
callback(crypto
.createHash('sha256')
.update(event)
.digest('hex'))
})
IOpipe also acts as an AWS Lambda Client where a Lambda function may be specified by its URN and included in the execution chain:
var iopipe = require("iopipe")()
var iopipe_aws = require("iopipe")(
exec_driver: 'aws'
exec_driver_opts: {
region: 'us-west-1',
access_key: 'itsasecrettoeverybody',
secret_key: 'itsasecrettoeverybody'
}
)
var crypto = require("crypto")
export.handler = iopipe_aws.define("urn:somefunction",
"urn:anotherfunction",
iopipe.property("property-of-result"),
iopipe.fetch, # fetch that as a URL
(event, callback) => {
callback(JSON.parse(event))
},
iopipe.map(
iopipe_aws.define(
"urn:spawn_this_on_aws_for_each_value_in_parallel"
)
))
For more information on using the NodeJS SDK, please refer to its documentation: https://github.com/iopipe/iopipe/blob/master/docs/nodejs.md
Requests and responses are translated using kernels, and may pipe to other kernels, or to/from web service endpoints.
Kernels simply receive request or response data and output translated request or response data.
Example:
module.exports = function(input, context) {
context.done("I'm doing something with input: {0}".format(input))
}
Functions should expect a "context" parameter which may be called directly as a callback, but also offers the methods 'done', 'success', and 'fail'. Users needing, for any reason, to create a context manually may call iopipe.create_context(callback).
For more on writing filters see: https://github.com/iopipe/iopipe/blob/master/docs/kernels.md
A Go-based CLI exists to create and export npm modules, share code, and provide runtime of magnetic kernels.
Find this tool in the IOpipe-Golang repo.
Kernels are executed in individual virtual machines whenever allowed by the executing environment. The definition of a virtual machine here is lax, such that it may describe a Javascript VM, a Linux container, or a hardware-assisted x86 virtual machine. Users should exercise caution when running community created kernels.
It is a project priority to make fetching, publishing, and execution of kernels secure for a production-ready 1.0.0 release.
Modules are fetched and stored using sha256 hashes, providing an advantage over module-hosting mechanisms which are based simply on a name and version. Future versions of IOpipe will likely implement TUF for state-of-the-art software assurance.
Contact security@iopipe.com for questions.
Apache 2.0