Super allows you to run a UNIX command line in parallel, using
auto-scaling Cloud compute resources, against a set of Cloud data.
Super takes care of provisioning the right amount of compute, memory,
and disk capacity, scheduling your jobs, granting the needed data
access authority to your work, and streaming out logs --- all in one
command: super run.
Super knows how to track the progress of any job against your Cloud data. You will know, without any extra work, how much of your Cloud input files has been processed!
Super uses containers running in IBM Cloud Code Engine as the compute layer, and gives your jobs access to data via IBM Cloud Object Storage.
The latest build of Super is available here. After you have downloaded your release:
tar jxf Super-darwin-x64.tar.bz2
export PATH=$PWD/Super-darwin-x64/Super.app/Contents/Resources:$PATH
superYou should now see usage information for Super, including the main sub-commands:
We suggest first trying super up, which
will validate your prerequisites. If you are good to go, then you can
try super run -p5 -- echo hello, which will execute that command as
five Cloud jobs. If this all looks good, then proceed to the
Tutorial in Further Reading.
To read more about a UNIX Way for the Cloud, read on: