CLI to set up a Numerai Compute node in AWS (Amazon Web Services) and deploy your models to it. This solution is architected to cost less than $5/mo on average (actual costs may vary). It has been tested and found working on MacOS/OSX, Windows 10, and Ubuntu 18.04, but should theoretically work anywhere that Docker and Python 3 are available.
If you have other questions or feedback, please join us on the RocketChat #compute Channel. Before posting a messaging the Rocketchat channel or creating a Github issue, please read through the following:
If you still cannot find a solution or answer, include the following information with your issue/message:
- The command you ran that caused the error
- Version from running
pip3 show numerai-cli
- System Information from running
- Mac:
system_profiler SPSoftwareDataType && system_profiler SPHardwareDataType
- Linux:
lsb_release -a && uname -a
- Windows:
powershell -command "Get-ComputerInfo"
- Mac:
If you do not include this information, we cannot help you.
- An AWS (Amazon Web Services) Account
- AWS Billing set up
- AWS Access Keys (See the wiki for detailed instructions)
- Numerai API Keys
- Python and Docker (see Environment Setup)
Our friend Arbitrage created this tutorial to help you get set up with your AWS/Numerai API Keys
For your convenience, we've included setup scripts in the scripts
directory that will ensure the prerequisites are installed.
You can download and run the setup script for your OS with one of the following commands:
-
Mac Terminal (cmd + space, type
terminal
, selectterminal.app
):curl https://raw.githubusercontent.com/numerai/numerai-cli/master/scripts/setup-mac.sh | bash
-
Ubuntu 18 Terminal (ctrl + alt + t):
apt update && apt install -y libcurl4 curl && curl https://raw.githubusercontent.com/numerai/numerai-cli/master/scripts/setup-ubu18.sh | bash
-
Windows Command Prompt (windows key, type
cmd
, select Command Prompt):powershell -command "$Script = Invoke-WebRequest 'https://raw.githubusercontent.com/numerai/numerai-cli/master/scripts/setup-win10.ps1'; $ScriptBlock = [ScriptBlock]::Create($Script.Content); Invoke-Command -ScriptBlock $ScriptBlock"
If you run into issues running one of these scripts, please report immediately to RocketChat Compute Channel.
After the setup script confirms Python and Docker, install numerai-cli
via:
pip3 install numerai-cli
If you know you have all the prerequisites and have your AWS and Numerai API Keys at hand, you can run these commands to get the example application running in minutes:
mkdir example-numerai
cd example-numerai
numerai setup
numerai docker copy-example
numerai docker deploy
Your compute node is now setup and ready to run. It saves the webhook URL in .numerai/submission_url.txt
that triggers your docker container.
You can configure your Numerai account to use this webhook by entering it in the "Compute" section.
It will be called Saturday morning right after a new round opens, and if your job fails (one ore more models have not submitted successfully)
then it will be triggered again around 24 hours later.
NOTE: The default example does not make stake changes; you will still have to do that manually. Please refer to the numerapi docs for the methods you must call to do this.
-
Test webhook URL (schedule a job in the cloud):
numerai compute test-webhook
-
Check your job status:
numerai compute status
-
View logs for "RUNNING" Job:
numerai compute logs
-
You can test locally too:
numerai docker run
-
NOTES:
- You can also view logs in the console for your ECS Cluster and your Container
- Scheduling a job does not immediately run the container. This takes several minutes the first time it runs, with subsequent runs starting a lot faster.
To get descriptions of each command available, run one of these:
numerai
numerai --help
numerai [command] --help
numerai [command] [sub-command] --help
Destroy the AWS environment
numerai destroy
And then uninstall the package:
pip3 uninstall numerai-cli