aicrowd
A CLI app to interact with AIcrowd
How to install the CLI for development
- Install the API locally https://github.com/AIcrowd/aicrowd-api
- Install python client for Evaluations API https://gitlab.aicrowd.com/aicrowd/evaluations-api-clients/tree/dev/python (for grader commands only)
- Clone this repository https://github.com/AIcrowd/aicrowd-cli.git
- inside the repository
pip install -r requirements.txt
- then do
python setup.py develop
- And now you are good to start developing/using the features
Project Features
Before using the CLI one has to generate the ssh keys and access token required to interact with Gitlab.
- To generate ssh keys and upload them to Gitlab:
aicrowd ssh create
- To genereate personal access token from Gitlab:
aicrowd access_token create
- To create a folder for a challenge with challenge spec:
aicrowd challenge init
The console will ask for the challenge id (which will be provided on the challenge home page)
- To view all the datasets related to a challenge, (has to be run inside the challenge folder):
aicrowd dataset list
- To download a dataset:
aicrowd dataset download
- To Start with a template for the challenge:
aicrowd template list
aicrowd template download
- To quickly start working on a baseline for the challenge:
aicrowd baseline list
aicrowd baseline download
- To make a submission:
aicrowd submit
(Submit requires one to have already run aicrowd ssh create)
- To check the proogress of the submission once submitted:
aicrowd status
AIcrowd Evaluations API Commands
- Login and save the auth token locally.
aicrowd evaluations login -e <email> -p <password>
- Validate the grader repository
git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository> --validate
- Creating a grader
git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository>
- Creating a grader and deploying to AIcrowd
aicrowd keys add -k AICROWD_API_KEY=<aicrowd_api_key>
git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository> --deploy
- Passing the secrets when creating grader
aicrowd evaluations grader create \
--repo <grader_repository> \
-s secret1=value \
-s secret2=value
Many more commands like aicrowd convert
and aicrowd capture
yet to come. Stay tuned.
- aicrowd
- a starter Click command-line application
- automated unit tests you can run with pytest
- a Sphinx documentation project
Getting Started
The project's documentation contains a section to help you get started as a developer or user of the library.
Development Prerequisites
If you're going to be working in the code (rather than just using the library), you'll want a few utilities.
Resources
Below are some handy resource links.
- Project Documentation
- Click is a Python package for creating beautiful command line interfaces in a composable way with as little code as necessary.
- Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Geog Brandl and licnsed under the BSD license.
- pytest helps you write better programs.
- GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files.
Authors
See also the list of contributors who participated in this project.
LicenseCopyright (c) AIcrowd SA
All rights reserved.