aicrowd

A CLI app to interact with AIcrowd

How to install the CLI for development

Install the API locally https://github.com/AIcrowd/aicrowd-api
Install python client for Evaluations API https://gitlab.aicrowd.com/aicrowd/evaluations-api-clients/tree/dev/python (for grader commands only)
Clone this repository https://github.com/AIcrowd/aicrowd-cli.git
inside the repository pip install -r requirements.txt
then do python setup.py develop
And now you are good to start developing/using the features

Project Features

Before using the CLI one has to generate the ssh keys and access token required to interact with Gitlab.

To generate ssh keys and upload them to Gitlab:

aicrowd ssh create

To genereate personal access token from Gitlab:

aicrowd access_token create

To create a folder for a challenge with challenge spec:

aicrowd challenge init

The console will ask for the challenge id (which will be provided on the challenge home page)

To view all the datasets related to a challenge, (has to be run inside the challenge folder):

aicrowd dataset list

To download a dataset:

aicrowd dataset download

To Start with a template for the challenge:

aicrowd template list
aicrowd template download

To quickly start working on a baseline for the challenge:

aicrowd baseline list
aicrowd baseline download

To make a submission:

aicrowd submit

(Submit requires one to have already run aicrowd ssh create)

To check the proogress of the submission once submitted:

aicrowd status

AIcrowd Evaluations API Commands

aicrowd evaluations login -e <email> -p <password>

Validate the grader repository

git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository> --validate

Creating a grader

git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository>

Creating a grader and deploying to AIcrowd

aicrowd keys add -k AICROWD_API_KEY=<aicrowd_api_key>
git clone <grader_repository>
cd <repository>
aicrowd evaluations grader create --repo <grader_repository> --deploy

Passing the secrets when creating grader

aicrowd evaluations grader create \ 
    --repo <grader_repository> \ 
    -s secret1=value \ 
    -s secret2=value

Many more commands like aicrowd convert and aicrowd capture yet to come. Stay tuned.

aicrowd
a starter Click command-line application
automated unit tests you can run with pytest
a Sphinx documentation project

Getting Started

The project's documentation contains a section to help you get started as a developer or user of the library.

Development Prerequisites

If you're going to be working in the code (rather than just using the library), you'll want a few utilities.

Resources

Below are some handy resource links.

Project Documentation
Click is a Python package for creating beautiful command line interfaces in a composable way with as little code as necessary.
Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Geog Brandl and licnsed under the BSD license.
pytest helps you write better programs.
GNU Make is a tool which controls the generation of executables and other non-source files of a program from the program's source files.

Authors

S.P. Mohanty - Initial work - github
Nikhil Rayaprolu - Initial work (Current Maintainer) - github

See also the list of contributors who participated in this project.

nikhilrayaprolu / aicrowd-cli