All CK components can be found at cKnowledge.io and in one GitHub repository!
Collective Knowledge repository for CNTK
All CK components for portable MLOps are now collected in one repository!
Introduction
This repository provides high-level, portable and customizable workflows for CNTK. It is a part of our long-term community initiative to unify and automate AI, ML and systems R&D using Collective Knowledge Framework (CK), and to collaboratively co-design efficient SW/HW stack for AI/ML during open ACM ReQuEST competitions as described in the ACM ReQuEST report. All benchmarking and optimization results are available in the public CK repository. See CK getting started guide for more details about CK.
Minimal CK installation
The minimal installation requires:
- Python 2.7 or 3.3+ (limitation is mainly due to unitests)
- Git command line client.
Linux/MacOS
You can install CK in your local user space as follows:
$ git clone http://github.com/ctuning/ck
$ export PATH=$PWD/ck/bin:$PATH
$ export PYTHONPATH=$PWD/ck:$PYTHONPATH
You can also install CK via PIP with sudo to avoid setting up environment variables yourself:
$ sudo pip install ck
Windows
First you need to download and install a few dependencies from the following sites:
- Git: https://git-for-windows.github.io
- Minimal Python: https://www.python.org/downloads/windows
You can then install CK as follows:
$ pip install ck
or
$ git clone https://github.com/ctuning/ck.git ck-master
$ set PATH={CURRENT PATH}\ck-master\bin;%PATH%
$ set PYTHONPATH={CURRENT PATH}\ck-master;%PYTHONPATH%
CNTK CK workflow installation (Linux or Windows)
Prerequisites
Linux:
- If problem with ligpng:
wget -q -O /tmp/libpng12.deb http://mirrors.kernel.org/ubuntu/pool/main/libp/libpng/libpng12-0_1.2.54-1ubuntu1_amd64.deb && sudo dpkg -i /tmp/libpng12.deb && rm /tmp/libpng12.deb
CPU
$ ck pull repo:ck-cntk
$ ck install package --tags=lib,cntk,vcpu
GPU including 1bit SGD
$ ck pull repo:ck-cntk
$ ck install package --tags=lib,cntk,vgpu
Checking classification example (and automatically installing available MXNet model(s) via CK)
$ ck run program:cntk
- Select 'demo-feed-forward-net' command line
- Observe result
Trying CK CNTK in Docker
Build Docket image:
$ ck build docker:ck-cntk-py35 --sudo
You can now run this Docker image:
$ ck run docker:ck-cntk-py35 --sudo
$ ck run program:cntk
Skip --sudo if you have local Docker installation.
Related Publications with long term CK vision
@inproceedings{ck-date16,
title = {{Collective Knowledge}: towards {R\&D} sustainability},
author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},
booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},
year = {2016},
month = {March},
url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}
}
@inproceedings{cm:29db2248aba45e59:0c7348dfbadd5b95,
author = {{Moreau}, Thierry and {Lokhmotov}, Anton and {Fursin}, Grigori},
title = "{Introducing ReQuEST: an Open Platform for Reproducible and Quality-Efficient Systems-ML Tournaments}",
journal = {ArXiv e-prints},
archivePrefix = "arXiv",
eprint = {1801.06378},
primaryClass = "stat.ML",
keywords = {Statistics - Machine Learning, Computer Science - Learning, Computer Science - Software Engineering},
year = 2018,
month = jan,
adsurl = {http://adsabs.harvard.edu/abs/2018arXiv180106378M},
adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}