sf-wind / ck

Collective Knowledge is a cross-platform customizable Python framework to share artifacts as reusable components with JSON API; assemble experimental workflows (such as multi-objective co-design of DNN); automate package installation; crowdsource and reproduce experiments; unify predictive analytics; enable interactive articles. Project website:

Linux/MacOS: Windows: Coverage:

Introduction

Originally, we developed Collective Knowledge framework (CK) to enable collaborative, reproducible, sustainable and agile research and experimentation for our partners based on Wikipedia and DevOps principles. However, after a growing number of use cases, we decided to continue it as a community effort to help researchers and developers:

• share their artifacts as portable, customizable and reusable Python plugins with a unified JSON API and meta information (programs, benchmarks, data sets, libraries, tools, predictive models, etc);
• quickly prototype research workflows from shared artifacts (such as customizable, multi-objective and input-aware autotuning and crowdsource/reproduce/build upon various experiments across diverse hardware and software stack provided by volunteers;
• make their research workflows portable and adaptable to any platform with Linux, Windows, MacOS and Android using CK cross-platform customizable package and environment manager;
• unify access to predictive analytics (TensorFlow, Caffe, scikit-learn, R, DNN, etc) via unified CK JSON API and CK web service;
• enable reproducible, interactive and "live" articles (see example);
• support Artifact Evaluation at premier systems and ML conferences;
• enable open, reproducible, multi-objective optimization and co-design tournaments of the whole AI/SW/HW stack (see ReQuEST).

We now work closely with the CK community, Computer Systems conferences, machine learning and systems community and ACM to gradually standardize all APIs and meta description of all shared artifacts and workflows to enable open and reproducible systems research

Feel free to contact CK authors if you want to participate in our activities, or contact CK mailing list if you need voluntarily help from the CK community to convert your artifacts and workflows to the CK format.

CK resources

• cKnowledge.org - project website with the latest news
• Academic and industrial partners
• CK documentation including "Getting Started Guide"
  • Shared CK repositories
  • Shared CK plugins
  • Shared software detection plugins
  • Shared CK packages to automate installation of workflows across diverse platforms
• Reproducible SW/HW co-design competitions for deep learning and other emerging workloads using CK
• Practical use cases
• CK-powered AI hub
• CK-related publications
• CK Mailing list

Reference article

• Online PDF

@inproceedings{ck-date16,
    title = {{Collective Knowledge}: towards {R\&D} sustainability},
    author = {Fursin, Grigori and Lokhmotov, Anton and Plowman, Ed},
    booktitle = {Proceedings of the Conference on Design, Automation and Test in Europe (DATE'16)},
    year = {2016},
    month = {March},
    url = {https://www.researchgate.net/publication/304010295_Collective_Knowledge_Towards_RD_Sustainability}
}

License

• Permissive 3-clause BSD license. (See LICENSE.txt for more details).

Testimonials and awards

• 2017: Test of time award for our ACM CGO'07 paper that led to creating CK
• 2017: ARM video about CK-powered optimization of DNN at the Embedded Vision Summit
• 2017: ACM evaluates CK technology to share experimental workflows in Digital Libraries
• 2017: CK-powered experimental workflow from the UCambridge received a distinguished artifact award at the CGO'17: GitHub, PDF, CK dashboard snapshot
• 2016: Imperial College London colleagues won HiPEAC TT award for their CK-based project to crowdsource OpenCL bug detection: GitHub;
• 2016: General Motors and dividiti use CK to crowdsource benchmarking and optimization of CAFFE: public CK repo
• 2015: ARM and the cTuning foundation use CK to accelerate computer engineering: HiPEAC Info'45 page 17, ARM TechCon'16 presentation and demo, public CK repo
• 2014: HiPEAC technology transfer award

Acknowledgments

CK development is coordinated by the cTuning foundation (non-profit research organization) and dividiti. We would like to thank the EU TETRACOM 609491 Coordination Action for initial funding, Microsoft for sponsoring the hosting of a public CK optimization repository in the Azure cloud, and all our partners for continuing support. We are also extremely grateful to all volunteers for their valuable feedback and contributions.

Minimal installation

The minimal installation requires:

• Python 2.7 or 3.3+ (limitation is mainly due to unitests)
• Python PIP (if you would like to install CK via PIP)
• Git command line client.

On Ubuntu, you can install these dependencies via

$ apt-get install -y python python-pip git

On Windows, you can download and install these tools from the following sites:

• Git: https://git-for-windows.github.io
• Minimal Python: https://www.python.org/downloads/windows
• Anaconda scientific Python with all packages: https://www.continuum.io/downloads#_windows

You can now install a stable CK version via PIP simply as following (you may need to prefix it with "sudo" on Linux):

$ pip install ck

Alternatively, you can install a development CK version in your local user space via GIT as following:

 $ git clone https://github.com/ctuning/ck.git ck

and then add CK to PATH on Linux as following:

 $ export PATH=$PWD/ck/bin:$PATH

or on Windows as following:

 $ set PATH={CURRENT PATH}\ck\bin;%PATH%

Further installation details can be found here.

Trying CK using Docker image

If you would like to try CK without installing it, you can run the following Docker image:

 $ docker run -it ctuning/ck

Also note that we added Docker automation to CK (to help evaluate artifacts at the conferences, share interactive and reproducible articles or crowdsource experiments for example).

Please check 'ck-docker' repository at GitHub:

 $ ck show repo:ck-docker

You can download and view one of our CK-based interactive and reproducible articles as following:

 $ ck pull repo:ck-docker
 $ ck run docker:ck-interactive-article --browser (--sudo)

See the list of other CK-related Docker images here.

However note that the main idea behind CK is to make experimental workflows and artifacts adaptable to any latest user environment and hardware. Thus, rather than just being archived, they can be continuously reused, customized, improved and built upon based on Wikipedia and DevOps principles!

Practical use cases

See the list of real use cases by the growing CK community.

Questions/comments/discussions?

• Mailing list for open, collaborative and reproducible R&D including knowledge preservation, sharing and reuse: http://groups.google.com/group/collective-knowledge
• Mailing list for software and hardware multi-objective (performance/energy/accuracy/size/reliability/cost) benchmarking, autotuning, crowdtuning and run-time adaptation: http://groups.google.com/group/ctuning-discussions
• Public wiki with CK-powered open challenges in computer engineering: https://github.com/ctuning/ck/wiki/Research-and-development-challenges

CK authors

• Grigori Fursin, cTuning foundation / dividiti
• Anton Lokhmotov, dividiti