What is Raven Distribution Framework?
The foundation for any Machine Learning or Deep Learning Framework. Simply put, it is more like a decentralized calculator, comparable to a decentralized version of the IBM machines that were used to launch the Apollo astronauts. Apart from building ML/DL frameworks, a lot more can be done on it, such as maximizing yield on your favorite DeFi protocols like Compound and more!
The Raven Distribution Framework (RDF) is a community-developed implementation of the decentralized computing model outlined by Raven Protocol. Within the Raven ecosystem today, there are two main actors:
- Developers: Create models that need to be trained
- Clients: Provide computational power to train the models
For Developers, there are three core libraries that drive the main function for computation distribution (RavOp, RavSock, and RavFTP), along with a growing list of libraries that extend the core to be more developer-friendly to use.
For Clients, there are two libraries, ravpy is the python client for federated and distributed computing and the javascript library RavJS enables anyone with a browser to contribute processing power. Additional clients are in consideration, such as Go and Rust (looking for community devs!).
- RavOp: Core operations models for distributed computation
- RavSock: Socket server to moderate client connections
- RavFTP: FTP server to facilitate the transfer of files
- RavML: Machine learning specific library
- RavDL (Coming soon): Deep learning specific library
- RavViz: A dashboard to visualize operations and client connections
- Ravpy: Python client for federated and distributed computing
- RavJS: Javascript library to retrieve and calculate operations
conda create -n <env_name> python=3.8
git clone https://github.com/ravenprotocol/raven-distribution-framework.git
For the latest build
sh setup.sh
For the stable build
sh setup_stable.sh
Navigate to ravsock/config.py and set the FTP_ENVIRON_DIR variable to the bin folder of your python virtual environment. For instance:
FTP_ENVIRON_DIR = "~/miniconda/envs/<env_name>/bin"
Note: Set ENCRYPTION = True in the same file if a layer of homomorphic encryption needs to be added for Federated Analytics.
Set RDF_DATABASE_URI in the same file.
RDF_DATABASE_URI = "sqlite:///rdf.db?check_same_thread=False"
Create database with tables required for the project.
python reset.py
The server is now configured correctly and ready to be fired up.
Ravsock is a crucial component of RDF that facilitates both federated and distributed functionalities of the framework.
It sits between the developer(who creates ops and writes algorithms) and the contributor who contributes the idle computing power. Its scheduling algorithm oversees the distribution and statuses of different Ops, Graphs and Subgraphs across multiple Clients.
python3 run_ravsock.py
Kindly visit FEDERATED_ANALYTICS.md for more info on creating and working with custom Federated Ops.
Kindly visit DISTRIBUTED_COMPUTING.md for more on creating graphs, initializing distributed clients in web browser and working with custom Ops to develop distributed ML algorithms.
Contributions are what make the open source community such a wonderful place to learn, be inspired, and create. You may contribute to our individual Repositories. Please read our contributor guide.
Any help you can give is much appreciated.
This project is licensed under the MIT License - see the LICENSE file for details