A super simple way to distribute rendering tasks across multiple machines.
pip install distributaur
- Python 3.8 or newer (tested on 3.10)
- Redis server
- Celery
Clone the repository and navigate to the project directory:
git clone https://github.com/RaccoonResearch/distributaur.git
cd distributaur
Install the required packages:
pip install -r requirements.txt
Install the distributaur package:
python setup.py install
Create a .env
file in the root directory of your project or set environment variables to match your setup:
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_USER=user
REDIS_PASSWORD=password
VAST_API_KEY=your_vast_api_key
To start processing tasks, you need to run a worker. You can start a worker using the provided script:
sh scripts/kill_redis_connections.sh # Optional: to clear previous Redis connections
celery -A distributaur.distributaur worker --loglevel=info
To run an example task and see Distributaur in action, you can execute the example script provided in the project:
python example.py
This script configures the environment, registers a sample function, dispatches a task, and monitors its execution.
- register_function(func): Decorator to register a function that can be called as a task.
- execute_function(func_name, args): Dispatch a registered function as a task with specified arguments.
- update_function_status(task_id, status): Update the status of a task in Redis.
- rent_nodes(max_price, max_nodes, image, api_key): Rent nodes from VAST.ai based on specified criteria.
- terminate_nodes(nodes): Terminate rented nodes on VAST.ai.
Contributions are welcome! For major changes, please open an issue first to discuss what you would like to change.
This project is licensed under the MIT License - see the LICENSE
file for details.