Bacalhau is a platform for public, transparent, and optionally verifiable distributed computation that helps you manage your parallel processing jobs. Bacalhau enables users to run arbitrary docker containers and wasm images as tasks against data stored in IPFS. This architecture is referred to as Compute Over Data (CoD). Bacalhau was coined from the Portuguese word for salted Cod fish.
- Features
- Getting started
- Documentation
- Developers guide
- Ways to contribute
- Current state of Bacalhau
- License
- Process jobs fast: Jobs in Bacalhau are processed where the data was created and all jobs are parallel by default.
- Low cost: You get to save on ingress/egress fees you encounter whenever you move data around since jobs are processed where the data is created.
- Secure: Data is not collected in a central location before processing, meaning all scrubbing and security can be applied at the point of collection.
- Large-scale data: Bacalhau operates on a network of open compute resources made available to serve any data processing workload. With Bacalhau, you can batch process petabytes (quadrillion bytes) of data.
Go to the folder directory that you want to store your job results
Install the bacalhau client
curl -sL https://get.bacalhau.org/install.sh | bash
Submit a "Hello World" job
bacalhau docker run ubuntu echo Hello World
Download your result
bacalhau get 63d08ff0..... # make sure to use the right job id from the docker run command
For a more detailed tutorial, check out our Getting Started tutorial.
- Understand Bacalhau Concepts
- Get an overview of the different usecases that you can use with Bacalhau.
- To see Bacalhau in action, check out the Bacalhau Examples
- You can check out this featured example video tutorial Text to image- Stable Diffusion GPU. You can watch more tutorials here
📚 Read the Bacalhau docs guide here! 📚
The Bacalhau docs is the best starting point as it contains all the information to ensure that everyone who uses Bacalhau is doing so efficiently.
Developers can spin up bacalhau and run a local demo using the devstack
command.
Please see docs/running_locally.md for instructions. Also, see docs/debugging_locally.md for some useful tricks for debugging.
Bacalhau's CI pipeline performs a variety of linting and formatting checks on new pull requests.
To have these checks run locally when you make a new commit, you can use the precommit hook in ./githooks
:
make install-pre-commit
# check if pre-commit works
make precommit
If you want to run the linter manually:
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sudo sh -s -- -b /usr/local/go/bin
golangci-lint --version
make lint
The config lives in .golangci.yml
We are excited to hear your feedback!
- For issues and feature requests, please open a GitHub issue.
- For questions, give feedback or answer questions that will help other user product please use GitHub Discussions.
- To engage with other members in the community, join us in our slack community
#bacalhau
channel 🙋
All manner of contributions are more than welcome!
We have highlighted the different ways you can contribute in our contributing guide. You can be part of community discussions, development, and more.
Building never stops 🛠️. Bacalhau is a work in progress!. Learn more about our future plans in this roadmap document