Joydeep75 / orchest

Build data pipelines, the easy way 🛠️

Build data pipelines, the easy way 🙌

No frameworks. No YAML. Just write Python and R code in Notebooks.

💡 Watch the full narrated video to learn more about building data pipelines in Orchest.

Read the docs; get the code; ask us anything!

Features

For a complete list of Orchest's features, check out the overview in our docs!

• Visually construct pipelines through our user-friendly UI.
• Code in Notebooks.
• Run any subset of a pipeline directly or on a cron-like schedule.
• Easily define your dependencies to run on any machine.

Who should use Orchest?

• Data Scientists who want to rapidly prototype.
• Data Scientists who like to work in Notebooks.
• Data Scientists who are looking to create pipelines through a visual interface instead of YAML.

Installation

NOTE: Orchest is in alpha.

For GPU support, language dependencies other than Python, and other installation methods, such as building from source, please refer to our installation docs.

Requirements

• Docker 🐳 (Engine version of >= 20.10.7 – run docker version to check).

NOTE: On Windows, Docker has to be configured to use WSL 2. Make sure to clone Orchest inside the Linux environment. For more info and installation steps for Docker with WSL 2 backend, please visit https://docs.docker.com/docker-for-windows/wsl/.

Linux, macOS and Windows

Alternatively, signup for the free-of-charge Orchest Cloud and get a fully configured Orchest instance out of the box!

git clone https://github.com/orchest/orchest.git && cd orchest
./orchest install

# Verify the installation.
./orchest version --ext

# Start Orchest.
./orchest start

Now that you have installed Orchest, get started with our quickstart tutorial, check out pipelines made by your fellow users, or have a look at our video tutorials explaining some of Orchest's core concepts.

License

The software in this repository is licensed as follows:

• All content residing under the "orchest-sdk/" directory of this repository is licensed under the "Apache-2.0" license as defined in "orchest-sdk/LICENSE".
• Content outside of the above mentioned directory is available under the "AGPL-3.0" license.

Slack Community

Join our Slack to chat about Orchest, ask questions, and share tips.

Contributing

Contributions are more than welcome! Please see our contributor guides for more details.

Contributors