Feedstock license: BSD-3-Clause
Home: https://github.com/dagster-io
Package license: Apache-2.0
Summary: The data orchestration platform built for productivity.
Documentation: https://docs.dagster.io
Dagster is a system for building modern data applications. Combining an elegant programming model and beautiful tools, Dagster allows infrastructure engineers, data engineers, and data scientists to seamlessly collaborate to process and produce the trusted, reliable data needed in today's world.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-airbyte
Package license: Apache-2.0
Summary: Package for integrating Airbyte with Dagster.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-aws
Package license: Apache-2.0
Summary: Package for AWS-specific Dagster framework solid and resource components.
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-celery
Package license: Apache-2.0
Summary: Package for using Celery as Dagster's execution engine.
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-census
Package license: Apache-2.0
Summary: Package for integrating Census with Dagster.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-dask
Package license: Apache-2.0
Summary: Package for using Dask as Dagster's execution engine.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-datadog
Package license: Apache-2.0
Summary: Package for datadog Dagster framework components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-dbt
Package license: Apache-2.0
Summary: A Dagster integration for dbt
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-deltalake
Package license: Apache-2.0
Summary: Package for Deltalake-specific Dagster framework op and resource components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-docker
Package license: Apache-2.0
Summary: A Dagster integration for docker
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb
Package license: Apache-2.0
Summary: Package for DuckDB-specific Dagster framework op and resource components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-fivetran
Package license: Apache-2.0
Summary: Package for integrating Fivetran with Dagster.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-github
Package license: Apache-2.0
Summary: Github plugin for Dagster
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-k8s
Package license: Apache-2.0
Summary: A Dagster integration for k8s
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-managed-elements
Package license: Apache-2.0
Summary: Package for Managed elements with Dagster.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-mlflow
Package license: Apache-2.0
Summary: Package for mlflow Dagster framework components
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-msteams
Package license: Apache-2.0
Summary: A Microsoft Teams client resource for posting to Microsoft Teams
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-mysql
Package license: Apache-2.0
Summary: A Dagster integration for MySQL
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-pagerduty
Package license: Apache-2.0
Summary: Package for pagerduty Dagster framework components.
Home: https://github.com/dagster-io/dagster
Package license: Apache-2.0
Summary: Utilities and examples for working with pandas and dagster, an opinionated framework for expressing data pipelines
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-pandera
Package license: Apache-2.0
Summary: Integration layer for dagster and pandera.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-papertrail
Package license: Apache-2.0
Summary: Package for papertrail Dagster framework components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-postgres
Package license: Apache-2.0
Summary: A Dagster integration for postgres
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-prometheus
Package license: Apache-2.0
Summary: A Dagster integration for prometheus
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-shell
Package license: Apache-2.0
Summary: Package for Dagster shell ops.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-slack
Package license: Apache-2.0
Summary: A Slack client resource for posting to Slack
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-snowflake
Package license: Apache-2.0
Summary: Package for Snowflake Dagster framework components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-spark
Package license: Apache-2.0
Summary: Package for Spark Dagster framework components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-ssh
Package license: Apache-2.0
Summary: Package for ssh Dagster framework components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-twilio
Package license: Apache-2.0
Summary: A Dagster integration for twilio
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-wandb
Package license: Apache-2.0
Summary: Package for wandb Dagster components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagstermill
Package license: Apache-2.0
Summary: A Dagster integration for papermill
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-celery-docker
Package license: Apache-2.0
Summary: A Dagster integration for celery-docker
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-celery-k8s
Package license: Apache-2.0
Summary: A Dagster integration for celery-k8s-executor
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-deltalake-pandas
Package license: Apache-2.0
Summary: Package for storing Pandas DataFrames in Delta tables
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/dagster-deltalake-polars
Package license: Apache-2.0
Summary: Package for storing Polars DataFrames in Delta tables.
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb-pandas
Package license: Apache-2.0
Summary: Package for storing Pandas DataFrames in DuckDB.
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb-polars
Package license: Apache-2.0
Summary: Package for storing Polars DataFrames in DuckDB.
Home: https://github.com/dagster-io/dagster/blob/master/python_modules/dagster-duckdb-pyspark
Package license: Apache-2.0
Summary: Package for storing PySpark DataFrames in DuckDB.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-gcp
Package license: Apache-2.0
Summary: Package for GCP-specific Dagster framework solid and resource components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-ge
Package license: Apache-2.0
Summary: Package for GE-specific Dagster framework solid and resource components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-pyspark
Package license: Apache-2.0
Summary: Package for PySpark Dagster framework components.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-snowflake-pandas
Package license: Apache-2.0
Summary: Package for integrating Snowflake and Pandas with Dagster.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-snowflake-pyspark
Package license: Apache-2.0
Summary: Package for integrating Snowflake and PySpark with Dagster.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-gcp-pandas
Package license: Apache-2.0
Summary: Package for storing Pandas DataFrames in GCP.
Home: https://github.com/dagster-io/dagster/tree/master/python_modules/libraries/dagster-gcp-pyspark
Package license: Apache-2.0
Summary: Package for storing PySpark DataFrames in GCP
All platforms: |
Name | Downloads | Version | Platforms |
---|---|---|---|
Installing dagster-libs
from the conda-forge
channel can be achieved by adding conda-forge
to your channels with:
conda config --add channels conda-forge
conda config --set channel_priority strict
Once the conda-forge
channel has been enabled, dagster-airbyte, dagster-aws, dagster-celery, dagster-celery-docker, dagster-celery-k8s, dagster-census, dagster-dask, dagster-datadog, dagster-dbt, dagster-deltalake, dagster-deltalake-pandas, dagster-deltalake-polars, dagster-docker, dagster-duckdb, dagster-duckdb-pandas, dagster-duckdb-polars, dagster-duckdb-pyspark, dagster-fivetran, dagster-gcp, dagster-gcp-pandas, dagster-gcp-pyspark, dagster-ge, dagster-github, dagster-k8s, dagster-managed-elements, dagster-mlflow, dagster-msteams, dagster-mysql, dagster-pagerduty, dagster-pandas, dagster-pandera, dagster-papertrail, dagster-postgres, dagster-prometheus, dagster-pyspark, dagster-shell, dagster-slack, dagster-snowflake, dagster-snowflake-pandas, dagster-snowflake-pyspark, dagster-spark, dagster-ssh, dagster-twilio, dagster-wandb, dagstermill
can be installed with conda
:
conda install dagster-airbyte dagster-aws dagster-celery dagster-celery-docker dagster-celery-k8s dagster-census dagster-dask dagster-datadog dagster-dbt dagster-deltalake dagster-deltalake-pandas dagster-deltalake-polars dagster-docker dagster-duckdb dagster-duckdb-pandas dagster-duckdb-polars dagster-duckdb-pyspark dagster-fivetran dagster-gcp dagster-gcp-pandas dagster-gcp-pyspark dagster-ge dagster-github dagster-k8s dagster-managed-elements dagster-mlflow dagster-msteams dagster-mysql dagster-pagerduty dagster-pandas dagster-pandera dagster-papertrail dagster-postgres dagster-prometheus dagster-pyspark dagster-shell dagster-slack dagster-snowflake dagster-snowflake-pandas dagster-snowflake-pyspark dagster-spark dagster-ssh dagster-twilio dagster-wandb dagstermill
or with mamba
:
mamba install dagster-airbyte dagster-aws dagster-celery dagster-celery-docker dagster-celery-k8s dagster-census dagster-dask dagster-datadog dagster-dbt dagster-deltalake dagster-deltalake-pandas dagster-deltalake-polars dagster-docker dagster-duckdb dagster-duckdb-pandas dagster-duckdb-polars dagster-duckdb-pyspark dagster-fivetran dagster-gcp dagster-gcp-pandas dagster-gcp-pyspark dagster-ge dagster-github dagster-k8s dagster-managed-elements dagster-mlflow dagster-msteams dagster-mysql dagster-pagerduty dagster-pandas dagster-pandera dagster-papertrail dagster-postgres dagster-prometheus dagster-pyspark dagster-shell dagster-slack dagster-snowflake dagster-snowflake-pandas dagster-snowflake-pyspark dagster-spark dagster-ssh dagster-twilio dagster-wandb dagstermill
It is possible to list all of the versions of dagster-airbyte
available on your platform with conda
:
conda search dagster-airbyte --channel conda-forge
or with mamba
:
mamba search dagster-airbyte --channel conda-forge
Alternatively, mamba repoquery
may provide more information:
# Search all versions available on your platform:
mamba repoquery search dagster-airbyte --channel conda-forge
# List packages depending on `dagster-airbyte`:
mamba repoquery whoneeds dagster-airbyte --channel conda-forge
# List dependencies of `dagster-airbyte`:
mamba repoquery depends dagster-airbyte --channel conda-forge
conda-forge is a community-led conda channel of installable packages. In order to provide high-quality builds, the process has been automated into the conda-forge GitHub organization. The conda-forge organization contains one repository for each of the installable packages. Such a repository is known as a feedstock.
A feedstock is made up of a conda recipe (the instructions on what and how to build the package) and the necessary configurations for automatic building using freely available continuous integration services. Thanks to the awesome service provided by Azure, GitHub, CircleCI, AppVeyor, Drone, and TravisCI it is possible to build and upload installable packages to the conda-forge anaconda.org channel for Linux, Windows and OSX respectively.
To manage the continuous integration and simplify feedstock maintenance
conda-smithy has been developed.
Using the conda-forge.yml
within this repository, it is possible to re-render all of
this feedstock's supporting files (e.g. the CI configuration files) with conda smithy rerender
.
For more information please check the conda-forge documentation.
feedstock - the conda recipe (raw material), supporting scripts and CI configuration.
conda-smithy - the tool which helps orchestrate the feedstock.
Its primary use is in the construction of the CI .yml
files
and simplify the management of many feedstocks.
conda-forge - the place where the feedstock and smithy live and work to produce the finished article (built conda distributions)
If you would like to improve the dagster-libs recipe or build a new
package version, please fork this repository and submit a PR. Upon submission,
your changes will be run on the appropriate platforms to give the reviewer an
opportunity to confirm that the changes result in a successful build. Once
merged, the recipe will be re-built and uploaded automatically to the
conda-forge
channel, whereupon the built conda packages will be available for
everybody to install and use from the conda-forge
channel.
Note that all branches in the conda-forge/dagster-libs-feedstock are
immediately built and any created packages are uploaded, so PRs should be based
on branches in forks and branches in the main repository should only be used to
build distinct package versions.
In order to produce a uniquely identifiable distribution:
- If the version of a package is not being increased, please add or increase
the
build/number
. - If the version of a package is being increased, please remember to return
the
build/number
back to 0.