datafusion

There are 3 repositories under datafusion topic.

apache / arrow-datafusion
Apache DataFusion SQL Query Engine
arrow big-data dataframe datafusion olap python query-engine rust sql
Language:Rust 4928
ibis-project / ibis
the portable Python dataframe library
bigquery clickhouse dask database datafusion duckdb impala mssql mysql pandas polars postgresql pyarrow pyspark python snowflake sql sqlalchemy sqlite trino
Language:Python 4181
roapi / roapi
Create full-fledged APIs for slowly moving datasets without writing a single line of code.
analytics arrow blob-storage cloud-native columnar datafusion datasets delta-lake graphql in-memory-database parquet query query-frontends rest-api rust s3 sql static-datasets
Language:Rust 3076
ballista-compute / ballista
Distributed compute platform implemented in Rust, and powered by Apache Arrow.
rust arrow datafusion kubernetes distributed spark kotlin java scala dataframe jvm
Language:Rust 2334
lakesoul-io / LakeSoul
LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.
lakesoul datalake lakehouse spark flink streaming big-data postgresql rust sql huggingface python pytorch arrow datafusion vectorized velox
Language:Java 2304
apache / arrow-datafusion-comet
Apache Arrow DataFusion Comet Spark Accelerator
arrow datafusion rust spark
Language:Rust 361
seafowl
splitgraph / seafowl
Analytical database for data-driven Web applications 🪶
api database datafusion delta-lake delta-rs edge http rust serverless sql visualization
Language:Rust 353
kamu-data / kamu-cli
New generation decentralized data lake and a streaming data pipeline
spark flink sql jupyter blockchain kamu open-data open-data-fabric data-management data-as-code data-science datafusion
Language:Rust 275
PRQL / prql-query
Query and transform data with PRQL
csv csv-converter csv-files data database datafusion duckdb json json-files parquet parquet-files parquet-tools parquet-viewer postgresql prql query querydsl
Language:Rust 121
datafusion-contrib / datafusion-python
Python binding for DataFusion
arrow datafusion maturin python
Language:Python 60
jorgecarleitao / datafusion-python
A Python library to run analytics workloads with the performance of Rust, the flexibility of Python and O(1) cost in moving data between the two. Uses Apache Arrow in-memory format and respective query engine DataFusion.
rust python udfs datafusion
Language:Rust 60
datafusion-contrib / datafusion-objectstore-s3
S3 as an ObjectStore for DataFusion
datafusion rust
Language:Rust 57
hw2499 / etl-engine
etl engine 轻量级跨平台流批一体ETL引擎数据抽取-转换-装载 ETL engine lightweight cross platform batch flow integration ETL engine data extraction transformation loading
cdc clickhouse crontab datafusion elastic etl etl-engine go hbase hive influx kafka mysql mysqlbinlog oracle postgresql prometheus redis rocketmq sqlite3
Language:Go 54
JanKaul / iceberg-rust
Rust implementation of Apache Iceberg with integration for Datafusion
arrow iceberg rust datafusion
Language:Rust 53
datafusion-contrib / datafusion-java
Java binding to Apache Arrow DataFusion
arrow ballista datafusion java
Language:Java 51
shauryashaurya / learn-data-munging
Notes on Data Engineering with Pandas, PySpark, Dask, Ray, Arrow DataFusion, Polars etc.
arrow dask dask-distributed data-engineering datafusion jupyter numpy pandas polars pyspark ray spark
Language:Jupyter Notebook 41
splitgraph / seafowl-gcsfuse
Scale to zero Seafowl hosting with Cloud Run
faas gcp rust severless datafusion seafowl
Language:Dockerfile 39
metrico / iox-community
Community InfluxDB 3.0 "IOx" static builds + containers + Examples for Developers & Integrators. Experiment with low-cost storage, unlimited cardinality and FlightSQL APIs
datafusion influxdb iox lineprotocol flightsql flux influx musl arrow rust influxdb3
Language:Shell 34
treebee / elixir-arrow
Experimental Elixir bindings for Apache Arrow including Parquet and DataFusion
arrow datafusion parquet query-engine
Language:Rust 32
baggiponte / awesome-pandas-alternatives
Awesome list of alternative dataframe libraries in Python.
awesome awesome-list python pandas sql apache-arrow arrow polars datafusion rapidsai datatable dask cudf blazingsql
29
wheretrue / exon
Exon is an OLAP query engine specifically for biology and life science applications.
arrow bioinformatics datafusion ngs proteomics rust sql
Language:Rust 27
datafusion-contrib / datafusion-c
C language bindings for DataFusion
apache-arrow c datafusion glib sql
Language:C 14
fmenat / MultiviewCropClassification
Public repository of our IGARSS 2023 submission
agriculture-research crop-classification crop-type-mapping croptypes data-fusion datafusion multi-modal-learning multi-view-learning multimodal-learning multisensor-fusion multiview-learning remote-sensing remote-sensing-image crop-type-mapp
Language:Python 12
blaze-init / spark-blaze-extension
Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.
spark datafusion arrow
Language:Shell 11
ModelarData / ModelarDB-RS
ModelarDB: Model-Based Time Series Management from Edge to Cloud
apache-arrow datafusion industrial-iot rust time-series time-series-database
Language:Rust 10
grouzen / zio-apache-arrow
ZIO-powered Apache Arrow library
arrow datafusion scala zio
Language:Scala 6
roeap / flight-fusion
arrow flight deltalake data-science datafusion
Language:Rust 6
sal-openlab / datafusion-server
Rust DataFusion Server
arrow datafusion rust sql
Language:Rust 5
splitgraph / experimental-datafusion-webassembly
proof-of-concept: compile datafusion to `wasm32-wasi` (run in `wasmedge`) and `wasm32-unknown-unknown` (run in browser)
arrow datafusion webassembly wasm32-unknown-unknown wasm32-wasi wasmedge
4
Caoxuheng / HyMS
Hyperspectral Image Super-resolution via Multi-stage Scheme without Employing Spatial Degradation
datafusion hyperspectral-imaging super-resolution total-variation
Language:Python 3
jychen7 / datafusion-bigtable
Bigtable data source for Apache Arrow Datafusion
rust sql bigtable arrow datafusion olap
Language:Rust 2
matsadler / bishop
Query MongoDB via Apache Arrow and DataFusion
apache-arrow datafusion mongodb
Language:Rust 2
milenkovicm / adhesive
Apache Datafusion JVM User Defined Functions (UDF), integration nobody asked for 😀
datafusion java rust sql userdefined-functions arrow jni jvm udf udf-libraries bytecode-compiler compiler
Language:Rust 2
milenkovicm / torchfusion
Torchfusion is a very opinionated torch inference on datafusion.
datafusion rust sql torch userdefined-functions pytorch batch-inference inference machine-learning
Language:Rust 2
hengfeiyang / how-query-engines-work-zh-CN
How Query Engines Work 中文版
arrow ballista datafusion parquet
1
jfrazier-eth / file-fusion
A file explorer for data warehouses
data-fusion datafusion nextjs objectstore react rust s3 sql tauri
Language:TypeScript 1

datafusion

apache / arrow-datafusion

ibis-project / ibis

roapi / roapi

ballista-compute / ballista

lakesoul-io / LakeSoul

apache / arrow-datafusion-comet

splitgraph / seafowl

kamu-data / kamu-cli

PRQL / prql-query

datafusion-contrib / datafusion-python

jorgecarleitao / datafusion-python

datafusion-contrib / datafusion-objectstore-s3

hw2499 / etl-engine

JanKaul / iceberg-rust

datafusion-contrib / datafusion-java

shauryashaurya / learn-data-munging

splitgraph / seafowl-gcsfuse

metrico / iox-community

treebee / elixir-arrow

baggiponte / awesome-pandas-alternatives

wheretrue / exon

datafusion-contrib / datafusion-c

fmenat / MultiviewCropClassification

blaze-init / spark-blaze-extension

ModelarData / ModelarDB-RS

grouzen / zio-apache-arrow

roeap / flight-fusion

sal-openlab / datafusion-server

splitgraph / experimental-datafusion-webassembly

Caoxuheng / HyMS

jychen7 / datafusion-bigtable

matsadler / bishop

milenkovicm / adhesive

milenkovicm / torchfusion

hengfeiyang / how-query-engines-work-zh-CN

jfrazier-eth / file-fusion