There are 4 repositories under apache-arrow topic.
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..
❄️ Coolest database around 🧊 Embeddable column database written in Go.
Manipulate JSON-like data with NumPy-like idioms.
Loaders for big data visualization. Website:
A Python library for fast, interactive geospatial vector data visualization in Jupyter.
Multi-Modal Database replacing MongoDB, Neo4J, and Elastic with 1 faster ACID solution, with NetworkX and Pandas interfaces, and bindings for C 99, C++ 17, Python 3, Java, GoLang 🗄️
Rust-based WebAssembly bindings to read and write Apache Parquet data
Infrastructures™ for Machine Learning Training/Inference in Production.
A Rust DataFrame implementation, built on Apache Arrow
A SQLite vtable extension to read Parquet files
GeoArrow in Rust, Python, and JavaScript (WebAssembly) with vectorized geometry operations
Fletcher: A framework to integrate FPGA accelerators with Apache Arrow
Manipulate arrays of complex data structures as easily as Numpy.
ParquetSharp is a .NET library for reading and writing Apache Parquet files.
Pod5: a high performance file format for nanopore reads.
Zero-copy reading of Arrow data from WebAssembly
Cross-filter millions (or even billions) of data entries with no interaction delay
OSM planet dump high performance data loader. Transform OpenStreetMap World/Region PBF dump into partitioned by H3 regions PostGIS pgsnapshot (lossless) OSM schema representation and/or into ArrowIPC/Parquet dumps
MongoDB integrations for Apache Arrow. Export MongoDB documents to numpy array, parquet files, and pandas dataframes in one line of code.
deck.gl layers for rendering GeoArrow data
A minimal Python library for Apache Arrow, connecting to the Rust arrow crate
A dataframe library for Rubyists.
A lightweight Logging and Tracing observability solution for Rust, built with Apache Arrow, Apache Parquet and Apache DataFusion.
Apache Arrow ADBC bindings for Elixir
A Spark Connector that reads data from / writes data to Arrow-Flight end-points with Arrow-Flight and Flight-SQL