data

There are 1,877 repositories under data topic.

firecrawl / firecrawl
The Web Data API for AI - Turn entire websites into LLM-ready markdown or structured data 🔥
ai crawler data markdown scraper html-to-markdown llm rag scraping web-crawler ai-scraping webscraping
Language:TypeScript 57404
query
TanStack / query
🤖 Powerful asynchronous state management, server-state utilities and data fetching for the web. TS/JS, React Query, Solid Query, Svelte Query and Vue Query.
react hooks fetch cache async rest graphql data query stale stale-while-revalidate update solid svelte typescript vue
Language:TypeScript 46636
run-llama / llama_index
LlamaIndex is the leading framework for building LLM-powered agents over your data.
agents application data fine-tuning framework llamaindex llm rag vector-database multi-agents
Language:Python 44256
metabase / metabase
The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
analytics businessintelligence dashboard reporting slack clojure database metabase postgres postgresql mysql bi visualization data data-analysis sql-editor data-visualization business-intelligence
Language:Clojure 43706
DataExpert-io / data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
apachespark awesome bigdata data dataengineering sql
Language:Jupyter Notebook 37743
sheetjs
SheetJS / sheetjs
📗 SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs
xlsx excel spreadsheet data xml csv database nodejs html5 javascript ios json react vue angular table html grid deno bun
35915
swr
vercel / swr
React Hooks for Data Fetching
cache data data-fetching fetch hook hooks nextjs react react-native stale-while-revalidate suspense swr vercel
Language:TypeScript 31918
sinaptik-ai / pandas-ai
Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.
llm pandas ai data-analysis data-science gpt-4 csv data sql database datalake data-visualization text-to-sql
Language:Python 22068
PrefectHQ / prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
python workflow data-engineering data-science workflow-engine prefect infrastructure ml-ops data-ops automation orchestration data observability pipeline
Language:Python 20355
airbytehq / airbyte
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
data pipeline data-analysis data-engineering java python etl change-data-capture data-collection data-integration elt bigquery redshift snowflake data-pipeline mssql mysql postgresql s3 self-hosted
Language:Python 19513
fivethirtyeight / data
Data and code behind the articles and graphics at FiveThirtyEight
data
Language:Jupyter Notebook 17187
presto
prestodb / presto
The official home of the Presto distributed SQL query engine for big data
java presto hive hadoop big-data sql data lakehouse query
Language:Java 16498
faker-js / faker
Generate massive amounts of fake data in the browser and node.js
faker data fake nodejs browser javascript typescript
Language:TypeScript 14313
oxnr / awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
awesome awesome-list bigdata data data-analytics data-science data-stream data-visualization data-warehouse database distributed-database series-database stream-processing streaming-data visualize-data
13871
akshare
akfamily / akshare
AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库
futures financial-data data-science quant fundamental akshare option stock data datasets bond asset-pricing academic currency finance finance-api economic-data economics data-analysis
Language:Python 13452
pwxcoo / chinese-xinhua
:orange_book: 中华新华字典数据库。包括歇后语，成语，词语，汉字。
data scraper chinese-traditional python3 chinese chinese-characters chinese-nlp chinese-language chinese-simplified json-dataset json json-data
Language:Python 11329
apple / pkl
A configuration as code language with rich validation and tooling.
pkl java kotlin config configuration data functional json language object-oriented programming-language properties propertylist validation xml yaml
Language:Java 10852
prql
PRQL / prql
PRQL is a modern language for transforming data — a simple, powerful, pipelined SQL replacement
data pipeline sql
Language:Rust 10469
Bogus
bchavez / Bogus
:card_index: A simple fake data generator for C#, F#, and VB.NET. Based on and ported from the famed faker.js.
fake data generator dotnet csharp faker c-sharp bogus poco data-access-layer database seed test-data data-generator
Language:C# 9432
rawgraphs-app
rawgraphs / rawgraphs-app
A web interface to create custom vector-based visualizations on top of RAWGraphs core
svg vector-graphics data-visualization rawgraphs d3js data ddj design visualization
Language:JavaScript 8859
mage-ai / mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
machine-learning artificial-intelligence data data-engineering data-science python elt etl pipelines data-pipelines orchestration data-integration sql spark dbt pipeline reverse-etl transformation
Language:Python 8464
machine-learning-roadmap
mrdbourke / machine-learning-roadmap
A roadmap connecting many of the most important concepts in machine learning, how to learn them and what tools to use to perform them.
machine-learning deep-learning data data-science
7734
olifolkerd / tabulator
Interactive Tables and Data Grids for JavaScript
tabulator javascript table grid grid-layout grid-system data widget cdnjs json ajax list sort jquery react tabulator-table
Language:JavaScript 7308
Scrapling
D4Vinci / Scrapling
🕷️ An undetectable, powerful, flexible, high-performance Python library to make Web Scraping Easy and Effortless as it should be!
crawler crawling crawling-python playwright python scraping selectors stealth web-scraper web-scraping web-scraping-python webscraping xpath automation ai ai-scraping data data-extraction mcp mcp-server
Language:Python 7298
snowplow
snowplow / snowplow
The leader in Customer Data Infrastructure
snowplow analytics data data-pipeline data-collection product-analytics marketing-analytics snowplow-pipeline snowplow-events
Language:Scala 6956
flyte
flyteorg / flyte
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
flyte machine-learning golang scale workflow data-science data-analysis data kubernetes-operator kubernetes orchestration-engine mlops dataops grpc python production declarative fine-tuning llm hacktoberfest
Language:Go 6498
dformoso / machine-learning-mindmap
A mindmap summarising Machine Learning concepts, from Data Analysis to Deep Learning.
cheatsheet data jupyter learning machine machine-learning mindmap python science
6223
cloudquery
cloudquery / cloudquery
The open source ELT framework powered by Apache Arrow
aws gcp azure sql data-integration elt etl etl-framework bigquery data-collection data-engineering kubernetes data airbyte github-api data-analysis google go cspm attack-surface-management
Language:Go 6205
Parsr
axa-group / Parsr
Transforms PDF, Documents and Images into Enriched Structured Data
parsr pdf images extraction data typescript python ocr document nlp hacktoberfest
Language:JavaScript 6008
Countly / countly-server
Countly is a product analytics platform that helps teams track, analyze and act-on their user actions and behaviour on mobile, web and desktop applications.
analytics coppa crash-analytics crash-reports dashboard data data-ownership data-privacy feature-flags gdpr hipaa insights mobile-analytics product-analytics product-management push-notifications remote-configuration tracking user-feedback web-analytics
Language:JavaScript 5758
cue-lang / cue
The home of the CUE language! Validate and define text-based and dynamic configuration
data validation configuration kubernetes
Language:Go 5664
airbnb / knowledge-repo
A next-generation curated knowledge sharing platform for data scientists and other technical professions.
data data-science knowledge data-analysis
Language:Python 5527
browser-compat-data
mdn / browser-compat-data
Browser compatibility data for Web technologies as displayed on MDN
compat compatibility data dataset json
Language:JSON 5396
superduper
superduper-io / superduper
Superduper: End-to-end framework for building custom AI applications and agents.
ai mlops torch transformers mongodb python pytorch ml database data inference distributed-ml llm-inference pretrained-models chatbot semantic-search llm-serving llmops vector-search rag
Language:Python 5208
modelscope / data-juicer
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
data-analysis data-science large-language-models llm data-visualization llms instruction-tuning pre-training multi-modal synthetic-data data data-pipeline data-processing foundation-models
Language:Python 5193
brianvoe / gofakeit
Random fake data generator written in go
go data generator random fake golang seed
Language:Go 5159

Related Topics