data-centric-ai

There are 12 repositories under data-centric-ai topic.

cleanlab / cleanlab
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
active-learning annotation data-centric-ai data-cleaning data-curation data-labeling data-profiling data-quality data-science data-validation dataops dataquality datasets exploratory-data-analysis labeling llms noisy-labels out-of-distribution-detection outlier-detection weak-supervision
Language:Python 11072
fiftyone
voxel51 / fiftyone
Refine high-quality datasets and visual AI models
active-learning artificial-intelligence computer-vision data-centric-ai data-cleaning data-curation data-quality data-science deep-learning developer-tools image-classification machine-learning object-detection python unstructured-data vector-search visualization
Language:Python 10009
docta
Docta-ai / docta
A Doctor for your data
data data-centric-ai data-centric-machine-learning data-curation data-diagnosis language-model rlhf
Language:Python 3497
refinery
code-kern-ai / refinery
The data scientist's open-source choice to scale, assess and maintain natural language data. Treat training data like a software artifact.
annotations data-centric-ai data-labeling deep-learning labeling labeling-tool machine-learning natural-language-processing neural-search nlp text-annotation transformers python human-in-the-loop spacy artificial-intelligence data-science text-classification active-learning supervised-learning
Language:Python 1459
Renumics / spotlight
Interactively explore unstructured datasets from your dataframe.
audio computer-vision data-centric-ai data-curation data-visualization exploratory-data-analysis hacktoberfest images machine-learning meshes timeseries unstructured-data video
Language:TypeScript 1202
data-centric-AI
daochenzha / data-centric-AI
A curated, but incomplete, list of data-centric AI resources.
artificial-intelligence data-centric-ai ai machine-learning data-curation data-centric data-centric-machine-learning data-science data-quality data-engineering
1135
cleanlab / cleanvision
Automatically find issues in image datasets and practice data-centric computer vision.
computer-vision data-centric-ai data-exploration data-quality data-validation deep-learning exploratory-data-analysis image-analysis image-classification image-generation image-quality image-segmentation data-profiling data-science
Language:Python 1130
HazyResearch / data-centric-ai
Resources for Data Centric AI
ai artificial-intelligence data-centric-ai machine-learning
Language:TeX 1129
awesome-open-data-centric-ai
Renumics / awesome-open-data-centric-ai
Curated list of open source tooling for data-centric AI on unstructured data.
awesome-list data-centric-ai data-curation data-versioning data-visualization explainable-ai active-learning feature-vector robust-machine-learning bias-detection computer-vision data-drift deep-learning nlp noisy-labels outlier-detection synthetic-data uncertainty-estimation machine-learning documentation-only
733
dcai-course / dcai-lab
Lab assignments for Introduction to Data-Centric AI, MIT IAP 2024 👩🏽‍💻
course data-centric-ai data-science deep-learning homework lab machine-learning
Language:Jupyter Notebook 476
gszfwsb / NCFM
Official PyTorch implementation of the paper "Dataset Distillation with Neural Characteristic Function: A Minmax Perspective" (NCFM) in CVPR 2025 (Highlight).
computer-vision data-centric-ai dataset-distillation synthetic-data
Language:Python 389
GAIR-NLP / ProX
[ICML 2025] Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
continual continual-pre-training data-centric-ai data-quality llama llm mistral neural-symbolic pre-training
Language:Python 263
JieyuZ2 / wrench
[NeurIPS 2021] WRENCH: Weak supeRvision bENCHmark
benchmark-framework data-centric-ai data-programming dataset deep-learning machine-learning nlp robust-learning sequence-labeling weak-supervision weakly-supervised-learning
Language:Python 224
yueyu1030 / AttrPrompt
[NeurIPS 2023] This is the code for the paper `Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias`.
attributed-text data-centric-ai large-language-models natural-language-processing pretrained-language-model text-classification training-data-generation zero-shot-learning
Language:Python 155
aai-institute / pyDVL
pyDVL is a library of stable implementations of algorithms for data valuation and influence function computation
data-valuation shapley-value machine-learning transferlab least-core influence-functions game-theory data-quality robust-machine-learning data-centric-ai banzhaf-index data-cleaning data-pruning
Language:Python 139
dcai-course / dcai-course
Introduction to Data-Centric AI, MIT IAP 2024 🤖
course data-centric-ai data-science deep-learning machine-learning
Language:CSS 103
opendataval / opendataval
OpenDataVal: a Unified Benchmark for Data Valuation in Python (NeurIPS 2023)
data-valuation machine-learning python research statistics data-centric-ai data-cleaning game-theory
Language:Python 99
SJTU-DMTai / awesome-ml-data-quality-papers
Papers about training data quality management for ML models.
ai4db data-centric-ai data-debugging data-management data-profiling data-quality data-valuation db4ai machine-learning
97
OFA-Sys / DiverseEvol
Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
data-centric-ai instruction-tuning self-improving large-language-models nlp efficiency
Language:Python 85
nbsynthetic
NextBrain-ai / nbsynthetic
nbsynthetic is simple and robust tabular synthetic data generation library for small and medium size datasets
data-augmentation data-centric-ai generative-adversarial-network synthetic-data synthetic-dataset-generation gan
Language:Jupyter Notebook 68
TonyLianLong / UnsupervisedSelectiveLabeling
[ECCV 2022] Official Implementation for Unsupervised Selective Labeling for More Effective Semi-Supervised Learning
deep-learning semi-supervised-learning computer-vision data-centric-ai labeling-algorithm machine-learning
Language:Python 62
Acharya
astutic / Acharya
A Data Centric NER annotation tool for your Named Entity Recognition projects
ai annota annotation-processing annotation-tool data-centric data-centric-ai data-labeling data-labeling-tools data-quality data-quality-monitoring datasets machine-learning mlops named-entity-recognition natural-language-processing ner text-annotation text-annotation-tool
47
koalazf99 / Awesome-DataCentric-LLM
Trending projects & awesome papers about data-centric llm studies.
data-centric-ai evaluation llm pre-training
38
luo-junyu / Awesome-Data-Efficient-LLM
A list of data-efficient and data-centric LLM (Large Language Model) papers. Our Survey Paper: Towards Efficient LLM Post Training: A Data-centric Perspective
data-centric data-centric-ai data-centric-machine-learning data-efficient efficient large-language-models llm
38
Digital-Dermatology / SelfClean
🧼🔎 A holistic self-supervised data cleaning strategy to detect irrelevant samples, near duplicates and label errors (NeurIPS'24).
data-centric-ai data-cleaning self-supervised-learning deep-learning machine-learning data-curation neurips-2024
Language:Python 36
KibromBerihu / ai4elife
This data-centric AI repository implements a robust deep learning method (LFBNet) for fully automated tumor segmentation in whole-body [18]F-FDG PET/CT images.
ai automated-pet-segmentation biomarkers data-analysis data-centric-ai deep-learning fdg-pet image-segmentation lymphoma medical-imaging pet pet-ct-segmentation pet-segmentation survival-analysis whole-body-segmentation
Language:Python 36
awesome-mlops / awesome-data-management
A curated list of awesome open source tools and commercial products to catalog, version, and manage data 🚀
awesome awesome-list data-management dataops dataops-platform dataops-principles data-catalog data-centric-ai data-lifecycle data-management-platform data-manager data-pipeline data-science data-tools data-workflows datascience
34
sail-sg / D-TRAK
Intriguing Properties of Data Attribution on Diffusion Models (ICLR 2024)
attribution diffusion-models influence-functions ddpm stable-diffusion data-attribution data-valuation data-centric-ai interpretability
Language:Jupyter Notebook 34
cleanlab / cleanlab-studio
Client interface to Cleanlab Studio
annotations automl computer-vision data-centric-ai data-labeling data-profiling data-quality data-science data-validation machine-learning natural-language-processing noisy-labels data-cleaning outlier-detection data-curation image-classification text-classification structured-data model-deployment llm
Language:Python 32
nachifur / LLPC
Frontiers in Neuroinformatics 2022: Local Label Point Correction for Edge Detection of Overlapping Cervical Cells
data-centric-ai dataset deep-learning edge-detection label-correction overlapping-cell point-correction segmantation segmentation
Language:Python 30
ear-team / bambird
Unsupervised classification to improve the quality of a bird song recording dataset. https://doi.org/10.1016/j.ecoinf.2022.101952
bioacoustics cluster-analysis data-centric-ai dataset ecoacoustics labeling labeling-tool labelling-tool machine-learning soundscape xeno-canto
Language:Python 27
voxel51 / reconstruction-error-ratios
Estimate dataset difficulty and detect label mistakes using reconstruction error ratios!
annotation clip-model computer-vision data-centric-ai machine-learning python umap
Language:Python 26
Lichang-Chen / AlpaGasus
A better Alpaca Model Trained with Less Data (only 9k instructions of the original set)
data-centric-ai filtering-data instruction-following large-language-models
Language:HTML 24
IS2AI / AnyFace
Input-Agnostic Face Detection
data-centric-ai face-detection facial-landmarks input-agnostic
Language:Jupyter Notebook 23
kennethleungty / Data-Centric-AI-Competition
Codes for a Top 5% finish in the Data-Centric AI Competition organized by Andrew Ng and DeepLearning.AI
deep-learning andrew-ng machine-learning data-science data-centric data-centric-ai ai
Language:Jupyter Notebook 23
autonlab / aqua
AQuA: A Benchmarking Tool for Label Quality Assessment, NeurIPS'23 D&B
data-centric-ai data-cleaning data-science label-errors machine-learning robust-machine-learning
Language:Jupyter Notebook 22