entity-resolution

There are 49 repositories under entity-resolution topic.

dedupeio / dedupe
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dedupe record-linkage python python-library entity-resolution dedupe-library de-duplicating datamade clustering
Language:Python 4398
moj-analytical-services / splink
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
record-linkage spark em-algorithm deduplication deduplicate-data entity-resolution data-matching fuzzy-matching data-science duckdb uk-gov-data-science
Language:Python 1768
zingg
zinggAI / zingg
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
fuzzymatch fuzzy-matching deduplication dedupe masterdata dataengineering entity-resolution identity-resolution data-science spark ml dataquality analytics datalake master-data-management customer-data-platform databricks snowflake cdp mdm
Language:Java 1113
recordlinkage
J535D165 / recordlinkage
A powerful and modular toolkit for record linkage and duplicate detection in Python
data-matching dedupe deduplication entity-resolution machine-learning privacy python python-library record-linkage similarity string-distance utrecht-university
Language:Python 1027
JohnSnowLabs / nlu
1 line for thousands of State of The Art NLP models in hundreds of languages The fastest and most accurate way to solve text problems.
nlu natural-language-understanding sentiment-classifier text-classification transformers language-detection named-entity-recognition seq2seq t5 lemmatizer spell-checker sentence-embeddings sentiment-analysis bert-embedding text-summarization text-translation streamlit pandas dependency-parsing entity-resolution
Language:Python 942
Picovoice / rhino
On-device Speech-to-Intent engine powered by deep learning
natural-language-understanding voice-recognition nlu spoken-language-understanding voice-assistant voice-ui voice-user-interface speech-recognition voice-commands voice-control vui on-device slu entity-resolution intent-inference slot-filling voice-command voice-command-control
Language:Python 685
dedupeio / csvdedupe
:id: Command line tool for deduplicating CSV files
dedupe cli record-linkage entity-resolution csv-files
Language:Python 431
dedupeio / dedupe-examples
:id: Examples for using the dedupe library
dedupe record-linkage entity-resolution python
Language:Python 415
J535D165 / data-matching-software
A list of free data matching and record linkage software.
awesome awesome-list data-matching deduplication entity-resolution fuzzy-matching machine-learning open-source record-linkage software
394
izuna385 / Entity-Linking-Recent-Trends
Recent trends of Entity Linking, Disambiguation, and Representation.
bert entity-disambiguation entity-language-model entity-linking entity-representation entity-resolution natural-language-processing nlp
346
microsoft / vert-papers
This repository contains code and datasets related to entity/knowledge papers from the VERT (Versatile Entity Recognition & disambiguation Toolkit) project, by the Knowledge Computing group at Microsoft Research Asia (MSRA).
ner entity-extraction entity-linking entity-resolution nlp nlp-resources ml named-entity-recognition linkingpark unitrans bertel can-ner grn entity-disambiguation language-understanding xl-ner cross-lingual-ner
Language:Python 279
amazon-science / ReFinED
ReFinED is an efficient and accurate entity linking (EL) system.
entity-extraction entity-linking entity-resolution nlp pytorch
Language:Python 223
scify / JedAIToolkit
An open source, high scalability toolkit in Java for Entity Resolution.
blocking entity-matching entity-resolution scalability
Language:Java 221
maxharlow / csvmatch
🔎 Finds fuzzy matches between CSV files
csv data-matching entity-resolution fuzzy-matching record-linkage
Language:Python 190
DerwenAI / strwythura
Construct knowledge graphs from unstructured data sources, use graph algorithms for enhanced GraphRAG with a DSPy-based chat bot locally, and curate semantics for optimizing AI app outcomes within a specific domain.
knowledge-graph nlp textgraphs unstructured-data vector-database entity-linking entity-resolution machine-learning named-entity-recognition relation-extraction semantic-expansion graphrag ontology semantic-layer ontology-pipeline semantic-random-walk dspy
Language:Jupyter Notebook 179
zentity-io / zentity
Entity resolution for Elasticsearch.
elasticsearch elasticsearch-plugin entity-resolution gdpr identity-resolution name-matching address-matching entity-matching
Language:Java 163
vintasoftware / entity-embed
PyTorch library for transforming entities like companies, products, etc. into vectors to support scalable Record Linkage / Entity Resolution using Approximate Nearest Neighbors.
entity-resolution record-linkage representation-learning embeddings approximate-nearest-neighbors deduplication entity-matching data-matching deep-learning python pytorch
Language:Jupyter Notebook 157
linktransformer
dell-research-harvard / linktransformer
A convenient way to link, deduplicate, aggregate and cluster data(frames) in Python using deep learning
deep-learning entity-matching entity-resolution huggingface-transformers nlp python record-linkage sentence-transformers transformers
Language:Python 132
ropeladder / record-linkage-resources
Resources for tackling record linkage / deduplication / data matching problems
data-matching deduplication entity-resolution java javascript python record-linkage
125
codeforkjeff / conciliator
OpenRefine reconciliation services for VIAF, ORCID, and Open Library + framework for creating more.
reconciliation-service viaf openrefine orcid solr entity-resolution openlibrary
Language:Java 120
usc-isi-i2 / rltk
Record Linkage ToolKit (Find and link entities)
linkage similarity similarity-metric string-similarity record-linkage entity-resolution deduplication
Language:Python 109
OlivierBinette / Awesome-Entity-Resolution
List of entity resolution software and resources.
awesome awesome-list deduplication entity-resolution identity-resolution record-linkage
96
Wikidata / soweego
Link Wikidata items to large catalogs
data-matching entity-linking entity-resolution identifiers knowledge-graph record-linkage wikidata wikimedia
Language:Python 96
pyJedAI
AI-team-UoA / pyJedAI
An open-source library that leverages Python’s data science ecosystem to build powerful end-to-end Entity Resolution workflows.
data-disambigation data-matching deduplication duplicate-detection entity-matching entity-resolution fuzzy-matching link-discovery machine-learning python
Language:Python 84
fritshermans / deduplipy
Python package for deduplication/entity resolution using active learning
deduplication entity-resolution fuzzy-matching record-linkage
Language:Python 82
data61 / anonlink
Python implementation of anonymous linkage using cryptographic linkage keys
cryptography entity-resolution privacy-enhancing-technologies privacy-preserving record-linkage
Language:Python 68
Gaglia88 / sparker
SparkER: an Entity Resolution framework for Apache Spark
apache-spark scala entity-resolution spark entity resolution apache meta-blocking python python27 python3 python-library
Language:Scala 65
wbsg-uni-mannheim / MatchGPT
This repository contains code and extensive prompt examples to reproduce and extend the experiments in our papers "Using ChatGPT for Entity Matching" and "Entity Matching using Large Language Models".
entity-resolution product-matching
Language:Jupyter Notebook 63
Senzing / awesome
Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.
senzing entity-resolution entity resolution record-linkage entity-matching resource awesome data-matching de-duplicating dedupe deduplication fuzzy-matching fuzzymatch identity identity-resolution disambiguation entities entity-linking senzing-community
Language:Python 62
cleanzr / dblink
Distributed Bayesian Entity Resolution in Apache Spark
entity-resolution apache-spark bayesian-inference record-linkage distributed-machine-learning mcmc
Language:Scala 58
databricks-industry-solutions / auto-data-linkage
Low effort linking and easy de-duplication. Databricks ARC provides a simple, automated, lakehouse integrated entity resolution solution for intra and inter data linking.
entity-resolution
Language:Python 51
wcmc-its / ReCiter
ReCiter: an enterprise open source author disambiguation system for academic institutions
machine-learning-algorithms entity-resolution clustering reciter spring-boot scopus maven aws dynamodb s3 elasticbeanstalk awscodebuild awscodepipeline pubmed algorithm java
Language:Java 49
J535D165 / recordlinkage-annotator
A browser user interface for manual labeling of record pairs.
record-linkage entity-resolution data-matching deduplication machine-learning annotation-tool labeling-tool
Language:JavaScript 47
vaneseltine / nominally
A maximum-strength name parser for record linkage.
data-matching data-science deduplication entity-resolution human-name parser parsing record-linkage
Language:Python 39
snowman
HPI-Information-Systems / snowman
Welcome to Snowman App – a Data Matching Benchmark Platform.
matching entity-resolution kpis data-matching duplicate-detection benchmark snowman data-stewards
Language:TypeScript 38
wbsg-uni-mannheim / productbert-intermediate
This repository contains code and data download scripts for the paper "Intermediate Training of BERT for Product Matching" by Ralph Peeters, Christian Bizer and Goran Glavaš.
entity-resolution product-matching bert
Language:Python 38

Related Topics

artificial-intelligence nlp