record-linkage

There are 25 repositories under record-linkage topic.

dedupeio / dedupe
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
dedupe record-linkage python python-library entity-resolution dedupe-library de-duplicating datamade clustering
Language:Python 4025
openvenues / libpostal
A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.
address address-parser c deduping deduplication international machine-learning natural-language-processing nlp record-linkage
Language:C 3991
moj-analytical-services / splink
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
record-linkage spark em-algorithm deduplication deduplicate-data entity-resolution data-matching fuzzy-matching data-science duckdb uk-gov-data-science
Language:Python 1154
recordlinkage
J535D165 / recordlinkage
A powerful and modular toolkit for record linkage and duplicate detection in Python
data-matching dedupe deduplication entity-resolution machine-learning privacy python python-library record-linkage similarity string-distance utrecht-university
Language:Python 925
Yomguithereal / talisman
Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.
natural-language-processing machine-learning fuzzy-matching clustering record-linkage information-retrieval deduplication
Language:JavaScript 701
dedupeio / csvdedupe
:id: Command line tool for deduplicating CSV files
cli csv-files dedupe entity-resolution record-linkage
Language:Python 406
dedupeio / dedupe-examples
:id: Examples for using the dedupe library
dedupe entity-resolution python record-linkage
Language:Python 397
J535D165 / data-matching-software
A list of free data matching and record linkage software.
awesome awesome-list data-matching deduplication entity-resolution fuzzy-matching machine-learning open-source record-linkage software
354
maxharlow / csvmatch
🔎 Finds fuzzy matches between CSV files
csv data-matching entity-resolution fuzzy-matching record-linkage
Language:Python 176
vintasoftware / entity-embed
PyTorch library for transforming entities like companies, products, etc. into vectors to support scalable Record Linkage / Entity Resolution using Approximate Nearest Neighbors.
entity-resolution record-linkage representation-learning embeddings approximate-nearest-neighbors deduplication entity-matching data-matching deep-learning python pytorch
Language:Jupyter Notebook 139
zouzias / spark-lucenerdd
Spark RDD with Lucene's query and entity linkage capabilities
spark lucene rdd spatial-search record-linkage deduplication entity-linking linkage hacktoberfest
Language:Scala 126
dice-group / LIMES
Link Discovery Framework for Metric Spaces.
artificial-intelligence linked-data machine-learning optimization rdf record-linkage scalability semantic-web
Language:JavaScript 125
ropeladder / record-linkage-resources
Resources for tackling record linkage / deduplication / data matching problems
data-matching deduplication entity-resolution java javascript python record-linkage
107
usc-isi-i2 / rltk
Record Linkage ToolKit (Find and link entities)
deduplication entity-resolution linkage record-linkage similarity similarity-metric string-similarity
Language:Python 104
Wikidata / soweego
Link Wikidata items to large catalogs
wikimedia wikidata knowledge-graph record-linkage data-matching entity-resolution identifiers entity-linking
Language:Python 95
linktransformer
dell-research-harvard / linktransformer
A convenient way to link, deduplicate, aggregate and cluster data(frames) in Python using deep learning
deep-learning entity-matching entity-resolution huggingface-transformers nlp python record-linkage sentence-transformers transformers
Language:Python 87
fritshermans / deduplipy
Python package for deduplication/entity resolution using active learning
deduplication entity-resolution fuzzy-matching record-linkage
Language:Python 75
data61 / anonlink
Python implementation of anonymous linkage using cryptographic linkage keys
privacy-enhancing-technologies privacy-preserving record-linkage entity-resolution cryptography
Language:Python 61
cleanzr / dblink
Distributed Bayesian Entity Resolution in Apache Spark
entity-resolution apache-spark bayesian-inference record-linkage distributed-machine-learning mcmc
Language:Scala 57
Senzing / awesome
Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.
senzing entity-resolution entity resolution record-linkage entity-matching resource awesome data-matching de-duplicating dedupe deduplication fuzzy-matching fuzzymatch identity identity-resolution disambiguation entities entity-linking senzing-community
Language:Python 49
data61 / clkhash
CLK hash: hash pii for entity matching
record-linkage privacy-enhancing-technologies hashing entity-matching privacy-preserving-record-linkage
Language:Python 48
dssg / pgdedupe
A simple command line interface to the datamade/dedupe library.
data-cleaning database dedupe deduplication postgresql python record-linkage
Language:Jupyter Notebook 42
J535D165 / recordlinkage-annotator
A browser user interface for manual labeling of record pairs.
annotation-tool data-matching deduplication entity-resolution labeling-tool machine-learning record-linkage
Language:JavaScript 41
entrepreneur-interet-general / Merge-Machine
Merge Dirty Data with Clean Reference Tables
elasticsearch entity-resolution entrepreneur-interet-general python record-linkage
Language:Python 35
iesl / stance
Learned string similarity for entity names using optimal transport.
string-similarity optimal-transport stance aliases entity-resolution string-distance string-matching record-linkage
Language:Python 34
ing-bank / spark-matcher
Record matching and entity resolution at scale in Spark
deduplication entity-resolution record-linkage spark
Language:Python 29
k3jph / phonics-in-r
Phonetic Spelling Algorithms in R
phonetic-spelling-algorithms soundex phonics nysiis metaphone text-processing linguistics record-linkage bsd-2-license
Language:R 28
vaneseltine / nominally
A maximum-strength name parser for record linkage.
data-matching data-science deduplication entity-resolution human-name parser parsing record-linkage
Language:Python 27
data61 / anonlink-entity-service
Privacy Preserving Record Linkage Service
privacy-preserving-record-linkage record-linkage privacy-enhancing-technologies
Language:Python 26
J535D165 / FEBRL-fork-v0.4.2
Fork of the Freely Extensible Biomedical Record Linkage program
deduplication entity-resolution matching python-library record-linkage
Language:Python 23
Valires / er-evaluation
An End-to-End Evaluation Framework for Entity Resolution Systems
data-science deduplication disambiguation duplicate-detection entity-resolution evaluation fuzzy-matching matching ml-evaluation ml-testing record-linkage statistics inventor-name-disambiguation author-name-disambiguation
Language:Python 22
Valires / Awesome-Entity-Resolution
List of entity resolution software and resources.
awesome awesome-list deduplication entity-resolution identity-resolution record-linkage
20
data61 / blocklib
Python implementations of record linkage blocking techniques.
privacy-preserving-record-linkage privacy-enhancing-technologies record-linkage
Language:Python 19
gpoulter / pydedupe
(Archived) A Python library for record linkage and deduplication.
deduplication record-linkage
Language:Python 19
ngmarchant / comparator
Similarity and distance measures for clustering and record linkage applications in R
r-package distance-measures similarity-measures string-similarity record-linkage distance-metrics entity-resolution clustering
Language:R 16
zouzias / spark-lucenerdd-examples
Examples of spark-lucenerdd
spark lucene lucene-query docker examples record-linkage entity-linking
Language:Scala 15