MinishLab / semhash

Fast Semantic Text Deduplication & Filtering

https://minish.ai/packages/semhash

Repository from Github https://github.comMinishLab/semhash

MinishLab/semhash Issues

Established way of getting duplicate ids?
Closed 3 months ago6
Support for Inline Deduplication Method
Closed 4 months ago4
Bug: IndexError in SemHash When Processing Identical Duplicates or Unique Records
Closed 5 months ago4
Using an own index instead of `Semhash.from_records()` takes forever
Closed 7 months ago3
Memory leakage when create semhash from a 1M data
Closed 7 months ago
Saving/Loading whole Semhash object
Closed 8 months ago1
Two-dimensional deduplication
Closed 8 months ago4
Support on GPU parallel computing
Closed 8 months ago2
Deduplication on hugginface datasets
Closed 8 months ago3
After "training", running inference on one datapoint
Closed 8 months ago1
Incompatibility with NumPy 2.x
Closed 9 months ago8
Retain reference to removed exact duplicates
Closed 8 months ago4
deduplication with expand columns
Closed 9 months ago4
Usearch dependency
Closed 9 months ago4
Usearch cannot be found
Closed 9 months ago3
Option to keep exact duplicates
Closed 9 months ago3
Couple of samples in readme
Closed 10 months ago10
How to get the idx of the deduplicated texts
Closed 10 months ago3
pandas dataframe
Closed 10 months ago5
[ENHANCEMENT] Get index back after deduplication
Closed 10 months ago2
FE
Closed 10 months ago