segment-any-text / wtpsplit

Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.

segment-any-text/wtpsplit Issues

Questions concerning configuring train_lora.py for custom corpus
Updated 7 days ago8
Which setting is the best for scientific sentence segmentation with inline citations and potential parsing error
Updated 7 days ago1
ImportError: cannot import name 'SaT' from 'wtpsplit' (/usr/local/lib/python3.10/site-packages/wtpsplit/__init__.py)
Closed 7 days ago4
AttributeError: 'SubwordXLMForTokenClassification' object has no attribute 'split'
Closed 13 days ago2
Newlines are treated like spaces
Closed 20 days ago5
Update README for LoRA
Closed 20 days ago
ImportError with language and style_or_domain arguments and huggingface-hub 0.26
Closed 20 days ago2
Korean text is not split well
Closed 20 days ago4
KeyError: 'xlm-token'
Closed 21 days ago1
no-limited-lookahead models
Closed a month ago1
Canine model and High VRAM usage
Closed 2 months ago7
InconsistentVersionWarning Issue everytime I start wtp
Closed 2 months ago2
wtp-canine-s-1l-no-adapters - missing mixtures.skops
Closed 2 months ago1
Add option to limit the maximum length of each split text segment
Updated 2 months ago1
Single words incorrectly segmented into character sequences
Closed 2 months ago4
Failed to Adapt to your own corpus via LoRA
Closed 2 months ago8
Update transformers dependency
Closed 3 months ago3
sat-12l-sm running on GPU
Closed 4 months ago6
Error when installing the requirements
Closed 4 months ago2
CUDA device error when segmenting Greek text
Closed 4 months ago1
SaT is slow
Closed 5 months ago2
Run models for Italian
Closed 5 months ago2
Inconsistent results with same sentences
Closed 5 months ago5
Model(s) use word capitlisation to segment
Closed 5 months ago11
Please explain length of output of wtp.predict_proba(text)
Closed 9 months ago3
Huggingface AutoModelForTokenClassification bug
Closed 9 months ago4
Accuracy: Error in Split (EN)
Closed 10 months ago1
remove_repetition
Closed a year ago2
Recursion in init
Closed a year ago2
How to use Universal Dependencies style ?
Closed a year ago3
model in huggingface cannot load mixtures.skops
Closed a year ago1
Rust bindings for wtpsplit
Updated a year ago
Could not find a mixture for the Universal Dependencies (UD) style in Thai language
Closed a year ago1
For GPU, ONNX WtP model is around 2x slower than PyTorch.
Closed a year ago2
Scoring metric, does definition make sense?
Closed a year ago1
Any string that isn't a multiple of 4 causes an assert failure
Closed a year ago2
Async - Skops import is failing
Closed a year ago7
Opus100 FR not in mixtures
Closed a year ago1
Error loading model to GPU
Closed a year ago1
[Bug] Pandas not listed in `install_requires`, resulting in import error
Closed a year ago1
Unusual splits in short sentence
Closed a year ago3
ValueError when using wtp-canine-s-12l-no-adapters on Danish
Closed a year ago4
get_threshold does not work
Closed a year ago1
Apple Silicon / Arm support
Closed a year ago3
ImportError: /lib64/libm.so.6: version `GLIBC_2.27' not found
Closed a year ago2
Control where the model is downloaded too?
Closed a year ago4
`AttributeError: 'InferenceSession' object has no attribute '_providers' Segmentation fault (core dumped)`
Closed a year ago1
Hi, pip install nnsplit doesn't work
Closed 2 years ago3
Python 3.10 wheel
Closed 2 years ago6
Unable to use own trained onnx models
Closed 3 years ago2