text-chunking

There are 2 repositories under text-chunking topic.

isaacus-dev / semchunk
A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.
chunking nlp python splitting text text-chunking text-splitting semantic-chunking isaacus
Language:Python 282
lazyFrogLOL / llmdocparser
A package for parsing PDFs and analyzing their content using LLMs.
llm nlp ocr pdfparser rag chunking document-analysis pdf-parser text-chunking
Language:Python 267
jparkerweb / semantic-chunking
🍱 semantic-chunking ⇢ semantically create chunks from large document for passing to LLM workflows
chunking embeddings llm semantic-chunking text-chunking text-splitter text-splitting vector
Language:JavaScript 80
drittich / SemanticSlicer
A recursive text chunker that attempts to break the text on meaningful boundaries.
ai azure-openai chat-gpt chatgpt chunker chunking embeddings gpt gpt-4 langchain llm openai text-chunking
Language:C# 18
ChenTaHung / HTML-Text-Parser
This project is designed to extract text from documents and prepare it for processing by Large Language Models (LLM). Implemented a feature to store and utilize text style information, enabling the program to identify and segment content based on potential headers and titles.
data-processing large-language-models llms python text-chunking text-parsing
Language:HTML 10
betcorg / llm-text-splitter
A lightweight TypeScript text splitter for RAG applications
chatbots rag text-chunking text-splitter
Language:TypeScript 1
philnash / chunkers
An exploration of text splitting and chunking in JavaScript
langchain-js llamaindex text-chunking text-splitter text-splitting
Language:TypeScript 1
ushakiranmai / text_summarization
This Text Summarization Tool uses advanced machine learning models to create concise, meaningful summaries of lengthy texts. Built with Hugging Face Transformers and Gradio, it efficiently handles various input lengths, ideal for summarizing articles, reports, and more
file-handling gradio-interface text-chunking text-summarization web-development model-handling output-formats python-libraries-and-tools
Language:Python 1
adityapathak-cubastion / cubastion-hr-chatbot
Presenting, Cubastion's HR chatbot - it can answer queries based on all the latest HR documents published by Cubastion's HR team. This conveniently saves time, allowing a Cubastion employee to resolve their query without having to comb through the actual documents. <<Developed with Python, sentence-transformers, Pinecone, llama3.2, and Streamlit>>
cosine-similarity huggingface llama3 pinecone prompt-engineering python sentence-transformers streamlit text-chunking text-embeddings text-extraction text-generation
Language:Python 0
adityapathakk / cubastion-hr-chatbot
Presenting, Cubastion's HR chatbot - it can answer queries based on all the latest HR documents published by Cubastion's HR team. This conveniently saves time, allowing a Cubastion employee to resolve their query without having to comb through the actual documents. <<Developed with Python, sentence-transformers, Pinecone, llama3.2, and Streamlit>>
cosine-similarity huggingface llama3 pinecone prompt-engineering python sentence-transformers streamlit text-chunking text-embeddings text-extraction text-generation
Language:Python 0

text-chunking

isaacus-dev / semchunk

lazyFrogLOL / llmdocparser

jparkerweb / semantic-chunking

drittich / SemanticSlicer

ChenTaHung / HTML-Text-Parser

betcorg / llm-text-splitter

philnash / chunkers

ushakiranmai / text_summarization

adityapathak-cubastion / cubastion-hr-chatbot

adityapathakk / cubastion-hr-chatbot