preprocessing

There are 3 repositories under preprocessing topic.

Unstructured-IO / unstructured
Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.
deep-learning document-parsing machine-learning nlp ocr information-retrieval data-pipelines ml preprocessing pdf-to-text natural-language-processing pdf pdf-to-json document-image-analysis donut document-image-processing document-parser docx langchain llm
Language:HTML 12660
dongrixinyu / JioNLP
中文 NLP 预处理、解析工具包，准确、高效、易用 A Chinese NLP Preprocessing & Parsing Package www.jionlp.com
apache2 chinese natural-language-processing ner nlp nlp-parse preprocessing python time-parse time-parsing
Language:Python 3567
igel
nidhaloff / igel
a delightful machine learning tool that allows you to train, test, and use models without writing code
machine-learning machinelearning machine-learning-algorithms machine-learning-library artificial-intelligence neural-network neural-networks sklearn scikit-learn scikitlearn-machine-learning data-science data-analysis preprocessing automation automl automl-experiments hacktoberfest hacktoberfest2021
Language:Python 3128
OpenGene / fastp
An ultra-fast all-in-one FASTQ preprocessor (QC/adapters/trimming/filtering/splitting/merging...)
adapter bioinformatics duplication fastq filter filtering illumina merging ngs overlap polyg preprocessing qc quality quality-control sequencing splitting trimming umi
Language:C++ 2171
MLBox
AxeldeRomblay / MLBox
MLBox is a powerful Automated Machine Learning python library.
machine-learning auto-ml kaggle deep-learning stacking pipeline optimization preprocessing encoding prediction distributed xgboost drift classification regression lightgbm keras automated-machine-learning automl data-science
Language:Python 1515
AutoTS
winedarksea / AutoTS
Automated Time Series Forecasting
automl autots deep-learning feature-engineering forecasting machine-learning preprocessing time-series
Language:Python 1263
PyHealth
sunlabuiuc / PyHealth
A Deep Learning Python Toolkit for Healthcare Applications.
clinical-data clinical-research data-mining deep-learning electronic-health-record electronic-medical-record healthcare medical-code preprocessing
Language:Python 1090
NVIDIA-Merlin / NVTabular
NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale datasets used to train deep learning based recommender systems.
deep-learning feature-engineering feature-selection gpu machine-learning nvidia preprocessing recommendation-system recommender-system
Language:Python 1078
nnAudio
KinWaiCheuk / nnAudio
Audio processing by using pytorch 1D convolution network
spectrogram-conversion-toolbox pytorch audio-processing preprocessing stft melspectrogram spectrogram cqt-spectrogram neural-network 1d-convolution
Language:Python 1058
TheAlgorithms / R
Collection of various algorithms implemented in R.
algorithms r education machine-learning practice learning preprocessing regression data-mining clustering datamanipulation classification algorithm r-programming r-language hacktoberfest
Language:R 1008
pytorch / torcharrow
High performance model preprocessing library on PyTorch
preprocessing python pytorch
Language:Python 649
MinishLab / semhash
Fast Semantic Text Deduplication
datasets deduplication model2vec preprocessing semantic-deduplication vicinity
Language:Python 605
contextualSpellCheck
R1j1t / contextualSpellCheck
✔️Contextual word checker for better suggestions (not actively maintained)
spacy spacy-extension nlp spellcheck preprocessing bert help-wanted chatbot spellchecker python-spelling-corrector oov spelling-correction spelling-corrections natural-language-processing python
Language:Python 412
qd-cae / awesome-CAE
A curated list of awesome CAE frameworks, libraries and software.
abaqus cae cfd collection fem libraries ls-dyna preprocessing scripting tools
405
msamogh / nonechucks
Deal with bad samples in your dataset dynamically, use Transforms as Filters, and more!
pytorch data-processing data-preprocessing data-pipeline data-cleaning preprocessing machine-learning torch
Language:Python 379
MaxHalford / xam
:dart: Personal data science and machine learning toolbox
data-science machine-learning preprocessing python stacking
Language:Python 365
DataCanvasIO / HyperGBM
A full pipeline AutoML tool for tabular data
automl gbm fullpipeline xgboost lightgbm catboost pseudo-labeling adversarial-validation semi-supervised-learning datacleaning preprocessing ensemble-learning tabular-data distributed-training dask gpu-acceleration dask-distributed rapidsai sklearn
Language:Python 357
ikegami-yukino / jaconv
Pure-Python Japanese character interconverter for Hiragana, Katakana, Hankaku, and Zenkaku
japanese-language preprocessing text-processing japanese-kana pure-python character-converter julius transliteration
Language:Python 325
advaitsave / Introduction-to-Time-Series-forecasting-Python
Introduction to time series preprocessing and forecasting in Python using AR, MA, ARMA, ARIMA, SARIMA and Prophet model with forecast evaluation.
arima arma dickey-fuller forecast-evaluation forecasting preprocessing prophet-model python sarima seasonality series-forecasting-python series-preprocessing stationarity time-series time-series-forecasting
Language:Jupyter Notebook 323
cylondata / cylon
Cylon is a fast, scalable, distributed memory, parallel runtime with a Pandas like DataFrame.
data dataframe dataframes-api deep-learning join mpi preprocessing shuffle
Language:C++ 300
nlpcl-lab / ace2005-preprocessing
ACE 2005 corpus preprocessing for Event Extraction task
ace2005 preprocessing event-extraction parsed-data
Language:Python 296
ikegami-yukino / neologdn
Japanese text normalizer for mecab-neologd
japanese-language mecab-ipadic-neologd nlp preprocessing text-normalization
Language:Cython 279
Razor12911 / xtool
Just some tool repackers like to use...
archiving compression encryption precompression preprocessing repacking
Language:Pascal 270
Deffro / text-preprocessing-techniques
16 Text Preprocessing Techniques in Python for Twitter Sentiment Analysis.
data-science preprocessing python sentiment-analysis twitter
Language:Python 224
dunky11 / voicesmith
[WIP] VoiceSmith makes training text to speech models easy.
dataset-manager preprocessing speech-synthesis text-to-speech tts voice-cloning delightfultts univnet toolkit
Language:Python 224
quqixun / BrainPrep
Preprocessing pipeline on Brain MR Images through FSL and ANTs, including registration, skull-stripping, bias field correction, enhancement and segmentation.
brain mri preprocessing fsl registration skull-stripping brain-segmentation ants enhancement bias-field-correction
Language:Python 203
basf / mamba-tabular
Mambular is a Python package that simplifies tabular deep learning by providing a suite of models for regression, classification, and distributional regression tasks. It includes models such as Mambular, TabM, FT-Transformer, TabulaRNN, TabTransformer, and tabular ResNets.
classification deeplearning distributional-regression ft-transformer mamba node piecewise-linear-encoding ple preprocessing regression resnet rnn saint ssm state-space-models tabm tabtransformer tabular tabular-deep-learning tabular-learning
Language:Python 202
xMIP
jbusecke / xMIP
Analysis ready CMIP6 data in python the easy way with pangeo tools.
analysis-ready-data climate-analysis climate-models cmip6 cmip6-data pangeo preprocessing xgcm
Language:Jupyter Notebook 199
google / tensorflow-recorder
TFRecorder makes it easy to create TensorFlow records (TFRecords) from Pandas DataFrames and CSVs files containing images or structured data.
tensorflow-recorder tfrecords tensorflow preprocessing image-processing apache-beam
Language:Python 182
jaeho3690 / LIDC-IDRI-Preprocessing
This is the preprocessing step of the LIDC-IDRI dataset
lidc-dataset lidc-idri lidc-preprocessing lung npy preprocessing
Language:Jupyter Notebook 174
ropensci / MODIStsp
An "R" package for automatic download and preprocessing of MODIS Land Products Time Series
modis gdal r preprocessing time-series remote-sensing satellite-imagery modis-data modis-land-products peer-reviewed r-package rstats
Language:R 155
githubharald / DeslantImg
The deslanting algorithm sets text upright in images. Python, C++ and OpenCL implementations provided.
c-plus-plus gpu handwriting-recognition image-processing ocr opencl opencv preprocessing python
Language:C++ 150
sappelhoff / pyprep
PyPREP: A Python implementation of the Preprocessing Pipeline (PREP) for EEG data
eeg preprocessing python neuroscience neuroimaging artifact data mne electroencephalogaphy
Language:Python 148
autoreject / autoreject
Automated rejection and repair of bad trials/sensors in M/EEG
electroencephalography preprocessing cross-validation mne-python magnetoencephalography meg
Language:Python 142
mlr3pipelines
mlr-org / mlr3pipelines
Dataflow Programming for Machine Learning in R
mlr3 machine-learning pipelines preprocessing bagging stacking ensemble-learning r dataflow-programming r-package data-science
Language:R 142
chakki-works / chariot
Deliver the ready-to-train data to your NLP model.
natural-language-processing python preprocessing tensorflow keras
Language:Jupyter Notebook 122