batch-processing

There are 14 repositories under batch-processing topic.

• NaiboWang / EasySpider

  A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

  code-freecrawlerguilaymanspiderparameterswebwwwinput-parametersfrontendhtmlbatch-processingbatch-scriptvisualvisualizationvisualprogrammingscraperdata-collectionrparobotics
  Language:JavaScript 36115

• argoproj / argo-workflows

  Workflow Engine for Kubernetes

  airflowargoargo-workflowsbatch-processingcloud-nativecncfdagdata-engineeringgitopshacktoberfestk8sknativekubernetesmachine-learningmlopspipelinesworkflowworkflow-engine
  Language:Go 15106

• pathwaycom / pathway

  Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

  batch-processingdata-analyticsdata-pipelinesdata-processingdataflowetletl-frameworkiot-analyticskafkamachine-learning-algorithmspathwaypythonreal-timeruststream-processingstreamingtime-series-analysis
  Language:Python 4325

• spring-projects / spring-batch

  Spring Batch is a framework for writing batch applications using Java and Spring

  batchbatch-processingjavaspring
  Language:Java 2733
• faast.js

  faastjs / faast.js

  Serverless batch computing made simple.

  typescriptjavascriptserverlessbatchbatch-processinglibraryfaas
  Language:TypeScript 1476

• bbottema / simple-java-mail

  Simple API, Complex Emails (Jakarta Mail smtp wrapper)

  javajava-mailoutlookbatch-processingsmimedkimdmarchigh-performancespring-supportconversioncli-supportbuilder-apiproxyconfigurableemailsmtpjakarta-mailjavamailoauth2smtp-client
  Language:Java 1230

• spring-cloud / spring-cloud-dataflow

  A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes

  microservices-architectureorchestrationstream-processingbatch-processingpredictive-analyticscloud-nativedatapipelines
  Language:Java 1115
• hazelcast-jet

  hazelcast / hazelcast-jet

  Distributed Stream and Batch Processing

  batch-processingbig-datacdcevent-processinghacktoberfestjavakafkalow-latencystream-processing
  Language:Java 1104
• PicView

  Ruben2776 / PicView

  Fast, free and customizable image viewer for Windows 10 and 11.

  base64-imagebatch-processingcomics-readercompress-imagesconvert-imagescsharpdesktop-appdotnetheifimage-effectsimage-filtersimage-galleryimage-viewerjpeg-xljxlpicture-viewerraw-imagewebpwindowswpf
  Language:C# 1063

• alessandrofrancesconi / gimp-plugin-bimp

  BIMP. Batch Image Manipulation Plugin for GIMP.

  batch-processingcgimpimage-manipulationplugin
  Language:C 894

• iusztinpaul / energy-forecasting

  🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

  airflowbatch-processingfeature-storegithub-actionsmlopspythoncicddockerfastapigcpgreat-expectationshopsworksml-monitoringpoetrysktimestreamlitweights-and-biases3-pipeline-designmodel-registrydata-versioning
  Language:Python 877

• bacalhau-project / bacalhau

  Compute over Data framework for public, transparent, and optionally verifiable computation

  ai-artai-data-collectionai-pipelinebatch-processingbioinformatics-pipelinedata-analysisdata-engineeringdata-sciencedecentralizeddecentralized-computingdistributedgene-sequencinginsulatorsiotlogging-frameworkorchestration-frameworkp2pvideo-processing
  Language:Go 705

• j-easy / easy-batch

  The simple, stupid batch framework for Java

  batchbatch-processingjava
  Language:Java 613

• guillermo-navas-palencia / optbinning

  Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.

  pythonbinningwoeoptimizationwoebinningstreamstreaming-databatch-processingscorecardcredit-scoringmdlpcounterfactual-explanations
  Language:Python 458

• spring-cloud / spring-cloud-task

  Short lived microservices with Spring Batch

  microservicesbatch-processingshort-lived-microservices
  Language:Java 426
• krita-batch-exporter

  GDQuest / krita-batch-exporter

  A Free Krita plugin to batch export layers and groups with maximum flexibility. Scale, trim, export multiple copies of your layers...

  kritakrita-pluginexporterbatch-processingimage-processinggame-developmentgraphic-design
  Language:Python 319

• bvn-architecture / RevitBatchProcessor

  Fully automated batch processing of Revit files with your own Python or Dynamo task scripts!

  revitbatch-processingrevit-filespythonrevit-addinsrevit-batch-processingdynamorevit-apirevit-batch-processor
  Language:Python 299

• reidmorrison / rocketjob

  Ruby's missing background and batch processing system

  batchbatch-processingbatch-jobbatch-filebatch-frameworkmongodbsidekiqdelayed-jobsresqueencryptioncompressionmongoid
  Language:Ruby 288

• lilydjwg / pssh

  Parallel SSH Tools

  batch-processingparallel-processingpsshssh
  Language:Python 285

• tintinweb / ida-batch_decompile

  *Decompile All the Things* - IDA Batch Decompile plugin and script for Hex-Ray's IDA Pro that adds the ability to batch decompile multiple files and their imports with additional annotations (xref, stack var size) to the pseudocode .c file

  idadecompileida-pluginbatch-processingida-batch-decompilereverse-engineering
  Language:Python 280

• streamnative / pulsar-flink

  Elastic data processing with Apache Pulsar and Apache Flink

  apache-pulsarapache-flinkdata-processingbatch-processingstream-processingflinkflink-connectorschema-registryschemasqlflink-stream-processingcatalogpulsar
  Language:Java 278

• Azure / batch-shipyard

  Simplify HPC and Batch workloads on Azure

  azureazure-batchazure-functionsbatch-processingcontainersdockerglusterfsgpuhpcinfinibandmpinfsrdmaserverlesssingularityslurmwindows-containers
  Language:Python 277

• nilenso / goose

  A powerful background job processing library for Clojure

  asynchronousbackground-processingbatch-processingclojurerabbitmqredisscheduler
  Language:Clojure 266

• avilum / linqit

  Extend python lists operations using .NET's LINQ syntax for clean and fast coding.

  awesomebatch-processingcleanclean-codecsharpdevelopment-toolsdotnetefficiencyfilterslazy-loadinglinqmetadatametaprogrammingpowerfulpythonpython3pythonicrobustnesssyntax
  Language:Python 247

• btargac / excel-parser-processor

  Automate downloads from Excel files in seconds. Simply does the tedious, repetitive operations for rows of Excel files and reports the results. It downloads files from URL(s) in column A, if a new filename is provided at column B it will rename before saving. It will even create sub folders if column C is filled with a valid folder name.

  excelelectronelectronjsnodejsdownloaderbatch-processingbatchdownloadbatch-downloadexcel-importexcel-parseexceldownloadexcel-downloadexcel-basedrenamesubfolderautodownloaderfiledownloaderimagedownloader
  Language:JavaScript 224

• apple / batch-processing-gateway

  The gateway component to make Spark on K8s much easier for Spark users.

  batch-processingk8skubernetesspark
  Language:Java 181

• heathbm / BatchPool

  The one-stop generic task batching and management library

  asyncbatchbatch-processingc-sharpcsharpdotnetdotnet-coresemaphoretasktask-managertask-runner
  Language:C# 181
• xl-converter

  JacobDev1 / xl-converter

  Easy-to-use image converter for modern formats. Supports multithreading, drag 'n drop, and downscaling.

  converterjpeg-xlavifwebpjpegpngimage-resizerbatch-processingimagesjxl
  Language:Python 181

• NaiboWang / Bilibili-XMLSubtitle-to-ASS

  可视化Bilibili本地视频XML弹幕转换ASS字幕转换器

  assbatch-processingbilibilidanmudecryptguipotplayersubtitleup-to-datevideovisualizationxmlrename-folderbatch-processorbatch-script
  Language:Python 167

• Ding-Kyoma / EasyTrans-mac

  PDF 批量翻译，翻译后的PDF格式基本不变。导出PDF和Docx。优化并精简了来自于QPromise 的 EasyTrans。优化了通过百度翻译API稳定进行长翻译！

  batch-processingen2zhpdf-translation
  Language:Python 130

• jberet / jsr352

  Implementation of Jakarta Batch Specification and API

  batchbatch-processingjakartajava
  Language:Java 127

• pszemraj / textsum

  CLI & Python API to easily summarize text-based files with transformers

  batch-processinginferenceinference-apipipelinesummarizationsummarytexttext-to-text-transformertransformertransformers
  Language:Python 126

• yornaath / batshit

  A batch manager that will deduplicate and batch requests for a certain data type made within a window. Useful to batch requests made from multiple react components that uses react-query

  asyncbatch-processingconcurrencydeduplicationfetchreact-querytypescriptreacttanstack
  Language:TypeScript 124

• asakusafw / asakusafw

  Asakusa Framework

  asakusa-frameworkbatchbatch-processingjavahadoopmapreducedata-flowframeworkbig-data
  Language:Java 116

• streamnative / pulsar-spark

  Spark Connector to read and write with Pulsar

  apache-pulsarapache-sparkbatch-processingdata-processingdata-scienceflinksparkspark-sqlstream-processingstructured-streaming
  Language:Scala 112

• Refefer / Dampr

  Python Data Processing library

  dataflowbatch-processingmapreducemachine-learning
  Language:Python 102