dask-distributed

There are 0 repository under dask-distributed topic.

• DataCanvasIO / HyperGBM

  A full pipeline AutoML tool for tabular data

  adversarial-validationautomlcatboostdaskdask-distributeddatacleaningdistributed-trainingensemble-learningfullpipelinegbmgpu-accelerationlightgbmpreprocessingpseudo-labelingrapidsaisemi-supervised-learningsklearntabular-dataxgboost
  Language:Python 329

• TimeEval / TimeEval

  Evaluation Tool for Anomaly Detection Algorithms on Time Series

  time-seriestime-series-analysisbenchmark-frameworkbenchmarkingtime-series-anomaly-detectiondistributeddaskdask-distributedpandasnumpypython3jupyter-notebooksanomaly-detection
  Language:Jupyter Notebook 95
• agatha

  JSybrandt / agatha

  AGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approach

  molierehypothesis-generationredisdaskdask-distributedspacyscispacyscibertbert
  Language:Python 56

• modin-project / unidist

  Unified Distributed Execution

  pythondistributedmpiraydask-distributedmultiprocessing
  Language:Python 46

• shauryashaurya / learn-data-munging

  Notes on Data Engineering with Pandas, PySpark, Dask, Ray, Arrow DataFusion, Polars etc.

  daskdata-engineeringjupyterpandaspysparkraysparkarrowdask-distributeddatafusionnumpypolars
  Language:Jupyter Notebook 44

• pyiron / pylammpsmpi

  Parallel Lammps Python interface - control a mpi4py parallel LAMMPS instance from a serial python process or a Jupyter notebook

  dask-distributedlammpslammps-python-interfacempi4pyopenmpi
  Language:Python 30

• aws-solutions-library-samples / distributed-compute-on-aws-with-cross-regional-dask

  Perform I/O intensive workloads on high-volume data sparsely located across multiple AWS regions through the use of Dask.

  daskdask-distributeddask-worker-pools
  Language:TypeScript 15

• elcorto / psweep

  Loop like a pro, make parameter studies fun.

  parameter-sweepparameter-searchparameter-estimationpandasparameter-scanparameter-studypythoncomputational-experimentdatabasedaskdask-distributeddask-jobqueue
  Language:Python 14

• jameslamb / lightgbm-dask-testing

  Test LightGBM's Dask integration on different cluster types

  lightgbmdaskdask-distributedawsmachine-learningdocker
  Language:Jupyter Notebook 12

• gdmarmerola / big-data-ml-training

  Code for "Training models when data doesn't fit in memory" post

  machine-learningdask-distributeddask-mldaskml-engineering
  Language:Jupyter Notebook 11
• SCIP

  ScalableCytometryImageProcessing / SCIP

  Scalable Cytometry Image Processing (SCIP) is an open-source tool that implements an image processing pipeline on top of Dask, a distributed computing framework written in Python. SCIP performs projection, illumination correction, image segmentation and masking, and feature extraction.

  distributed-computingdask-distributeddaskbioimage-analysiscytometry-analysis-pipeline
  Language:Python 7

• gandalf1819 / NYCOpenData-Profiling-Analysis

  Open Data Profiling, Quality and Analysis on NYC OpenData dataset with semantic profiling using fuzzy ratio, Levenshtein distance and regex

  data-profilingnyc-opendatanyc-311-datasetbig-datapysparkhdfsmodindaskdask-distributedpandasfuzzy-logicregular-expressionsfuzzywuzzylevenshtein-distance
  Language:Jupyter Notebook 6

• gjoseph92 / sneks

  Launch a Dask cluster from a Poetry environment

  daskcoileddask-distributedpoetry-python
  Language:Python 6

• epiviz / epivizFileServer

  Python library to query and transform genomic data from indexed files

  apidask-distributedfile-servergenomic-data-analysisgenomicspython3webservice
  Language:Python 5

• octoenergy / dask-remote

  Procurement: Dask Cluster as a Process.

  pythondaskdask-distributedprocurement
  Language:Python 5

• pleiszenburg / scherbelberg

  HPC cluster deployment and management for the Hetzner Cloud

  cloudhetznerclusterpythonhigh-performance-computinghigh-performancehpchpc-clustershpc-clusterdask-distributeddaskdeploymentmanagementcluster-management
  Language:Python 5

• eth-cscs / ipcluster_magic

  Magic commands to support running MPI python code as well as multi-node Dask workloads on Jupyter notebooks.

  dask-distributedipyparalleljupyter-notebookmpi4py
  Language:Python 4

• IncubatorShokuhou / dask-tutorial-chinese

  Dask tutorial；Dask汉化教程

  daskdask-distributeddask-mldask-dataframesdask-arraydelayedchinese-translation
  Language:Jupyter Notebook 4

• leosmerling-hopeit / fraud-poc

  Fraud detection ML pipeline and serving POC using Dask and hopeit.engine. Project created with nbdev: https://www.fast.ai/2019/12/02/nbdev/

  fraud-detectionnbdevmachine-learningml-pipelinesmicroservicesdaskdask-mldask-distributed
  Language:Jupyter Notebook 4

• LimnoTech / Xarray-DataAccessor

  Efficiently read climate/meteorology data into Xarray using Dask for parallelization. Transform the data for your modelling needs.

  climate-datadask-distributeddata-engineeringmeteorologyxarray-accessor
  Language:Python 4

• VorGeo / earthengine-dask

  Scale up concurrent requests to Earth Engine interactive endpoints with Dask

  dask-distributedearthengine
  Language:HTML 4

• antarcticrainforest / esm_analysis

  Python 3 tools for distributed analysis and visualisation of big climate data on HPC systems.

  climate-dataxarraydask-distributeddaskatmospheric-scienceocean-sciencesearth-sciencesearthsystem
  Language:Python 3

• JulianWgs / dask-log-server

  Preserve all necessary runtime data of a Dask client in order to "replay" and analyze the performance and behavior of the client after the fact

  daskdask-distributedloggingdashboard
  Language:Python 3

• comp-dev-cms-ita / dask-remote-jobqueue

  A custom dask remote jobqueue for HTCondor.

  daskdask-distributeddask-jobqueuehtcondor
  Language:Python 2

• maawoo / stac-access-performance

  Testing access performance of Sentinel-1 RTC metadata catalogs

  analysis-ready-datadask-distributedearth-observationmetadatasentinel-1xarray
  Language:HTML 2

• vlfom / nyc-taxi-data

  Code for fetching, sampling, and analysis of NYC taxi data from TLC and Uber for 2009-2018

  nyc-taxi-datasetnyc-taxitaxi-datauber-datadask-distributedjupyter-notebook
  Language:Jupyter Notebook 2

• fabidick22 / add-worker-DaskCluster

  Script para configuración e installacion de requermientos de un worker de Dask Distributed

  daskdask-distributedworker
  Language:Shell 1

• JBris / pycaret-fugue-dask-test

  Testing PyCaret, Fugue, and Dask

  daskdask-distributedfuguepycaretpycaret-library
  Language:Python 1

• lebedov / dask-ml-on-azure-ml

  Using Dask-ML on Azure ML

  azure-mldask-mldask-distributed
  Language:Python 1

• mabaszadeh / distributed-tsp

  Distributed solution for Traveling Salesman Problem using Dask.distributed and OR-Tools

  daskdask-distributeddistributedor-toolstraveling-salesman-problemtravelling-salesman-problemtsp
  Language:Python 1

• rolani / dask-ecs-lib

  dask-ecs-lib is a Python library that effortlessly spins up a Dask cluster on AWS ECS using Fargate, allowing you to seamlessly execute and parallelize your functions.

  aws-ecsdask-distributedfargateparallel-computingpython
  Language:Python 1

• sulis-hpc / sulis-hpc.github.io

  User documentation website for the Sulis tier 2 HPC service. Built using Jekyll.

  cdask-distributedensemble-methodsfortran90hpcmpimpi4pyopenmpparallel-computingpythonrslurmukwarwickuni
  Language:SCSS 1

• arjunsawhney1 / scalable-ML

  In this repo, I build a LogisticRegression prediction model with Dask and PySpark and initialize an AWS EMR cluster to run the entire pipeline.

  awsaws-ec2aws-emr-clustersaws-s3daskdask-distributeddask-mllogistic-regressionpysparkscalabilityspark
  Language:Python 0

• gyasi87 / NYC_Taxi_Data

  NY City Taxi Analysis using Dask

  analysisdaskdask-distributedpythontaxi
  Language:Jupyter Notebook 0

• ivanbgd / dask_demo_reins

  A Dask library for Big Data processing in Python demo

  daskdask-distributeddistributeddistributed-computingpythonpython3insurancereinsurancelarger-than-memory
  Language:Python

• jkanche / asynchronous-api-dask-terraform

  Asynchronous API using Dask and AWS Fargate

  awsdask-distributedfargate-containersfastapi
  Language:HCL