arehmandev / awesome-mesos

Everything about Apache Mesos

awesome-mesos

Everything about awesome Apache Mesos.

Share your Mesos ❤️ through pull requests :)

What can you expect to see here?

• Frameworks

• Data Processing

• Storage and Serving

• Service/Meta Schedulers/ PaaS

• Consensus

• Continuous Integration

• One-off tasks/commands

• Tracing

• Metric collection, logging and visualization

• Experimental/Example/Unsorted

• Language Bindings

• Tools

• Alternative UI/Dashboards

• Tools for Mesos Developers

• Tools for Mesos Framework Developers

• Command line tools

• Vagrant based setups

• Docker based setups

• Trace Visualization

• Deployment

• Cloud

• Ansible

• Chef

• Puppet

• Babushka

• Cloudformation

• Terraform

• Systemd

• Packaging

• Monitoring

• Service discovery and Load balancing

• Networking

• Modules

• Platforms and microservice architectures

• Other Projects and Integrations

• Where to look for more?

##Frameworks

###Data Processing

####Batch Processing

• Apache Spark
• Apache Hadoop
• Apache Hama
• Dpark
• Flink

####Stream/Event Processing

• Apache Storm
• Samza
• Concord
• Heron

###Storage and Serving

• Apache Cassandra
• ArangoDB
• Hypertable
• ElasticSearch
• Tachyon
• HDFS (and HDFS and HDFS)
• Riak
• Apache Kafka
• Kafka client
• Phoenix
• Apache Cotton - MySQL on Mesos
• MongoDB
• Hemlock
• Apache Drill
• Ceph
• Crate
• Apache Accumulo
• memSQL
• MrRedis - Mesos runs Redis
• Apache Ignite
• DataStax
• Kafaka -> Cassandra mirroring

###Service/Meta Schedulers/ PaaS

• Apache Aurora
• Marathon
• Singularity
• Chronos
• Cook Scheduler
• CloudFoundry
• Myriad - Elastic YARN on Mesos
• Kubernetes

###Consensus

• ZooKeeper and ZooKeeper
• Etcd

###Continuous Integration

• Jenkins and Jenkins
• GitLab CI
• Teamcity Plugin

###One-off tasks/commands

• Eremetic
• R scripts

###Tracing

• Zipkin

###Metric collection, logging and visualization

• Kibana
• Logstash
• Statsd -> Kafka and Statsd -> Kafka
• Construct - Deploy a single task on all agents of the cluster
• Go based Syslog service and Go based Syslog service
• Mesos slave metrics -> Kafka

###Experimental/Example/Unsorted

• distcc
• Exelixi
• MPI
• Chapel Parallel Programming Language
• SSSP
• JobServer
• RENDLER
• OwlCrawler
• Tryant - Distributed job scheduler in go
• Volt
• Gozer - Prototype with low-level go API
• Portainer - builds docker images using Mesos cluster
• Autoscaling
• Matrix Multiplication
• Jetty
• JobTree
• VilfredoMesos: a chameleon Mesos framework
• Sun Grid Engine
• Checkswarm
• gasc - Generic Mesos Gang Scheduler for HPC tooling
• Mesosaurus
• pinspider
• Example Python Framework
• Amazon ECS Integration (proof-of-concept)
• Charmander
• Elastic Sentiment Analysis
• Tiniest Mesos Scheduler in Python
• Anagram Finder
• Clojure Example
• Example Mesos framework in Java to launch Docker containers
• Example framework for Apache Mesos Essentials book
• Bitcoin Miner
• Closest-pairs in 2D with divide-and-conquer
• QoSon
• Logo Generator
• https://github.com/dropbox/changes-mesos-framework
• https://github.com/parker20121/mesos-joshua
• https://github.com/kaysoky/InverseOfferExampleFramework
• Deimos (deprecated when native Docker support was added to Mesos v0.20)
• Slurm (deprecated)
• Satyr

Language Bindings

• C/C++
• Java
• Pure Java - Jesos
• Pure JVM - Framework API
• RxJava
• Python
• Pure Python - Pesos
• Pure Python - Pymesos
• Go
• Pure go - Gomes
• Erlang with example framework
• Haskell
• Clojure and mesomatic
• Node.js - Marathon/Chronos API wrapper
• Node.js - Scheduler/Executor API wrapper
• Ruby
• Perl
• Rust
• CLR

##Tools

###Alternative UI/Dashboards

• https://github.com/Capgemini/mesos-ui
• https://github.com/bspaans/simple-mesos-dashboard
• https://github.com/Clever/mesos-visualizer
• https://github.com/triforkse/mesos-ui

###Tools for Mesos Developers

• https://github.com/tillt/xcode-mesos
• https://github.com/mesos/modules
• Docker image for Mesos modules
• https://github.com/MesosWindows/mesoswin

###Tools for Mesos Framework Developers

• https://github.com/mesosphere/scala-sbt-mesos-framework.g8
• https://github.com/drexin/akka-mesos
• Simple Mesos "Hello world" in scala - https://gist.github.com/guenter/7471695
• Mesos State Backed Collections
• mini-mesos - Testing infrastructure for Mesos frameworks
• Fenzo - Cross framework pluggable task scheduling library
• Spring Boot starter for Mesos
• Go-Mesos-Utils

###Command line tools

• mesosctl

###Vagrant based setups

• https://github.com/tobilg/coreos-mesos-cluster
• https://github.com/everpeace/vagrant-mesos
• https://github.com/mesosphere/playa-mesos
• https://github.com/bskaggs/vagrant-deimos
• https://github.com/aharwood/vagrant-mesos-spark
• https://github.com/Woorank/vagrant-mesos-cluster
• https://github.com/ahunnargikar/vagrant-mesos
• https://github.com/liubin/mesos-marathon-deimos-vagrant
• https://github.com/antonlindstrom/mesos_playground
• https://github.com/mdevilliers/vagrant-mesos-development-environment
• https://github.com/gavinln/mesos-marathon
• https://github.com/rasputnik/mesos-centos
• https://github.com/Banno/vagrant-mesos - using Atlas

###Docker based setups

• https://github.com/breerly/fig-mesos
• https://github.com/dontrebootme/compose-mesos
• https://github.com/redjack/docker-mesos
• https://github.com/yaronr/docker-mesos
• https://github.com/schibsted/mesoscope

###Trace Visualization

• https://github.com/tnachen/mesos_traces_vis
• https://github.com/mesosphere/mesos-tracing

##Deployment

###Cloud

• Azure Meoss
• Azure DC/OS

###Ansible

• https://github.com/mhamrah/ansible-mesos-playbook
• https://github.com/fupelaqu/ansible-mesos
• https://github.com/frankhinek/ansible-mesos-cluster
• https://github.com/curtisgithub/ansible-mmd
• https://github.com/AnsibleShipyard/ansible-mesos
• https://github.com/AnsibleShipyard/ansible-mesos-docker
• https://github.com/AnsibleShipyard/ansible-chronos
• https://github.com/AnsibleShipyard/ansible-marathon
• Deploy apps on marathon from ansible
• roger-mesos with Bamboo

###Chef

• https://github.com/everpeace/cookbook-mesos
• https://github.com/mdsol/mesos_cookbook

###Puppet

• https://github.com/deric/puppet-mesos

###Babushka

• https://github.com/parolkar/mesos-babushka

###Cloudformation

• https://github.com/thefactory/cloudformation-mesos

###Terraform

• https://github.com/ContainerSolutions/terraform-mesos
• https://github.com/tonyjchong/terraform-mesos

###Systemd

• https://github.com/adobe-platform/mesos-systemd

###Packaging

• https://github.com/nmilford/rpm-mesos
• https://github.com/berngp/mesos-rpm
• https://github.com/deric/mesos-deb-packaging
• https://github.com/mesosphere/mesos-deb-packaging

###Monitoring/Alerting

• https://github.com/opentable/nagios-mesos
• https://github.com/bobrik/collectd-mesos-tasks and (https://github.com/rayrod2030/collectd-mesos and its dockerized version - https://github.com/bobrik/docker-collectd-mesos)
• Satellite
• Prometheus and Prometheus
• https://github.com/kpacha/mesos-influxdb-collector
• Complainer

###Service discovery and Load balancing

• Automated HAProxy reconfiguration for Marathon
• DNS based Service Discovery for Mesos
• Service Discovery script for Mesos and Marathon
• Marathoner - Service discovery in Marathon
• Bamboo - Automatically configuring HAProxy for Mesos+Marathon
• Mesos-Consul and Service Discovery & Orchestration With Mesos and Consul
• Ralph
• Zoidberg
• Aurproxy
• Marathon-Consul - Register Marathon Tasks as Consul Services for service discovery.
• roger-bamboo
• traefik

###Networking

• Project Calico

Modules

• A customer allocator module
• Serenity
• Metaswitch
• Network Isolator
• Remote Commands Execution
• Flocker volumes
• Docker Volume Driver Isolator

Platforms and microservice architectures

• DC/OS
• Mantl
• Apollo
• PanteraS - PanteraS - Platform as a Service in a box
• Vamp - The Very Awesome Microservices Platform
• Compute platform
• PaaSTA
• Appsoma Welder

##Other projects and Integrations

• BigDataScript
• https://github.com/mesosphere/sample_mesos_executor
• https://github.com/CloudCredo/mesos-boshrelease
• https://github.com/cf-platform-eng/mesos-boshrelease
• https://github.com/riywo/sample-fluentd-on-mesos-docker
• https://github.com/mesosphere/mesos-utils
• https://github.com/mesosphere/mesos-cli
• https://github.com/opentable/mesoshub
• https://github.com/thefactory/marathon-python
• https://github.com/thefactory/marathon-logger
• https://github.com/mesosphere/marathon_client
• https://github.com/mesosphere/service-bridge
• https://github.com/mesosphere/marathon-pkg
• https://github.com/jbdalido/gomarathon
• https://github.com/mesosphere/chronos-pkg
• https://github.com/mesosphere/chronos-utils
• Mesos in Hadoop
• CDH patched for Mesos - old
• https://github.com/siliconcow/docker_paas - old
• https://github.com/jbdalido/hecate
• https://github.com/nlamirault/magneto
• https://github.com/alfongj/mesos-akaros
• https://github.com/charlescearl/VirtualMesos - old
• https://github.com/mohitsoni/mammoth
• https://github.com/ptorrestr/mesos-spark
• https://github.com/gavinln/mesos-jenkins
• https://github.com/smarthall/packer-mesos
• https://github.com/JasonGiedymin/chef-mesos
• https://github.com/preillyme/mpm
• https://github.com/cashoefman/chronos
• https://github.com/thefactory/docker-marathon
• https://github.com/veverjak/coreos-mesos-marathon
• https://github.com/tnolet/mesos_on_coreos
• https://github.com/jayusor/mesos
• https://github.com/nqn/angstrom
• supervisor - https://github.com/tnn1t1s/learn-mesos-marathon
• NixOps - https://github.com/wmertens/nixops-mesos
• Universe - Mesos package repository
• https://github.com/sheepkiller/presto-marathon-docker
• https://github.com/ortoo/mesos-nerve
• https://github.com/theclaymethod/Foundry-vagrant-mesos-kafka-cluster
• Autoscaling Mesos
• Aurora REST interface - https://github.com/misho-kr/mesos-aurora-restful and https://github.com/smarth-madan/incubator-aurora
• Storm Marathon
• https://github.com/tailhook/mesos-tests
• REPL-MESOS
• Relay.Mesos
• Ochopod
• Mesos on Mesos
• Load replaying
• Vamp
• Dispatch - execute scripts on Mesos cluster
• Charmander
• Compose-executor
• Triathlon - Marathon wrapper for distributed Mesos cluster selection
• megos - Go(lang) client library for accessing information of a Apache Mesos cluster
• Weave Mesos Integration
• OpenStack Kolla
• Chimp
• Depcon
• Vault
• Go Mesos Kafka Consumer
• Apache Mesos Platform as a Service Deploy
• JIRA on Mesos
• Dask Mesos backend
• Spring Cloud Data Flow

##Where to look for more?

• MesosCon Videos - 2014, 2015, Europe 2015, Seattle 2015
• Mesos User Groups
• Powered By Mesos
• Mesos Community
• List of Mesos related conferences & meetups
• Apache Mesos Essentials book