hnjm / memphis-broker

Next-Generation Real-Time Data Processing Platform

Home Page:https://docs.memphis.dev

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Memphis light logo

Memphis light logo

Simple as RabbitMQ, robust as Apache Kafka, and perfect for busy developers.

Memphis UI

CNCF Silver Member

CNCF Silver Member

Sandbox - Docs - Twitter - YouTube

Discord Code Of Conduct GitHub release (latest by date)

Memphis is a next-generation alternative to traditional message brokers.

A simple, robust, and durable cloud-native message broker wrapped with
an entire ecosystem that enables cost-effective, fast, and reliable development of modern queue-based use cases.

Memphis enables the building of modern queue-based applications that require
large volumes of streamed and enriched data, modern protocols, zero ops, rapid development,
extreme cost reduction, and a significantly lower amount of dev time for data-oriented developers and data engineers.

🫣 A world without Memphis

When your application requires a message broker or a queue,
Implementing one will require you to -

  • Build a dead-letter queue
  • Create observability
  • Build a scalable environment
  • Create client wrappers
  • Handle back pressure. Client or queue side
  • Create a retry mechanism
  • Configure monitoring and real-time alerts
  • Deal with ACLs
  • Somehow create a cloud-agnostic implementation
  • Create config alignment between production to a dev environment
  • Spent weeks and months learning the internals through archival documentation, ebooks, and courses
  • Onboard your developers
    And the list continues...

Or, you can just use Memphis and focus your resources on tasks that matter 😎

✨ Features v0.4.5

Roadmap

20 Production-ready message broker in under 3 minutes
20 Easy-to-use UI, CLI, and SDKs
20 Data-level observability
20 Dead-Letter Queue with automatic message retransmit
20 Schemaverse - Embedded schema management for produced data (Protobuf/JSON/GraphQL/Avro)
20 Storage tiering
20 SDKs: Node.JS, Go, Python, Typescript, NestJS, REST, .NET, Kotlin
20 Kubernetes-native
20 Community driven

🚀 Getting Started

Helm for Kubernetes☸

helm repo add memphis https://k8s.memphis.dev/charts/ --force-update && \
helm install my-memphis memphis/memphis --create-namespace --namespace memphis

Docker🐳 Compose

curl -s https://memphisdev.github.io/memphis-docker/docker-compose.yml -o docker-compose.yml && \
docker compose -f docker-compose.yml -p memphis up

connect your first app

Tutorials
Sandbox
Installation videos

High-Level Architecture

memphis.dev Architecture

Local access

Via Kubernetes

To access Memphis using UI/CLI/SDK from localhost, run the below commands:

  - kubectl port-forward service/memphis-cluster 6666:6666 9000:9000 7770:7770 --namespace memphis > /dev/null &

For interacting with the broker via HTTP:

  - kubectl port-forward service/memphis-rest-gateway 4444:4444 --namespace memphis > /dev/null &

Dashboard/CLI: http://localhost:9000
Broker: localhost:6666 (Client Connections)
REST gateway: localhost:4444 (Data + Mgmt)

For Production Environments Please expose the UI, Cluster, and Control-plane via k8s ingress / load balancer / nodeport

Via Docker

Dashboard/CLI: http://localhost:9000
Broker: localhost:6666

👉 Use-cases

  • Async task management
  • Real-time streaming pipelines
  • Data ingestion
  • Cloud Messaging
    • Services (microservices, service mesh)
    • Event/Data Streaming (observability, analytics, ML/AI)
  • Queuing
  • N:N communication patterns
  • Ingest Grafana Loki logs at scale

Support 🙋‍♂️🤝

Ask a question ❓ about Memphis{dev} or something related to us:

We welcome you to our discord server with your questions, doubts and feedback.

Create a bug 🐞 report

If you see an error message or run into an issue, please create bug report. This effort is valued and it will help all Memphis{dev} users.

Submit a feature 💡 request

If you have an idea, or you think that we're missing a capability that would make development easier and more robust, please Submit feature request.

If an issue❗with similar feature request already exists, don't forget to leave a "+1". If you add some more information such as your thoughts and vision about the feature, your comments will be embraced warmly :)

Contributing

Memphis{dev} is an open-source project.
We are committed to a fully transparent development process and appreciate highly any contributions.
Whether you are helping us fix bugs, proposing new features, improving our documentation or spreading the word - we would love to have you as part of the Memphis{dev} community.

Please refer to our Contribution Guidelines and Code of Conduct.

Contributors ✨

Thanks goes to these wonderful people ❤:

License 📃

Memphis is open-sourced and operates under the "Memphis Business Source License 1.0" license Built out of Apache 2.0, the main difference between the licenses is: "You may make use of the Licensed Work (i) only as part of your own product or service, provided it is not a message broker or a message queue product or service; and (ii) provided that you do not use, provide, distribute, or make available the Licensed Work as a Service. A “Service” is a commercial offering, product, hosted, or managed service, that allows third parties (other than your own employees and contractors acting on your behalf) to access and/or use the Licensed Work or a substantial set of the features or functionality of the Licensed Work to third parties as a software-as-a-service, platform-as-a-service, infrastructure-as-a-service or other similar services that compete with Licensor products or services." Please check out License to read the full text.

About

Next-Generation Real-Time Data Processing Platform

https://docs.memphis.dev

License:Other


Languages

Language:Go 85.6%Language:JavaScript 11.3%Language:SCSS 2.8%Language:Shell 0.2%Language:HTML 0.1%Language:Dockerfile 0.0%