shursulei / scaleph

Open data platform based on flink. Now scaleph is supporting data integration with seatunnel on flink

scaleph

The Scaleph project features data integration, develop, job schedule and orchestration and aims to run on cloud environment.

It trys to provide one-stop data platform for simplifying development of data application. Scaleph hopes to help peoples to aggregate and analyze data, free data internal worth and make profit from them.

Scaleph is driven by personal interest and evolves actively through faithful developer, flowerfine is open and appreciates any helps.

Features

• Web-ui click-and-drag data integration ways backended by out-of-the-box connectors.
• Multiple versions, different deployment mode and different resource provider flink job execution ways, where we develop flinkful for solving these troubles.
• Job version management.
• Project configuration, dependency and resource.

Quick Start

Whenever people want to explore Scaleph system, they want a running Scaleph application, then people can interact with Scaleph through Scaleph Admin.

Luckily, deploy Scaleph locally just takes three steps.

• Make sure Docker installed on your machine.
• Clone the repository
• Use Docker Compose and Scaleph Docker image quickly install and run Scaleph.

git clone https://github.com/flowerfine/scaleph.git
cd scaleph/tools/docker/deploy/scaleph
docker-compose up

Once all containers have started, the UI is ready to go at http://localhost, user can login by sys_admin/123456

Documentation

please refer wiki

Build and Deployment

• develop. This doc describes how to set up local development environment of Scaleph project.
• checkstyle. Scaleph project requires clean and robust code, which can help Scaleph go further and develop better.
• build. This doc describes how to build the Scaleph project from source. Scaleph adopts maven as its build system, for more information about build from source and deployment.
• docker. As more application runs in container on cloud then bare metal machine, Scaleph provides own image.
• deploy. For different deployment purpose such as develop, test or production, Scaleph make the best effort for people deploy project on local, docker and kubernetes.
  • local. require fluent network.
  • docker. require fluent network.
  • kubernetes. work in process.

RoadMap

features

1. data ingress and egress.
   1. Flink way data integration . Scaleph reduces complexity of integration through seatunnel, flink-cdc-connectors and other flink connectors.
   2. Friendly web-ui to newbies .
2. data develop
   1. udf + sql.
   2. support multi-layer data warehouse development.
3. job schedule and orchestrate

architectures

1. cloud native
   1. container and kubernetes development and runtime environment.
      1. flink operator
      2. seatunnel operator
      3. scaleph operator
   2. java 17, quarkus.
2. plugins. https://dubbo.apache.org/zh/docsv2.7/dev/principals/

Code of Conduct

This project adheres to the Contributor Covenant code of conduct

Contributing

For contributions, please refer CONTRIBUTING

Thanks for all people who already contributed to Scaleph!

Contact

• Bugs and Features: Issues

• Chinese user:

  

Sponsor

Thanks to JetBrains for supporting us free open source licenses.

License

Scaleph is licenced under the Apache License Version 2.0, link is here.