sunnights / rill-flow

Rill Flow is a high-performance, scalable distributed workflow orchestration service

Rill Flow

Overview

Rill Flow is a high-performance, scalable distributed workflow orchestration service with the following core features:

• High performance: Supports the execution of tens of millions of tasks per day, with task execution latency less than 100ms
• Distributed: Supports the orchestration and scheduling of heterogeneous distributed systems
• Ease to use: supports visual process orchestration and plug-in access
• Cloud native: Supports cloud native container deployment and cloud native function orchestration
• AIGC: supports rapid integration of LLM model services

QuickStart

Environment Preparation

Before you begin, ensure that the following tools are installed:

• Environment suitable for OSX/Linux
• Docker
• Docker-Compose

Service Deployment

Install Rill Flow services on your local environment using Docker-Compose:

cat << EOF > docker-compose.yaml
version: '3'
services:
  rill-flow:
    image: weibocom/rill-flow
    depends_on:
      - cache
      - jaeger
    ports:
      - "8080:8080"
    environment:
      - RILL_FLOW_DESCRIPTOR_REDIS_HOST=cache
      - RILL_FLOW_DEFAULT_REDIS_HOST=cache
      - RILL_FLOW_TRACE_ENDPOINT=http://jaeger:4317
      - RILL_FLOW_CALLBACK_URL=http://rill-flow:8080/flow/finish.json
      - RILL_FLOW_TRACE_QUERY_HOST=http://jaeger:16686
  cache:
    image: redis:6.2-alpine
    restart: always
    command: redis-server --save 20 1 --loglevel warning
  jaeger:
    image: jaegertracing/all-in-one:1.39
    restart: always
    environment:
      - COLLECTOR_OTLP_ENABLED=true
  ui:
    image: weibocom/rill-flow-ui
    ports:
      - "8088:80"
    depends_on:
      - rill-flow
      - jaeger
    environment:
      - BACKEND_SERVER=http://rill-flow:8080
  sample-executor:
    image: weibocom/rill-flow-sample:sample-executor 
EOF
docker-compose up -d

Execution Example

• Step 1: Submit a YAML file defining a workflow

curl --location  --request POST 'http://127.0.0.1:8080/flow/bg/manage/descriptor/add_descriptor.json?business_id=rillFlowSimple&feature_name=greet&alias=release' \
--header 'Content-Type: text/plain' \
--data-raw '---
version: 1.0.0
workspace: rillFlowSimple
dagName: greet
type: flow
tasks:
  - category: function
    name: Bob 
    resourceName: http://sample-executor:8000/greet.json?user=Bob
    pattern: task_sync
    tolerance: false
    next: Alice
    inputMappings:
      - source: "$.context.Bob"
        target: "$.input.Bob"
  - category: function
    name: Alice 
    resourceName: http://sample-executor:8000/greet.json?user=Alice
    pattern: task_sync
    tolerance: false
    inputMappings:
      - source: "$.context.Alice"
        target: "$.input.Alice"
'

• Step 2: Submit a task to execute the workflow

curl -X POST 'http://127.0.0.1:8080/flow/submit.json?descriptor_id=rillFlowSimple:greet'  -d '{"Bob":"Hello, I am Bob!", "Alice": "Hi, I am Alice"}' -H 'Content-Type:application/json'

• Step 3：Query the task execution results

  Query the execution details via the Rill Flow UI.(admin/admin)

http://127.0.0.1:8088/#/flow-instance/list

Document

• Document
• 中文文档

Contributors

The following are contributors to the project along with their GitHub links:

• axb (@qdaxb) Maintainer
• techlog (@techloghub) Maintainer
• ch15084 (@ch15084) Maintainer
• Ocean (@hhh041)
• xilong-t (@xilong-t)
• qfl (@qiaofenlin)
• Kylen (@Kylen)
• zzfzzf (@zzfzzf)
• feifei (@feifei325)
• moqimoqidea (@moqimoqidea)
• Guo, Jiansheng (@guojiansheng0925)

License

Rill Flow is an open-source project under the Apache License 2.0.