Giters

apache / dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

Home Page:https://dolphinscheduler.apache.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature][EMR Task] Amazon EMR task jobFlowDefineJson supports parameter passing functionality

SEZ9 opened this issue · comments

commented

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

On the AWS platform, many customers use EMR to submit tasks like Spark. DolphinScheduler uses jobFlowDefineJson to define EMR task parameters, but currently does not support parameter passing from upstream tasks. Users hope to define parameters in Spark, such as date and time, by passing parameters.
image

Use case

On the AWS platform, many customers use EMR to submit tasks like Spark. DolphinScheduler uses jobFlowDefineJson to define EMR task parameters, but currently does not support parameter passing from upstream tasks. Users hope to define parameters in Spark, such as date and time, by passing parameters.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct