Create Intermediate Representation Between arroyo-sql::SqlOperator and arroyo-datastream::Program
jacksonrnewhouse opened this issue · comments
Currently this is translated in a single call to get_program_from_operator().
However, this code path is overloaded, responsible for code generation, creation of the operator graph, edge construction and several in-lined optimizations. This results in unclear, complex code.
For this issue, the task is to define another representation, SqlPlanGraph which can be more directly translated to the Program.
Before
[SqlOperator] - (graph construction, code gen, optimization) -> [Program].
After
[SqlOperator] -(graph construction) -> [SqlPlanGraph] - (optimizations) -> [SqlPlanGraph] - (code gen) -> [Program]
Finished with #80.