weld-project / weld

High-performance runtime for data analytics applications

Home Page:https://www.weld.rs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Loop fusion rule incorrect when loop indices accessed

sppalkia opened this issue · comments

The loop fusion rule incorrectly fuses loops when the loop index is accessed in the second loop.

Example:

|v: vec[i32]|
  let intermediate = result(for(v, appender[i32], |b,i,e|
    if (e > 0, merge(b, e), b)
  )); 

  # After fusion, i here will be incorrect.
  result(for(intermediate, appender[{i32,i64}], |b,i,e|
    merge(b, {e, i})
  ))

With the input [0, 1, 0, 0, 2], the expected value of intermediate is [1, 2], and the expected final output is [{1, 0}, {2, 1}].

However, loop fusion will produce the following code:

|v:vec[i32]|
  result(
    for(
      v,
      appender[{i32,i64}],
      |b__2,i__2,e|
        if(
          (e>0),
          merge(b__2,{e,i__2}),
          b__2
        )
    )
  )

This produces a final output of [{1, 1}, {2, 4}], which is incorrect.

@kraftp this is the issue you're seeing.