dashbitco / flow

Computational parallel flows on top of GenStage

Home Page:https://hexdocs.pm/flow

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`emit_and_reduce/3` doesn't keep the shape of accumulator

wrachwal opened this issue · comments

The following code:

[1, 2, 3]
|> Flow.from_enumerable(stages: 1)
|> Flow.flat_map(&[&1])
|> Flow.emit_and_reduce(fn -> %{} end, fn event, acc ->
  IO.inspect acc, label: "ACC"
  {[event], Map.update(acc, event, 1, & &1 + 1)}
end)
|> Enum.to_list()

gives the output:

ACC: %{}
ACC: %{1 => 1}
ACC: %{1 => 1, 2 => 1}
[1, 2, 3, {1, 1}, {2, 1}, {3, 1}]

Change the line with Flow.flat_map/2 to be:

|> Flow.flat_map(&[&1, &1])

and now the shape of the accumulator in Flow.emit_and_reduce/3 gets corrupted:

ACC: %{}
ACC: {[1], %{1 => 1}}

15:55:30.022 [error] GenServer #PID<0.170.0> terminating
** (BadMapError) expected a map, got: {[1], %{1 => 1}}
    (elixir) lib/map.ex:595: Map.update({[1], %{1 => 1}}, 1, 1, #Function<12.115662033/1 in TestFlow.emit_and_reduce/0>)
    reduce.exs:47: anonymous fn/2 in TestFlow.emit_and_reduce/0
    (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
    (flow) lib/flow/materialize.ex:623: anonymous fn/3 in Flow.Materialize.build_emit_and_reducer/2
    (flow) lib/flow/materialize.ex:630: Flow.Materialize."-build_emit_and_reducer/2-lists^foldl/2-1-"/3
    (flow) lib/flow/materialize.ex:630: anonymous fn/5 in Flow.Materialize.build_emit_and_reducer/2
    (flow) lib/flow/map_reducer.ex:54: Flow.MapReducer.handle_events/3
    (gen_stage) lib/gen_stage.ex:2315: GenStage.consumer_dispatch/6
Last message: {:"$gen_consumer", {#PID<0.169.0>, #Reference<0.210650528.1048579.149223>}, [1, 2, 3]}
State: {%{#Reference<0.210650528.1048579.149223> => nil}, %{done?: false, producers: %{#Reference<0.210650528.1048579.149223> => #PID<0.169.0>}, trigger: #Function<2.13930487/3 in Flow.Window.Global.materialize/5>}, {0, 1}, %{}, #Function<2.60253262/4 in Flow.Materialize.build_emit_and_reducer/2>}
** (exit) exited in: GenStage.close_stream(%{})
    ** (EXIT) an exception was raised:
        ** (BadMapError) expected a map, got: {[1], %{1 => 1}}
            (elixir) lib/map.ex:595: Map.update({[1], %{1 => 1}}, 1, 1, #Function<12.115662033/1 in TestFlow.emit_and_reduce/0>)
            reduce.exs:47: anonymous fn/2 in TestFlow.emit_and_reduce/0
            (elixir) lib/enum.ex:1925: Enum."-reduce/3-lists^foldl/2-0-"/3
            (flow) lib/flow/materialize.ex:623: anonymous fn/3 in Flow.Materialize.build_emit_and_reducer/2
            (flow) lib/flow/materialize.ex:630: Flow.Materialize."-build_emit_and_reducer/2-lists^foldl/2-1-"/3
            (flow) lib/flow/materialize.ex:630: anonymous fn/5 in Flow.Materialize.build_emit_and_reducer/2
            (flow) lib/flow/map_reducer.ex:54: Flow.MapReducer.handle_events/3
            (gen_stage) lib/gen_stage.ex:2315: GenStage.consumer_dispatch/6
    (gen_stage) lib/gen_stage/stream.ex:160: GenStage.Stream.close_stream/1
    (elixir) lib/stream.ex:1370: Stream.do_resource/5
    (elixir) lib/enum.ex:2979: Enum.reverse/1
    (elixir) lib/enum.ex:2611: Enum.to_list/1
    reduce.exs:57: (file)