Issue with Kino.JS.Live.Server.init/1 not shutting down properly

[fireindark707]

I've encountered an issue where instances of Kino.JS.Live.Server.init/1 are not shutting down properly after their use, leading to an accumulation of processes in the system. This seems to persist across sessions where Kino.JS.Live.Server is expected to terminate after completing its tasks. This can potentially lead to resource leaks and performance degradation over time.

Environment: Nerves Livebook on raspi zero w
I think this bug can be reproduced on usual livebook without nerves

Relevant Code Snippet:

defmodule SensorUpdate do
  use GenServer
  alias VegaLite, as: Vl
  @time_interval 3_000
  @data_count 500

  def start_link(frame) do
    GenServer.start_link(__MODULE__, frame, name: __MODULE__)
  end

  def init(frame) do
    :timer.send_interval(@time_interval, :draw)

    state = %{data: [], frame: frame}
    {:ok, state}
  end

  def handle_info(:draw, state) do
    {:ok, time_now} = DateTime.now("Etc/UTC")

    new_data =
      [
        %{
          datetime: time_now,
          temperatur: 36,
          humidity: 65,
          voc: 100
        }
      ] ++ state.data

    reversed_data = Enum.reverse(new_data)

    Kino.Frame.render(
      state.frame.frame_chart,
      Kino.Layout.grid(
        [
          Vl.new(height: 300, title: "VOC")
          |> Vl.data_from_values(reversed_data, only: ["datetime", "voc"])
          |> Vl.mark(:line)
          |> Vl.encode_field(:x, "datetime", type: :temporal)
          |> Vl.encode_field(:y, "voc", type: :quantitative),
          Vl.new(height: 300, title: "Temperature")
          |> Vl.data_from_values(reversed_data, only: ["datetime", "temperatur"])
          |> Vl.mark(:line)
          |> Vl.encode_field(:x, "datetime", type: :temporal)
          |> Vl.encode_field(:y, "temperatur", type: :quantitative, scale: %{domain: [25, 35]}),
          Vl.new(height: 300, title: "Humidity")
          |> Vl.data_from_values(reversed_data, only: ["datetime", "humidity"])
          |> Vl.mark(:line)
          |> Vl.encode_field(:x, "datetime", type: :temporal)
          |> Vl.encode_field(:y, "humidity", type: :quantitative, scale: %{domain: [45, 75]})
        ],
        columns: 3
      )
    )

    Kino.Frame.render(
      state.frame.frame_data,
      Kino.DataTable.new(new_data)
    )

    new_state = %{
      data: Enum.take(new_data, @data_count - 1),
      frame: state.frame
    }

    {:noreply, new_state}
  end
end

frame_chart = Kino.Frame.new()

frame_data = Kino.Frame.new()

SensorUpdate.start_link(%{frame_chart: frame_chart, frame_data: frame_data})

after running as an app for 5 minutes, there are nearly 300 processes of Kino.JS.Live.Server.init

I understand this is not a suitable use case for livebook and kino, but I just want to test the stability for Kino. I think if this issue can be solved, it will help extend the use field of livebook.

[José Valim]

Can you please isolate this into a smaller example that we can run to reproduce this? Maybe kinos are being persisted between frame renders?

[Jonatan Kłosko]

The call to Kino.DataTable.new(new_data) is going to create a new Kino.JS.Live.Server process every time, and the process is only terminated once the parent process (SensorUpdate) terminates.

For the data table specifically, #382 would address the issue.

[José Valim]

@jonatanklosko should it be tied to the current frame instead?

[Jonatan Kłosko]

@josevalim the frame does reference the kino, but Kino.start_child/1 always adds a reference from the caller to the started kino process, and in this case the process stays alive. Also, in general you could keep a kino in state and reuse across different frame renders, so I think tying directly would have false positives.

[fireindark707]

Can you please isolate this into a smaller example that we can run to reproduce this? Maybe kinos are being persisted between frame renders?

Yes, of course. I have updated

[José Valim]

@jonatanklosko so the whole issue is that it is happening inside the process. I see. Is there a way we can force it to be closed?

@fireindark707 thank you!

[Jonatan Kłosko]

@josevalim exactly. Also, I've just realised that the frame process is going to reference the kino, but dereferencing would only happen if the frame process dies (dereferencing explicitly is tricky). At the moment I can't think of a good way to close them, we could make kinos transient and expose stop functions, but it seems weird and tedious for the user to deal with that (and realistically most users wouldn't know/remember to do that).

[José Valim]

@jonatanklosko sorry, to be clear, don't you think the frame in particular could deal with dereferences when we render new content? The frame always holds its current state, doesn't it?

[Jonatan Kłosko]

@josevalim it's tricky because referencing is kino-specific, so it happens in the render protocol, we would need a new callback specifically for dereferencing. But even that's not enough, because frames may be per-user, in which case we would need to keep track of all kinos for all users and listen to know when the users go away.

[José Valim]

Alright, let's track this on #382 then, thanks!