State does not get edited in long lasting (background) tasks

[Mike W]

I have a slow task that I need to run. I can approach it as a background task or trigger it as a normal event. The problem is that I cannot use async because it is Selenium. Maybe it is worth mentioning that the task uses multiprocessing. After running it for a minute, I cannot modify the state anymore eg.

State(rx.State):
  ...

  def on_click(self):
     return State.slow_task

  @rx.background
  async def slow_task(self):
     async with self:
       print("10% done")
       self.var = "10% done"

     sleep(60) # slow code, I use sleep and not async sleep to simulate Selenium

     async with self:
       print("40% done"
       self.var = "40% done"
     # usually by this point var is not longer updates the UI.
     # var does not get updated in the UI but I can see in the terminal "40% done"

    ...

I have played with setting the following variables (see below) but sometimes it works and sometimes it does not work, sometimes even the on_click event does not get executed. I have also tried making slow_task a not background task and running it.

config.Expiration.PING = 10000
config.Expiration.TOKEN = 10000
config.Expiration.LOCK = 1000 * 60 * 10 
config.Ping.INTERVAL = 250_000

What is the best way of running these type of very slow functions?

[Masen Furer]

You really do not want to block in a background task.

Because reflex is an async-based framework, any non-async functions that are long running will block the whole server from processing other requests, sending deltas, etc.

If you need to execute blocking code in reflex, then you can farm it out to a separate process or thread and monitor the result in the background task.

In your code example above, you'll notice that if you open a second tab and try to click the button it actually wont do anything while the first tab is crunching along.

For simple stuff, you can use await asyncio.get_event_loop().run_in_executor(my_func) to run the function in a thread and await the result.

For heavier or CPU-bound tasks, then you can run with a process pool. Here's some example

from __future__ import annotations

import asyncio
import multiprocessing
import multiprocessing.pool
import time

import reflex as rx


_global_pool: multiprocessing.Pool | None = None


class State(rx.State):
    var: str = "Not Started"

    def _pool(self) -> multiprocessing.Pool:
        global _global_pool
        if _global_pool is None:
            _global_pool = multiprocessing.Pool()
        return _global_pool

    @rx.background
    async def slow_task(self):
        start_time = time.time()
        async_result = self._pool().apply_async(time.sleep, (60,))
        async with self:
            self.var = "Working..."
        print("Dispatched task to process pool")

        # wait for the task to complete
        while not async_result.ready():
            await asyncio.sleep(1)

        print(f"elapsed time: {time.time() - start_time}")
        print(f"result: {async_result.get()}")
        async with self:
            self.var = "Done!"


def index() -> rx.Component:
    return rx.container(
        rx.vstack(
            rx.button("Run slow task", on_click=State.slow_task),
            rx.text(f"Status: {State.var}"),
            align="center",
        ),
    )


app = rx.App()
app.add_page(index)

[Mike W]

Thank you for your answer, I will give it a try and get back to you!

[Mike W]

Your idea worked great. thanks for suggesting it!