Multithreading or "--fast" tasks issue

[dinosaurtirex]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

If you are using built-in tasks in sanic with multithreading and more than 1 worker, when you try to get task by name, it will raise error that task dont found, until you randomly got yourself to your thread, then it shows

Code snippet

async def check_any_task_status(request: Request, task_id: str) -> dict:
    print(request.app._task_registry) # If you are using 8 workers, 1/8 chance it will be not empty 
    task = request.app.get_task(task_id)
    if task._state != "FINISHED":
        return {"state": task._state, "result": ""}
    else:
        return {"state": task._state, "result": task.result()}

Expected Behavior

I expect to always got my task also with more than 1 worker. Is it even possible?

How do you run Sanic?

Sanic CLI

Operating System

Ubuntu 22.04

Sanic Version

Sanic 23.3.0; Routing 22.8.0

Additional context

Thanks to developers for such grate framework, hope to solve this issue soon...

[dinosaurtirex]

I read source code of tasks realization and understand that is basically working with sanic event loop, which is unique for each worker, that's why it's error occur. I guess better idea is to use other tasks backend than built in

[Adam Hopkins]

@dinosaurtirex That is correct. The reason is that the tasks you are creating and retrieving are event loop tasks. Instead, it sounds like you want something more durable this this: https://amhopkins.com/posts/background-job-worker.html, which could be accessible by each of your workers.