superduper-io / superduper

Superduper: build end-2-end AI applications and templates using your existing data infrastructure and tools of choice

Home Page:https://superduper.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[BUG] Logic at startup missing

blythed opened this issue · comments

When we load superduper the queue should be initialized with existing Listener instances.
This logic is missing.

class LocalSequentialQueue:
    """
    LocalSequentialQueue for handling publisher and consumer process.

    Local queue which holds listeners, vector indices as queue which
    consists of events to be consumed by the corresponding components.
    """
    def __init__(self):
        self.queue = {}
        self.components = {}  # --> should be prepopulated
        self._db = None
        self._component_map = {}

    def declare_component(self, component):
        """Declare component and add it to queue."""
        identifier = f'{component.type_id}.{component.identifier}'
        self.queue[identifier] = []
        self.components[identifier] = component

It would be better to load the queue dynamically. Initialize the queue only when the corresponding event is received and the corresponding component does not exist.

WDYT? @blythed