Giters

realazthat / comfy-fulcrum

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ComfyUI Fulcrum

Audience: Developers Platform: Linux

🏠Home  •  🎇Features  •  🔨Install  •  🚜Usage  •  💻CLI  •  💡Examples

🤖Jinja2 API  •  ✅Requirements  •  🐳Docker  •  🚸Gotchas

Top language GitHub License PyPI - Version Python Version

Load balancer for ComfyUI instances

Status Stable Unstable
Master Build and Test since tagged last commit
Develop Build and Test since tagged since tagged last commit

Demo

❔ What

What it does: ComfyUI Fulcrum lets you manage exclusive access to multiple resources (ComfyUI API URLs).

🎇 Features

  • Channels.
  • Priority: Clients can specify priority for a lease request.
  • Expiry: Clients can specify how long they need the lease; if they fail to "touch" the lease before expiry, it is released.

🔨 Install

  • cli: CLI that gives access to fastapi_{server, client}, backed by db, from the command line.
  • fastapi_client: Client, library. Install this if you want to use the API from python.
  • fastapi_server: Server, library. Install this if you want to serve the API from python.
  • fastapi_mgmt_ui: Management UI, library. Install this if you want to serve the management UI from python.
  • db: DB implementation, library. Can be combined with fastapi_server as a serving solution, from python.
  • base: Interfaces library. Install this if you want to implement your own implementations.
# Install from pypi (https://pypi.org/project/comfy-fulcrum/)
pip install comfy-fulcrum[cli,db,fastapi_server,fastapi_mgmt_ui,fastapi_client]

# Install from git (https://github.com/realazthat/comfy-fulcrum)
pip install git+https://github.com/realazthat/comfy-fulcrum.git@v0.0.1[cli,db,fastapi_server,fastapi_mgmt_ui,fastapi_client]

🚜 Usage

Server via CLI:

python -m comfy_fulcrum.cli server \
  --dsn "${DSN}" \
  --host 0.0.0.0 --port "${FULCRUM_PORT}"

Client via CLI:

python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

RESOURCE_A_ID=a
python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"resource_id": "'"${RESOURCE_A_ID}"'","channels": ["main"], "data": "{\"comfy_api_url\": \"url\"}"}' \
  register

python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  list

python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

TICKET=$(python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"client_name": "me","channels": ["main"], "priority": "1"}' \
  get)

# Example TICKET:
# {"id": "4fe26e1fe5974e68b788f82cc9460b0b", "client_name": "me", "lease_timeout": 180.0, "ends": "2024-07-21T01:18:16.317032"}
echo "TICKET: ${TICKET}"

TICKET_ID=$(echo "${TICKET}" | jq -r '.id')
echo "TICKET_ID: ${TICKET_ID}"

RESOURCE_ID=
while [[ -z "${RESOURCE_ID}" ]]; do
  echo -e "${BLUE}Checking if resource is ready${NC}"
  TICKET=$(python -m comfy_fulcrum.cli client \
    --fulcrum_api_url "${FULCRUM_API_URL}" \
    --data '{"id": "'"${TICKET_ID}"'"}' \
    touch)
  echo "TICKET: ${TICKET}"
  RESOURCE_ID=$(echo "${TICKET}" | jq -r '.resource_id')
  echo "RESOURCE_ID: ${RESOURCE_ID}"
  sleep 2
done

RESOURCE_DATA=$(echo "${TICKET}" | jq -r '.data')
COMFY_API_URL=$(echo "${RESOURCE_DATA}" | jq -r '.comfy_api_url')

echo -e "${BLUE}COMFY_API_URL: ${COMFY_API_URL}${NC}"

python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

sleep 2
python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"id": "'"${TICKET_ID}"'", "report": "success", "report_extra": {}}' \
  release

python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"resource_id": "'"${RESOURCE_A_ID}"'"}' \
  remove

python -m comfy_fulcrum.cli client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

Server via Python:

  app = fastapi.FastAPI()

  db_engine: AsyncEngine = create_async_engine(dsn,
                                               echo=False,
                                               pool_size=10,
                                               max_overflow=20,
                                               pool_timeout=5,
                                               pool_recycle=1800)
  fulcrum = DBFulcrum(engine=db_engine,
                      lease_timeout=60. * 5,
                      service_sleep_interval=1.0)
  await fulcrum.Initialize()
  routes = FulcrumServerRoutes(fulcrum=fulcrum)
  app.include_router(routes.Router())

  hypercorn_config = hypercorn.Config()
  hypercorn_config.bind = [f'0.0.0.0:{port}']
  hypercorn_config.use_reloader = True
  hypercorn_config.loglevel = 'debug'
  hypercorn_config.accesslog = '-'
  hypercorn_config.errorlog = '-'
  hypercorn_config.debug = True

  await hypercorn.asyncio.serve(app, hypercorn_config)  # type: ignore

Client (resource provider) via Python:

  client = FulcrumClient(fulcrum_api_url=fulcrum_api_url)

  print(await client.Stats(), file=sys.stderr)

  print(await client.RegisterResource(channels=[ChannelID('main')],
                                      resource_id=ResourceID(uuid.uuid4().hex),
                                      data=json.dumps(
                                          {'comfy_api_url': comfy_api_url})),
        file=sys.stderr)
  print(await client.ListResources(), file=sys.stderr)

Client (resource consumer) via Python:

  client = FulcrumClient(fulcrum_api_url=fulcrum_api_url)

  print(await client.Stats(), file=sys.stderr)

  lease: Union[Lease, Ticket] = await client.Get(client_name=ClientName('test'),
                                                 channels=[ChannelID('main')],
                                                 priority=1)
  print(lease, file=sys.stderr)

  async def WaitForResource(id: LeaseID) -> Lease:
    while True:
      lease_or = await client.TouchTicket(id=id)
      if isinstance(lease_or, Lease):
        return lease_or
      elif isinstance(lease_or, Ticket):
        await asyncio.sleep(1)
      elif lease_or is None:
        print('Lease has expired', file=sys.stderr)
        exit(1)

      await asyncio.sleep(1)

  lease = await WaitForResource(id=lease.id)

  print(lease, file=sys.stderr)

  print(await client.Stats(), file=sys.stderr)

  # Now that we have the lease, let's get the data, which we happen to know is
  # json with the comfy_api_url as {'comfy_api_url': <url>}.
  resource_data = json.loads(lease.data)
  comfy_api_url = resource_data['comfy_api_url']

  # Do something with the comfy_api_url here:
  print(comfy_api_url, file=sys.stderr)
  await asyncio.sleep(1)

  # Don't forget to touch the lease occasionally, otherwise it will expire.
  await client.TouchLease(id=lease.id)
  await asyncio.sleep(1)

  # Once done with the lease, release it.
  await client.Release(id=lease.id, report='success', report_extra=None)

  print(await client.Stats(), file=sys.stderr)

💻 Command Line Options

Output of `python -m comfy_fulcrum.cli --help`

Output of `python -m comfy_fulcrum.cli server --help`

Output of `python -m comfy_fulcrum.cli client --help`

🐳 Docker Image

Docker images are published to ghcr.io/realazthat/comfy-fulcrum at each tag.

Gotcha: --tty will output extra hidden characters.

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 server \
  --dsn "${DSN}" \
  --host 0.0.0.0 --port "${FULCRUM_PORT}"
docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

RESOURCE_A_ID=a
docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"resource_id": "'"${RESOURCE_A_ID}"'","channels": ["main"], "data": "{\"comfy_api_url\": \"url\"}"}' \
  register

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  list

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

TICKET=$(docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"client_name": "me","channels": ["main"], "priority": "1"}' \
  get)

# Example TICKET:
# {"id": "4fe26e1fe5974e68b788f82cc9460b0b", "client_name": "me", "lease_timeout": 180.0, "ends": "2024-07-21T01:18:16.317032"}
echo "TICKET: ${TICKET}"

TICKET_ID=$(echo "${TICKET}" | jq -r '.id')
echo "TICKET_ID: ${TICKET_ID}"

RESOURCE_ID=
while [[ -z "${RESOURCE_ID}" ]]; do
  echo -e "${BLUE}Checking if resource is ready${NC}"
  TICKET=$(docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
    --fulcrum_api_url "${FULCRUM_API_URL}" \
    --data '{"id": "'"${TICKET_ID}"'"}' \
    touch)
  echo "TICKET: ${TICKET}"
  RESOURCE_ID=$(echo "${TICKET}" | jq -r '.resource_id')
  echo "RESOURCE_ID: ${RESOURCE_ID}"
  sleep 2
done

RESOURCE_DATA=$(echo "${TICKET}" | jq -r '.data')
COMFY_API_URL=$(echo "${RESOURCE_DATA}" | jq -r '.comfy_api_url')

echo -e "${BLUE}COMFY_API_URL: ${COMFY_API_URL}${NC}"

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

sleep 2
docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"id": "'"${TICKET_ID}"'", "report": "success", "report_extra": {}}' \
  release

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"resource_id": "'"${RESOURCE_A_ID}"'"}' \
  remove

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  ghcr.io/realazthat/comfy_fulcrum:v0.0.1 client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

If you want to build the image yourself, you can use the Dockerfile in the repository.

Build:

docker build -t my-comfy_fulcrum-image .

Server:

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image server \
  --dsn "${DSN}" \
  --host 0.0.0.0 --port "${FULCRUM_PORT}"

Client:

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

RESOURCE_A_ID=a
docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"resource_id": "'"${RESOURCE_A_ID}"'","channels": ["main"], "data": "{\"comfy_api_url\": \"url\"}"}' \
  register

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  list

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

TICKET=$(docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"client_name": "me","channels": ["main"], "priority": "1"}' \
  get)

# Example TICKET:
# {"id": "4fe26e1fe5974e68b788f82cc9460b0b", "client_name": "me", "lease_timeout": 180.0, "ends": "2024-07-21T01:18:16.317032"}
echo "TICKET: ${TICKET}"

TICKET_ID=$(echo "${TICKET}" | jq -r '.id')
echo "TICKET_ID: ${TICKET_ID}"

RESOURCE_ID=
while [[ -z "${RESOURCE_ID}" ]]; do
  echo -e "${BLUE}Checking if resource is ready${NC}"
  TICKET=$(docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
    --fulcrum_api_url "${FULCRUM_API_URL}" \
    --data '{"id": "'"${TICKET_ID}"'"}' \
    touch)
  echo "TICKET: ${TICKET}"
  RESOURCE_ID=$(echo "${TICKET}" | jq -r '.resource_id')
  echo "RESOURCE_ID: ${RESOURCE_ID}"
  sleep 2
done

RESOURCE_DATA=$(echo "${TICKET}" | jq -r '.data')
COMFY_API_URL=$(echo "${RESOURCE_DATA}" | jq -r '.comfy_api_url')

echo -e "${BLUE}COMFY_API_URL: ${COMFY_API_URL}${NC}"

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

sleep 2
docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"id": "'"${TICKET_ID}"'", "report": "success", "report_extra": {}}' \
  release

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '{"resource_id": "'"${RESOURCE_A_ID}"'"}' \
  remove

docker run --rm --network="host" \
  -v "${PWD}:/data" \
  my-comfy_fulcrum-image client \
  --fulcrum_api_url "${FULCRUM_API_URL}" \
  --data '' \
  stats

🤏 Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

🔑 License

This project is licensed under the MIT License - see the ./LICENSE.md file for details.

About

License:MIT License

Languages

Language:Python 75.9%Language:Shell 14.8%Language:Jinja 5.2%Language:HTML 3.0%Language:Dockerfile 1.0%