prometheus / client_python

Prometheus instrumentation library for Python applications

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Why is the CPU usage so high?

rourke opened this issue · comments

commented

I've created a simple test script with a gauge that outputs a random number. I'm using the latest v0.17.1. I've put my script in a docker python image. To my surprise the CPU usage is way higher than I would expect from a script as simple as this. Docker reports the container uses 100% of the CPU resources, and I see an 18% increase on my host CPU usage. The CPU being an Intel i3-10105 @ 3.70GHz.

I'm by no means an experienced python programmer, but I can manage myself. I'm really curious what causes this amount of CPU usage and what I can maybe do to decrease it.

app.py

import prometheus_client
import random

# Disable useless default metrics
prometheus_client.REGISTRY.unregister(prometheus_client.GC_COLLECTOR)
prometheus_client.REGISTRY.unregister(prometheus_client.PLATFORM_COLLECTOR)
prometheus_client.REGISTRY.unregister(prometheus_client.PROCESS_COLLECTOR)

g = prometheus_client.Gauge(
    "my_test_gauge", "Test gauge to output a random number between 0 and 1, 2 decimals"
)

def process_request(t):
    """A dummy function that sets the gauge value"""
    g.set(t)  # Set to a given value


if __name__ == "__main__":
    # Start up the server to expose the metrics.
    prometheus_client.start_http_server(8000)
    # Generate some requests.
    while True:
        process_request(round(random.random(), 2))

Dockerfile

FROM python:3.11-slim-bullseye

# Copy files
RUN mkdir /app
ADD . /app
WORKDIR /app

# Install python packages
RUN pip install -q --root-user-action=ignore --no-cache --upgrade pip \
    && pip install -q --root-user-action=ignore --no-cache -r requirements.txt

EXPOSE 8000

# Run app.py
CMD ["python", "app.py"]
while True:
    process_request(round(random.random(), 2))

You've written an infinite loop with no sleep calls or waits for IO, so it will run as fast as it possibly can, and use 100% of the CPU core

Closing this as the above answer is correct.

commented

All the readme examples contain this part. Maybe it's a good idea to provide an exanple that doesn't consume 100% CPU.

The README example runs time.sleep in process_request to avoid this issue.