Giters

stephenhillier / starlette_exporter

Prometheus exporter for Starlette and FastAPI

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Wrong data using startlette_exporter with fastAPI BackgroudTasks on starlette_request_duration_seconds_bucket and sum

kiwimic opened this issue · comments

commented

I added have custom endpoint where I just added one backgroud tasks which is just inserting some data log to database.

Average response time it's about 100 -150 ms

image

But exporter show's that this request was more than one 1.8 second long

image

My import is: from starlette_exporter import PrometheusMiddleware, handle_metrics

It seems to me that the exporter additionally waits for the backgroud task instead of specifying the time only for the request

commented

Hi @kiwimic , thanks for posting an issue.
I was able to reproduce the problem with a test and found a solution - #17
Unfortunately it may be a few days before I get a chance to upload a new release to PyPI.

commented

No problem thanks for fast reaction :).

Right now I found a workaround with custom metrics as my middleware, but I'm waiting for official solutions :)

HTTP_RESPONSE_TIME = Counter("custom_request_duration_seconds_total", "custom_request_duration_seconds HTTP request duration, in seconds", ("path","status_code",))
HTTP_RESPONSE_TIME_HISTOGRAM = Histogram("custom_request_duration_seconds_hist", "custom_request_duration_seconds histogram HTTP request duration, in seconds", ("path","status_code",))
HTTP_RESPONSE_TIME_SUMMARY = Summary("custom_request_duration_seconds_summary", "custom_request_duration_seconds Summary HTTP request duration, in seconds", ("path","status_code",))

@app.middleware("http")
async def add_process_time_header(request: Request, call_next):
    start_time = time.time()
    response = await call_next(request)
    process_time = time.time() - start_time
    endpoint_path = "/" + str(request.url).replace(str(request.base_url), '')
    print(endpoint_path, " - ", process_time)    
    HTTP_RESPONSE_TIME.labels(path=endpoint_path, status_code = response.status_code).inc(process_time)
    HTTP_RESPONSE_TIME_SUMMARY.labels(path=endpoint_path, status_code = response.status_code).observe(process_time)
    HTTP_RESPONSE_TIME_HISTOGRAM.labels(path=endpoint_path, status_code = response.status_code).observe(process_time)
    return response
commented

v0.8.0 has been published with the fix.
Please let me know if it's working for you after you get a chance to test it. Thanks for taking the time to post an issue!