cortexproject / cortex

A horizontally scalable, highly available, multi-tenant, long term Prometheus.

Home Page:https://cortexmetrics.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Flaky TestQueryFrontendMaxQueryLengthLimits integration test

yeya24 opened this issue · comments

Describe the bug

https://github.com/cortexproject/cortex/actions/runs/8619846095/job/23625586404

17:38:42 Ports for container: e2e-cortex-test-ingester Mapping: map[80:33019 9095:33018]
17:38:43 Starting query-frontend
17:38:43 Ports for container: e2e-cortex-test-query-frontend Mapping: map[80:33021 9095:33020]
17:38:43 Starting query-frontend-sharding
17:38:44 Ports for container: e2e-cortex-test-query-frontend-sharding Mapping: map[80:33023 9095:33022]
    query_frontend_test.go:584: 
        	Error Trace:	/home/runner/work/cortex/cortex/integration/query_frontend_test.go:584
        	Error:      	Received unexpected error:
        	            	Get "http://127.0.0.1:33023/api/prom/api/v1/query_range?query=rate%28test%5B1m%5D%29&start=1704908324.2222755&end=1712684324.2222755&step=36000": read tcp 127.0.0.1:56040->127.0.0.1:33023: read: connection reset by peer
        	Test:       	TestQueryFrontendMaxQueryLengthLimits

We could add some backoff retries when sending query requests in the integration tests.

Expected behavior
Test should pass