cortexproject / cortex

A horizontally scalable, highly available, multi-tenant, long term Prometheus.

Home Page:https://cortexmetrics.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

TestStoreGatewayLazyExpandedPostingsSeriesFuzz failure

yeya24 opened this issue · comments

Describe the bug
https://github.com/cortexproject/cortex/actions/runs/8606208912/job/23584279307

TestStoreGatewayLazyExpandedPostingsSeriesFuzz test failed on current master after merging #5844.
However, there is no error in the fuzz test against Prometheus.

Example error log. res1 was the result from store gateway without lazy postings. I expect the results should only off on store gateway with lazy expanded postings.

    query_fuzz_test.go:454: case 20 results mismatch for the second attempt.
        {series="4", series!~"1.*", status_code!="not_exist_value", __name__=~"http_requ.*", __name__="http_requests_total", job="test"}
        res1 len: 0 data: []
        res2 len: 1 data: [{__name__="http_requests_total", job="test", series="4", status_code="502"}]

Need to investigate the issue and understand whether it is a bug from #5844 or a lazy expanded posting bug.

Expected behavior
Test should pass

Test passed after retry. It looks like the error was on the store gateway without lazy postings side. I wonder if there is something wrong on the test

Let's close it since it happened only once and it happened to Store Gateway without lazy expanded postings enabled.