All fields saved into _all field

[Hengfei Yang]

Which OpenObserve functionalities are the source of the bug?

streams

Is this a regression?

Yes

Description

All the fields saved into _all field after the user change data retention values.

Please provide a link to a minimal reproduction of the bug

No response

Please provide the exception or error you saw

The problem is the setting API playload:

{
    "partition_keys": [],
    "full_text_search_keys": [
        "f1"
    ],
    "bloom_filter_fields": [],
    "defined_schema_fields": [],
    "data_retention": 3
}

This will set the defined_schema_fields to empty array, the backend only check if it is a null, if it is an empty array, it will save everything into _all field.

Please provide the version you discovered this bug in (check about page for version information)

version: v0.10.1

Anything else?

No response

[Hengfei Yang]

Now, you can manually fix it by use curl:

curl 'http://localhost:5080/api/default/streams/default/settings?type=logs' \
  -X 'PUT' \
  -H 'Cookie: access_token=Basic cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=; auth_tokens={"access_token":"Basic cm9vdEBleGFtcGxlLmNvbTpDb21wbGV4cGFzcyMxMjM=","refresh_token":""}' \
  -H 'Content-Type: application/json' \
  --data-raw '{"partition_keys":[],"full_text_search_keys":["f1"],"bloom_filter_fields":[],"defined_schema_fields":null,"data_retention":7}'

The payload must have this "defined_schema_fields":null