Support filter log search result with user defined function

[Randy]

Which OpenObserve functionalities are relevant/related to the feature request?

log search, functions

Description

Our scenario

We plan to use openobserve in the process of ci/cd. And provide users a place to search and filter error logs. The logs usually are very big (10M+) . And the error message may appear in any position for parallel reason. So we need very custom function to filter the logs.

Proposed solution

Provide function filter in query time

Alternatives considered

plugin for query

[Prabhat Sharma]

Have you seen VRL functions? Could that do what you are looking for? https://openobserve.ai/docs/user-guide/functions/#example

[Hengfei Yang]

@prabhatsharma He wants to use VRL function to filter data in a SQL, like:

select * from table where vrl_md5('logs') = 'xxx'
select vrl_md5('logs') as md5, count(*) as cnt from table group by md5

[Prabhat Sharma]

I think @oasisk can help here for detailed info. He implemented it.

[Randy]

@prabhatsharma He wants to use VRL function to filter data in a SQL, like:

select * from table where vrl_md5('logs') = 'xxx'
select vrl_md5('logs') as md5, count(*) as cnt from table group by md5

yes, you got me