Giters

y-scope / clp

Compressed Log Processor (CLP) is a free log management tool capable of compressing text logs and searching the compressed logs without decompression.

Home Page:https://yscope.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Searching and writting data with an SQL query engine?

twoMatches-admin opened this issue · comments

commented

Request

I'm looking at CLP and would like to know if you have tried writing an SQL query engine or a query engine for any other declarative language that would make it easier to search over CLP.

Possible implementation

Implementing SQL over CLP using https://github.com/dolthub/go-mysql-server

commented

Hey @twoMatches-admin, that sounds interesting. Are you looking specifically for a way to use SQL syntax when querying CLP or a MySQL-compatible interface (such that you can use MySQL client libraries) to query CLP?

  • If it's the former, we have been looking into providing support for querying using the Kibana Query Language (KQL). Would that be something you're interested in or do you want SQL specifically?
  • If it's the latter, we hadn't considered that but the library you mentioned does sound promising.
  • Another area we have been dabbling with off and on is writing a connector for a system like Presto so that Presto users could query CLP.

Let me know which of those sounds most useful to you. On another note, do you have any example queries top of mind that you'd like to run atop CLP?

commented

A way to use SQL syntax and already available tools. Specifically SQL to use with in house analytics service that don’t support KQL.