Analyzing Azure Monitor KQL query fails
mmaitre314 opened this issue · comments
Matthieu Maitre commented
Azure Monitor has workspace
and adx
keywords for cross-resource KQL queries which does not seem to be handled by Kusto-Query-Language
:
- Analysis succeeds:
SecurityAlert | extend ExtendedProperties = parse_json(ExtendedProperties)
- Analysis fails:
workspace('e7a8b8c9-5a5e-44a3-afc2-7885fc6f8b00').SecurityAlert | extend ExtendedProperties = parse_json(ExtendedProperties)
Error:
{
"Code": "KS107",
"Category": "General",
"Severity": "Error",
"Description": "A value of type 'string' expected.",
"Message": "A value of type 'string' expected.",
"HasLocation": true,
"Start": 105,
"Length": 18,
"End": 123
}
- Analysis fails:
adx('https://help.kusto.windows.net/Samples').SecurityAlert | extend ExtendedProperties = parse_json(ExtendedProperties)
Error:
{
"Code": "KS107",
"Category": "General",
"Severity": "Error",
"Description": "A value of type 'string' expected.",
"Message": "A value of type 'string' expected.",
"HasLocation": true,
"Start": 101,
"Length": 18,
"End": 119
}
Doc: https://docs.microsoft.com/en-us/azure/azure-monitor/logs/cross-workspace-query
Code:
var query = "workspace('e7a8b8c9-5a5e-44a3-afc2-7885fc6f8b00').SecurityAlert | extend ExtendedProperties = parse_json(ExtendedProperties)";
var globals = GlobalState.Default
.WithCluster(new ClusterSymbol("mycluster.kusto.windows.net"))
.WithDatabase(new DatabaseSymbol("MyDatabase"));
var parse = KustoCode.Parse(query, globals);
var parseErrors = parse.GetDiagnostics();
if (parseErrors.Count > 0)
{
throw new ApplicationException($"KQL query parsing error: {JsonSerializer.Serialize(parseErrors)}");
}
parse = parse.Analyze(globals);
var analysisErrors = parse.GetDiagnostics()
// Ignore expected errors
.Where(
d =>
d.Code != "KS204" && // "The name '{name}' does not refer to any known table, tabular variable or function."
d.Code != "KS142" && // "The name '{name}' does not refer to any known column, table, variable or function."
d.Code != "KS143" // "The function '{name}' is not defined."
).ToList();
if (analysisErrors.Count > 0)
{
throw new ApplicationException($"KQL query analysis error: {JsonSerializer.Serialize(analysisErrors)}");
}
Matthieu Maitre commented
Discussing offline, this needs to be handled through patterns. Ex:
declare pattern workspace = (workspaceId:string)[tableName:string]
{
('e7a8b8c9-5a5e-44a3-afc2-7885fc6f8b00').['SecurityAlert'] = { cluster('bar').database('foo').SecurityAlert };
};
workspace('e7a8b8c9-5a5e-44a3-afc2-7885fc6f8b00').SecurityAlert | extend ExtendedProperties = parse_json(ExtendedProperties)