microsoft / Kusto-Query-Language

Kusto Query Language is a simple and productive language for querying Big Data.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

False positives on KustoCode.ParseAndAnalyze (join subquery doesn't recognize bool fields of referenced table or function)

jkindwall opened this issue · comments

Our team stores a fairly large library of .kql files in source control that are used to ensure the schema of our main database is properly in sync with the rest of our code. We frequently run into deployment failures due to errors in these kql files that were not caught during code review and we don't find out about them until we are attempting to deploy the changes to our database.

I would like to use this library and specifically the KustoCode.ParseAndAnalyze method in a validation step as part of our PR builds to try and catch these types of errors before they are merged into our main branch. However, I'm currently running into far too many false positives in kql code that actually executes without issue against the actual database.

I am using this library: https://github.com/mattwar/Kusto.Toolkit to automatically generate the GlobalState object based on the current state of our test environment database, then running each command through the ParseAndAnalyze() method to detect errors.

One such error is that the subquery on a join doesn't recognize fields of the referenced table or function as bool.

| join kind=inner hint.strategy=shuffle hint.num_partitions=6 (vwMyView | where Enabled) on $left.Id == $right.Id

KS141 - The expression must have the type bool.
The vwMyView function returns tabular data where "Enabled" is a bool field, however the analyzer doesn't seem to recognize it as such.

This example works fine:

let v = view () { datatable(Id: long, Enabled: bool)[1, true, 2, false] };
datatable(Id: long)[1, 2]
| join kind=inner hint.strategy=shuffle hint.num_partitions=6 (v | where Enabled) on $left.Id == $right.Id

It is possible that the column 'Enabled' is not actually typed as bool.

I investigated this a little further and it seems your right. The problem is not with this library, it's with the Kusto Toolkit library that imports the schema from the existing database. The "Enabled" field IS a bool in our database, but I examined the TableSymbol for that table as imported by the KustoToolkit and it was typed as an int.