Giters

microsoft / SqlScriptDOM

ScriptDOM/SqlDOM is a .NET library for parsing T-SQL statements and interacting with its abstract syntax tree

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Stack overflow when parsing script with thousands of UNIONs

zijchen opened this issue · comments

commented

TSqlParser results in a stack overflow exception when parsing large SQL files containing a lot of UNIONs (example: in a view or stored procedure). Culprit is likely the recursive visitor pattern, changing it to iterative may fix the issue.

Stack trace:

Microsoft.SqlServer.TransactSql.ScriptDom.TSqlFragmentVisitor.Visit(Microsoft.SqlServer.TransactSql.ScriptDom.BinaryQueryExpression)
Microsoft.SqlServer.TransactSql.ScriptDom.TSqlFragmentVisitor.ExplicitVisit(Microsoft.SqlServer.TransactSql.ScriptDom.BinaryQueryExpression)
Microsoft.SqlServer.TransactSql.ScriptDom.BinaryQueryExpression.Accept(Microsoft.SqlServer.TransactSql.ScriptDom.TSqlFragmentVisitor)
Microsoft.SqlServer.TransactSql.ScriptDom.BinaryQueryExpression.AcceptChildren(Microsoft.SqlServer.TransactSql.ScriptDom.TSqlFragmentVisitor)
Microsoft.SqlServer.TransactSql.ScriptDom.BinaryQueryExpression.Accept(Microsoft.SqlServer.TransactSql.ScriptDom.TSqlFragmentVisitor)
Microsoft.SqlServer.TransactSql.ScriptDom.BinaryQueryExpression.AcceptChildren(Microsoft.SqlServer.TransactSql.ScriptDom.TSqlFragmentVisitor)
commented

Some background on this:

This is a regression in Visual Studio that appeared between VS2019 and VS2022.

I raised a support case with Microsoft on 29th Nov 2021 and its been bouncing back and forth on email for 18 months without resolution before finally arriving as an issue logged here :-)

As an interim solution to allow our SQL Project to load in VS2022 without causing devenv.exe to crash we refactored part of our schema to reduce the number of UNIONS.

The schema has since continued to grow so it's only a matter of time before we hit the limit again.

If that happens then we currently have a couple of options:

1 - fall back to using VS2019 for just the SQL project
2 - build the sql dacpac in VS2022 using https://github.com/rr-wfm/MSBuild.Sdk.SqlProj/ (however this lacks a lot of the capabilities of .sqlproj files)

commented

@MarkWTW What capabilities are you missing? You are aware of the ability to have both project types sharing the same files in your solution?

commented

@MarkWTW What capabilities are you missing? You are aware of the ability to have both project types sharing the same files in your solution?

Hi yes - I've got all that set up and most things work very nicely - so much so that I use it to do build our dacpac in our pipeline, so that I can build the dacpac on linux build agents and reduce the build time by around 40% compared to .sqlproj.

It's a few issues with the VS integration that stop us moving over to it completely from .sqlproj - so those ones mentioned here https://github.com/rr-wfm/MSBuild.Sdk.SqlProj/#known-limitations

but really ease of use for our developers who like to just right-click publish in VS to deploy to local SQL server dev instance.

For our very complex sql project I couldn't get the dotnet publish to work (it errors out - see below)

Following your reply I've just updated to using 2.5.0 (we were using 2.3.0) and the MS published master package, instead of the nuget package that wraps it which I created in our private feed.

image

any suggestions on how we would troubleshoot that?

commented

@MarkWTW report an error in the repo and one of the maintainers will have a look.

commented

@MarkWTW also I recommend using sqlpackage for publish. It is very easy to install as a dotnet global tool these days.