ISKEYWORD ignores words in hints such as nolock, hash

[dmaloo]

• SqlPackage or DacFx Version:
• .NET Framework (Windows-only) or .NET Core: .NET Framework
• Environment (local platform and source/target platforms):
  Parser for 2019, Windows
  Steps to Reproduce:

1. Powershell script as below - does not recognize nolock , returns no for iskeyword.
   $ScriptData = @"
   --Testing tokens
   begin tran
   SELECT * FROM mytable (nolock);
   commit tran
   "@

Add-Type -Path "C:\ugpresentation\ScriptDom\Microsoft.SqlServer.TransactSql.ScriptDom.dll"
Write-Host "Attempting to parse..."
try {
$parser = New-Object Microsoft.SqlServer.TransactSql.ScriptDom.TSql150Parser($true)
#object to handle parsing errors if any
$parseErrors = New-Object System.Collections.Generic.List[Microsoft.SqlServer.TransactSql.ScriptDom.ParseError]
#object that handles strings script to parse
$stringReader = New-Object System.IO.StringReader($ScriptData)

#parser creates parsed objects from strings 
$parsedObjects = $parser.Parse($stringReader, [ref]$parseErrors)
$ctr = 0

 foreach ($token in $parsedobjects.ScriptTokenStream)
{
    write-host "Text: " $token.text -ForegroundColor Cyan
    write-host "Type: " $token.tokentype -ForegroundColor green
    write-host "Location: Line" $parsedobjects.ScriptTokenStream[$ctr].Line "Column " $parsedobjects.ScriptTokenStream[$ctr].column  -ForegroundColor yellow
    write-host "Is this a Reserved Word? " $token.IsKeyword()-ForegroundColor white
    $ctr++
} #token
if($parseErrors.Count -gt 0) {
    throw "$($parseErrors.Count) parsing error(s): $(($parseErrors | ConvertTo-Json))"
}
Write-Host "Complete!" -ForegroundColor Green

}
catch {
throw
}

Did this occur in prior versions? If not - which version(s) did it work in?
Works same on every version
(DacFx/SqlPackage/SSMS/Azure Data Studio)

[Arvind Shyamsundar]

@dmaloo - could you clarify the eventual goal? Is it to implement syntax highlighting similar to what SSMS / ADS do?

To explain why this does not work the way you want it to - you can see that the way IsKeyword is implemented, it relies on checking if the token belongs to one of these formal keywords. This is closely aligned with the published list of reserved keywords.

NOLOCK is technically a member of the TableHintKind enum enum, and technically not a keyword, though I can see why we may informally consider it to be one.

Similarly, a HASH JOIN hint is a member of the OptimizerHintKint enum, and not technically a keyword.

So depending on what your eventual goal is, we can think about how to handle this. But as of now, I don't consider this a bug - it's more an enhancement request.